开发者

WWW::CURL::Multi module gives Write Error

I'm trying to download a bunch of files on the web in parallel with the Perl's WWW::Curl::Multi module but I'm receiving the following errors:

  • Use of uninitialized value in ref-to-glob cast at AA.pm line 17.

  • syswrite() on unopened filehandle at AA.pm line 17.

  • Use of uninitialized value in

    subroutine entry at AA.pm line 60.

I'll appreciate if someone can figure out why those errors come from.

package AA;

use strict;
use warnings;
use WWW::Curl::Easy;
use WWW::Curl::Multi;
use Data::Dumper;

sub new {
 my $class = shift;
 my $self = {};
 bless $self, $class;
 return $self;
}

sub tofile { 
 return syswrite($_[1], $_[0]); #This is where first and the second error occurs
}

sub downloadfiles{
 my $self = shift;
 my $files = shift;
 my %easy;
 my $curl = WWW::Curl::Easy->new;
 my $active_handles = 0;
 my $curlm = WWW::Curl::Multi->new;
 my $dir = "dl/"; 
 my $fh;
 foreach my $file (@$files) {
  my $curl_id = $active_handles + 1; # This should be a handle unique id.
  my $code = 0;
  $easy{$curl_id} = $curl;
  $code+= $curl->setopt(CURLOPT_PRIVATE, $curl_id);

  #Open the filehandle
  open($fh, ">$dir$curl_id") or die "\nopen: $!\n\n";
  binmode $fh;

  # do the usual configuration on the handle
  $code+= $curl->setopt(CURLOPT_FILE, *$fh);
  $code+= $curl->setopt(CURLOPT_FAILONERROR, 1);
  $code+= $curl->setopt(CURLOPT_HEADER, 1);
  $code+= $curl->setopt(CURLOPT_CONNECTTIMEOUT, 2);
  $code+= $curl->setopt(CURLOPT_URL, $file);
  $code+= $curl->setopt(CURLOPT_WRITEFUNCTION, \&tofile);
  $code+= $curl->setopt(CURLOPT_NOPROGRESS, 1);
  $code+= $curl->setopt(CURLOPT_VERBOSE, 0);
  $code+= $curl->setopt(CURLOPT_HEADER, 0开发者_JAVA技巧);

  if ($code ne 0) {
   die("Failed to initialize curl");
  }

  # Add some easy handles
  $curlm->add_handle($curl);
  $active_handles++;
 }
 print "\nActive handles: ".$active_handles."\n";
 while ($active_handles) {
  my $active_transfers = $curlm->perform; #This is where second error comes from
  if ($active_transfers != $active_handles) {
   while (my ($id, $return_value) = $curlm->info_read) {
    print $id;
    if ($id) {
     $active_handles--;
     my $actual_easy_handle = $easy{$id};
     delete $easy{$id};
    }
   }
  }
 }
 close $fh;
}

1;


I see you initialize a filehandle $fh to store the downloaded data, but I don't see how that value ever gets passed to your tofile function. Do you need to say

$code+= $curl->setopt(CURLOPT_FILE, $fh);   # or
$code+= $curl->setopt(CURLOPT_FILE, *$fh);

instead of

$code+= $curl->setopt(CURLOPT_FILE, *TBZ2);  # whatever TBZ2 is

?

What do you see if you put an extra line in tofile that prints out the inputs (@_) to that function?


I've realized that the problem was not about the script, it was rather about the old BSD machine(with the old version of perl) I was using.

You can test the code with the following lines:

my @rrds=qw(http://www.google.com http://www.stackoverflow.com http://www.reddit.com);
print Dumper(@rrds); 
my $fDl = AA->new;
$fDl->downloadfiles(\@rrds);

But there is another bug in that script after being able to run it, it outputs the contents to only the last file. Because we are changing the values of $fh. You can fix by defining $fh locally in the for loop and put it in a hash. When exiting the subroutine close the fh's in the hash.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜