开发者

Download files with Perl

I have updated my code to look like this. When I run it though it says it cannot find the specified link. Also what is a good way to test that it is indeed connecting to the page?

#!/usr/bin/perl -w
use strict;
use LWP;
use WWW::Mechanize;

my $mech = WWW::Mechanize->new();
my $browser = LWP::UserAgent->new;

$browser->credentials(
    'Apache/2.2.3 (CentOS):80',
    'datawww2.wxc.com',
    '************' => '*************'
);

my $response = $browser->get(
'http://datawww2.wxc.com/kml/echo/MESH_Max_180min/'
);

$mech->follow_link( n => 8);

(Original Post)

What is the best way to download small files with Perl?

I looked on CPAN and found lwp-download, but it seems to only download from the link. I have a page with links that change every thirty minutes with the date and time in the name so they are never the same. Is there a built-in function I can use? Everyone on Google keeps saying to use Wget, but I was kind of wanting to stick with Perl if possible just to help me learn it better while I program with it.

Also there is a user name and password to log into the site. I know how to access the 开发者_C百科site using Perl still, but I thought that might change what I can use to download with.


As stated in a comment in your other question: here

You can use the same method to retrieve .csv files as .html, or any other text-based file for the matter.

#!/usr/bin/perl -w
use strict;
use LWP::Simple;

my $csv = get("http://www.spc.noaa.gov/climo/reports/last3hours_hail.csv")
           or die "Could not fetch NWS CSV page.";

To login, you may need to use WWW::Mechanize to fill out the webform (look at $mech->get(), $mech->submit_form(), and $mech->follow_link())


Basically, you need to fetch the page, parse it to get the URL, and then download the file.

Personally, I'd use HTML::TreeBuilder::XPath, write a quick XPath expression to go straight to the correct href attribute node, and then plug that into LWP.

use HTML::TreeBuilder::XPath;
my $tree = HTML::TreeBuilder::XPath->new;
$tree->parse({put page content here});
foreach($tree->findnodes({put xpath expression here}){
    {download the file}
}
0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜