开发者

Using cron + cURL to download files at night, network takes a few seconds to connect

I start a cronjob at 1:00 A.M开发者_高级运维. to use curl to check for updates on the web for some files, save needed updates to a text file, and then read the text file and download off of that. I then output the curl responce to a log file so I can see what's happening in the morning. It seems my connection doesn't start for about 4 seconds after cron calls my script. i.e. The first few lines of my log are:

1:00:01 A.M. curl: (6) cannot resolve host 'somehost.com/dir/file.zip'
# which is the error I see when I have no internet connection

and repeated a few times until about 4 seconds after 1:00 when the files start downloading and I see the normal curl output. Unfortunately this means I always miss about 7-8 files at the top of my update list that never get downloaded.

The only cause I can think of is that even though I allowed my network adapter to wake my computer, it takes ~4 seconds to connect to the internet after my computer wakes up; in those same 4 seconds my script is running down my text file, calling curl, and failing to download anything.

Is there anything in curl that deals with this?

I know I can write a wake up script or pass the error urls back through the script to download later, but possible infinite loops could come up if a site is down or I actually don't have internet service and it'd be nice if I could just use curl to keep trying to wake my network.


It could be caused by any number of things from a network perspective.

I would have the script test the response of curl to make sure the connection worked before moving down the update list.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜