开发者

Can I use Watir to scrape data from a website on a linux server without monitor?

Can I use Watir to sc开发者_StackOverflowrape data from a website (AJAX used) but on a linux server without monitor? (linode.com) ?


There are several ways to do this:

  1. Use HtmlUnit, either Celerity or watir-webdriver (through the remote Selenium2/WebDriver server).

  2. Use a real browser + a virtual X server (Xvfb). I'd recommend using watir-webdriver's Firefox driver and the Headless gem for a simple way to control this from Ruby.

This is basically a tradeoff between speed and realism. Personally I'd go with #2 if the site has any complex JavaScript or invalid HTML, but both approaches could be worth investigation.

For the future, I'm keeping an eye on this project, which looks like a terrific idea.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜