开发者

Prevent images from downloading with ScrAPI

I need to scrape some websites, and would like to avoid downloading images from the pages I am scraping - I only need the text. I am hoping this will speed up the process. Any ideas on how to manage this?

Thanks开发者_运维知识库, Jon


While scraping you do not download images but the reference IMG tag along with the entire body. You can always remove the IMG tag on the server side before storing into your database/rendering to the view. I would suggest you use nokogiri to parse the content received and remove all occurrences of the IMG tag.

This however does not speed up the process. Its just plain old html that is scraped. If you want fast fetching and parsing go for Feedzirra if you are dealing with feeds or Typhoeus for fetching just the html content.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜