Prevent images from downloading with ScrAPI
I need to scrape some websites, and would like to avoid downloading images from the pages I am scraping - I only need the text. I am hoping this will speed up the process. Any ideas on how to manage this?
Thanks开发者_运维知识库, Jon
While scraping you do not download images but the reference IMG
tag along with the entire body
. You can always remove the IMG
tag on the server side before storing into your database/rendering to the view. I would suggest you use nokogiri to parse the content received and remove all occurrences of the IMG
tag.
This however does not speed up the process. Its just plain old html
that is scraped. If you want fast fetching and parsing go for Feedzirra
if you are dealing with feeds or Typhoeus
for fetching just the html content.
精彩评论