开发者

How to fetch multiple feeds concurrently

I am new to ruby on rails and I have just started watching rails casts tutorials.

To parse feeds, I have started using feed zirra.

To fetch multiple feeds at once, feedzirra has this featur开发者_如何学Pythone

feed_urls = ["http://feeds.feedburner.com/PaulDixExplainsNothing",
"http://feeds.feedburner.com/trottercashion"]
feeds = Feedzirra::Feed.fetch_and_parse(feed_urls)

If I have 100 feeds, this procedure takes some time to index the 100th feed so,

How to parse fetch all these lets say 100 feeds concurrently?

Looking forward for your help and support


You can try to combine Feedzirra for feed parsing and Typhoeus for concurency, something like:

#!/usr/bin/env ruby
require 'rubygems'
require 'typhoeus'
require 'feedzirra'

feed_urls = ["http://feeds.feedburner.com/PaulDixExplainsNothing", "http://feeds.feedburner.com/trottercashion"]

hydra = Typhoeus::Hydra.new
feeds = {}

feed_urls.each do |feed|
        r = Typhoeus::Request.new(feed)
        r.on_complete do |response|
                feeds[r.url] = Feedzirra::Feed.parse(response.body)
        end
        hydra.queue r
end
hydra.run
puts feeds.inspect

The idea is to use hydra to be concurent. Up to you then to do something else than inspect or populating the feeds variable.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜