Ruby on Rails safari reader like text extraction and boilerplating

I have a digg like web service which briefly explained has a page parser and when people submit stories, the parser returns title and summary based on hpricot and some other small extraction principles I wrote.

I want to take it to the next level and try a content discovery and extraction from the pages people submit to the parser. I want to extract the content of an article for example.

Of course I don't really want to start from scra开发者_StackOverflow中文版tch and write my own boilerplate and extraction algorithms.

Is there a gem or something I can hook to my rails app that can be relevant to some degree to what I want to achieve?

I'm really stuck on this and any help is appreciated.

Best regards.

Hpricot is deprecated in ruby in favor of nokogiri. You can use any xpath expression with nokogiri for example

require 'rubygems'
require 'nokogiri'
require 'open-uri'

doc = Nokogiri::HTML(open('http://www.google.com/').read)
puts doc.xpath('//title')[0].text

Give the Readability gem a try. It's works very well:

https://github.com/cantino/ruby-readability

require 'rubygems'
require 'readability'
require 'open-uri'

source = open('http://lab.arc90.com/experiments/readability/').read
puts Readability::Document.new(source).content