Simple keyword / key phrase analysis in Ruby

I would like to create a simple list of popular keywords or phrases within tweets containing a specific hashtag.

For example, for all tweets with the '#justinbieber' hashtag, I would like t开发者_运维技巧o obtain an ordered list of the top ten most popular words and/or phrases used in those tweets, disregarding the usual irrelevancies such as 'and', 'the', etc. It doesn't have to be perfect, just meaningful.

What Ruby tools are there available for performing text analysis? Of course, the analysis part does not have to be specific to Twitter.

I will most likely be periodically requesting and storing tweets with the given hashtag, and then applying analysis to tweets within a given time frame.

The work will be done within a Rails or Sinatra app on Heroku, but the analysis would be done in a rake task or scheduled job of some kind. I haven't decided how I'll be storing the tweets yet.

I was pretty satisfied with using OpenNLP through JRuby. For simple stuff like this, a simpler approach might also suffice though. Let's take a random tweet from the Twitter search for #justinbieber:

s = "If u never give up and if u fight for everything that u want, u can live our dreams. #JustinBieber"

Remove some unnecessary words:

words = s.split(/\W/).reject(&:empty?) - %w(the and u our if for that)
# => ["If", "never", "give", "up", "fight", "everything", "want", "can", "live", "dreams", "JustinBieber"]

Create counts:

words.each_with_object(Hash.new{ |h,k| h[k] = 0}) { |w, h| h[w] += 1 }
#=> {"If"=>1, "never"=>1, "give"=>1, "up"=>1, "fight"=>1, "everything"=>1, "want"=>1, "can"=>1, "live"=>1, "dreams"=>1, "JustinBieber"=>1}

If you do this for more than 1 tweet the counts will make more sense. Plus since you already have a Ruby hash, it's easy to just store that in e.g. a MongoDB collection.