Scraping information from site using lxml

I'm trying to grab a list of all titles from the site Reddit.com using lxml. I used this query:

  reddit = etree.HTML( urllib.urlopen("http://www.reddit.com/r/all/top").read() )
  reddit.xpath("//div[contains(@class,'title')]//b/text()")

However, when I run the expression nothing comes up in the Python shell. Is the XPath incorrect?

Running with Python 2.7

Here's the full code:

import urllib
import os, random, sys, math
from lxml import etree

def main():

    reddit = etree.HTML( urllib.urlopen("http://www.reddit.com/r/all/top").read() )
    reddit.xpath("//div[contains(@class,'title')]//b/text()")



if __name__ == "__main__":
    main()开发者_如何学运维

Reddit has API. You don't need to scrape it. Just add '.json' at the end of the url:

#!/usr/bin/env python
import json
import urllib2

url = "http://www.reddit.com/r/all/top/.json"
data = json.load(urllib2.urlopen(url))
for child in data['data']['children']:
    print child['data']['title']

Example Output

Dear America, I Saw You Naked: And yes, we were laughing. Confessions of an ex-TSA agent
My wife and I are expecting our son in June, so I installed a fiber-optic star ceiling :)
You wouldn't download a car: Honda releases concept car 3D printing files
So my liquor store I managed closed today, the VP came in to collect the liquor but told me "we're not going to resell the beer, we'll be here about an hour fill up your car."
Baby Olinguito (Recently Discovered Species!)
Bower Bird- in a desperate bid for attention from the opposite sex, Bower males build nests, then decorate with objects of a single color. (xpost- /r/everythingscience)
My friend works as a English teacher in Sweden.
My kid's homework, I think the page designer has had enough.
Man Washes up in Marshall Islands 'After 16 Months Adrift' at sea
Kitten plays the air harp
New roommate already started off on a bad note with us.
MRW a program crashes and asks to contact tech support... and I am tech support.
Jack Black just posted this to facebook. "This is fan art. But it's exactly how I remember it."
Looks like Colorado's legalization has caused problems after all. [4]
My new kitten likes to "hold hands." She does this for as long as you offer your finger.
Ahahaha he got you go-wahhhh
Shipwrecked man makes land 'after 16 months adrift'
As someone who's taken math at university
Footage released of Guardian editors destroying Snowden hard drives: GCHQ technicians watched as journalists took angle grinders and drills to computers after weeks of tense negotiations
TIL Mike Tyson offered a zoo attendant $10,000 to open the cage of a bullying gorilla so he could "smash that silverback's snotbox." His offer was declined.
Microsoft being helpful as always
President Barack Obama says in a new interview that he would support efforts to remove marijuana from the federal government’s list of the most serious narcotics, but that Congress must act to make the change.
advisory
Vila Franca's Islet, Azores Archipelago, Portugal [1440x900] - How can that be so spherical?
The dad on my Child Development book is putting the kids helmet on backwards.

You were not connected to the internet. Try again.

AND/OR

Your Python installation is either trashed or you have mixed together two stacktraces ... note how the paths suddenly change from 3.1 to 2.7 !!!!!!!

Update

Nothing appears in the shell because you don't print anything.

At least if instead of reddit.xpath("blahblah") you do:

result = reddit.xpath("blahblah")
print result

you will see that your current version of "blahblah" produces [] and be in a good shape to notice if fiddling with "blahblah" improves the situation.