Decoding response while opening a URL

I am using the following code to open a url and retrieve it's response :

def get_issue_report(query):
    request = urllib2.Request(query)
    response = urllib2.urlopen(request)
    response_headers = response.info()
    print response.read()

The response I get is as follows :

<?xml version='1.0' encoding='UTF-8'?><entry xmlns='http://www.w3.org/2005/Atom' xmlns:gd='http://schemas.google.com/g/2005' xmlns:issues='http://schemas.google.com/projecthosting/issues/2009' gd:etag='W/&quot;DUUFQH47eCl7ImA9WxBbFEg.&quot;'><id>http://code.google.com/feeds/issues/p/chromium/issues/full/2</id><published>2008-08-30T16:00:21.000Z</published><updated>2010-03-13T05:13:31.000Z</updated><title>Testing if chromium id works</title><content type='html'>&lt;b&gt;What steps will reproduce the problem?&lt;/b&gt;
&lt;b&gt;1.&lt;/b&gt;
&lt;b&gt;2.&lt;/b&gt;
&lt;b&gt;3.&lt;/b&gt;

&lt;b&gt;What is the expected output? What do you see instead?&lt;/b&gt;


&lt;b&gt;Please use labels and text to provide additional information.&lt;/b&gt;
 </content><link rel='replies' type='application/atom+xml' href='http://code.google.com/feeds/issues/p/chromium/issues/2/comments/full'/><link rel='alternate' type='text/html' href='http://code.google.com/p/chromium/issues/detail?id=2'/><link rel='self' type='application/atom+xml' href='https://code.google.com/feeds/issues/p/chromium/issues/full/2'/><author><name>rah...@google.com</name><uri>/u/@VBJVRVdXDhZCVgJ%2FF3tbUV5SAw%3D%3D/</uri></author><issues:closedDate>2008-08-30T20:48:43.000Z</issues:closedDate><issues:id>2</issues:id><issues:label>Type-Bug</issues:label><issues:label>Priority-Medium</issues:label><issues:owner><issues:uri>/u/kuchhal@chromium.org/</issues:uri><issues:username>kuchhal@chromium.org</issues:username></issues:owner><issues:stars>4</issu开发者_如何学JAVAes:stars><issues:state>closed</issues:state><issues:status>Invalid</issues:status></entry>

I would like to get rid of the characters like &lt, &gt etc. I tried using

response.read().decode('utf-8')

but this doesn't help much.

Just in case, the response.info() prints the following :

Content-Type: application/atom+xml; charset=UTF-8; type=entry
Expires: Fri, 01 Jul 2011 11:15:17 GMT
Date: Fri, 01 Jul 2011 11:15:17 GMT
Cache-Control: private, max-age=0, must-revalidate, no-transform
Vary: Accept, X-GData-Authorization, GData-Version
GData-Version: 1.0
ETag: W/"DUUFQH47eCl7ImA9WxBbFEg."
Last-Modified: Sat, 13 Mar 2010 05:13:31 GMT
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Connection: close

Here's the URL : https://code.google.com/feeds/issues/p/chromium/issues/full/2

Sentinel has explained how you can decode entity references like < but there's a bit more to the problem than that.

The example you give suggests that you are reading an Atom feed. If you want to do this reliably in Python, then I recommend using Mark Pilgrim's Universal Feed Parser.

Here's how one would read the feed in your example:

>>> import feedparser
>>> d = feedparser.parse('http://code.google.com/feeds/issues/p/chromium/issues/full/2')
>>> len(d.entries)
1
>>> print d.entries[0].title
Testing if chromium id works
>>> print d.entries[0].description
<b>What steps will reproduce the problem?</b>
<b>1.</b>
<b>2.</b>
<b>3.</b>

<b>What is the expected output? What do you see instead?</b>


<b>Please use labels and text to provide additional information.</b>

Using feedparser is likely to be much more reliable and convenient than trying to do your own XML parsing, entity decoding, date parsing, HTML sanitization, and so on.

from HTMLParser import HTMLParser
import urllib2


query="http://code.google.com/feeds/issues/p/chromium/issues/full/2"

def get_issue_report(query):
    request = urllib2.Request(query)
    response = urllib2.urlopen(request)
    response_headers = response.info()
    return response.read()

s = get_issue_report(query)

p = HTMLParser()

print p.unescape(s)

p.close()

Use

xml.sax.saxutils.unescape()

http://docs.python.org/library/xml.sax.utils.html#module-xml.sax.saxutils