HttpGet - weird encoding/characters

I have a following class

public class MyHttpClient {
private static HttpClient httpClient = null;

public static HttpClient getHttpClient() {
    if (httpClient == null)
        httpClient = new DefaultHttpClient();

    return httpClient;
}

public static String HttpGetRequest(String url) throws IOException {
    HttpGet request = new HttpGet(url);
    HttpResponse response = null;
    InputStream stream = null;

    String result = "";
    try {
        response = getHttpClient().execute(request);
        if (response.getStatusLine().getStatusCode() != 200)
            response = null;
        else
            stream = response.getEntity().getContent();

            String line = "";
            StringBuilder total = new StringBuilder();
            BufferedReader rd = new BufferedReader(new InputStreamReader(stream));
            while ((line = rd.readLine()) != null) { 
                total.append(line); 
            }

            // Return full string
            result = total.toString();
    } catch (ClientProtocolException e) {
        开发者_开发问答response = null;
        stream = null;
        result = null;
    } catch (IllegalStateException e) {
        response = null;
        stream = null;
        result = null;
    }

    return result;
}
}

and a web-service which response header is (I can't provide direct link because of privacy)

Status: HTTP/1.1 200

OK Cache-Control: private

Content-Type: application/json;

charset=utf-8

Content-Encoding: gzip

Server: Microsoft-IIS/7.5

X-AspNetMvc-Version: 3.0

X-AspNet-Version: 4.0.30319

X-Powered-By: ASP.NET

Date: Sun, 03

Jul 2011 11:00:43 GMT

Connection: close

Content-Length: 8134

In the end I get as a result series of weird, unreadable characters (I should get regular JSON like I get in regular desktop browser).

Where is the problem? (The same code for ex. google.com works perfectly and I get nice result)

EDIT: Solution (see below for description) Replace

HttpGet request = new HttpGet(url);

with

HttpUriRequest request = new HttpGet(url);
request.addHeader("Accept-Encoding", "gzip");

and replace

stream = response.getEntity().getContent();

with

stream = response.getEntity().getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
    stream = new GZIPInputStream(stream);
}

The problem is here:

Content-Encoding: gzip

This means the weird characters you are getting are the gzip compressed version of the expected JSON. Your browser does the decmpression so there you see the decoded result. You should have a look at your request header and at the configuration of your server.

Well, gzip encoding is generally good practice - for JSON data (especially big one) it can actually get between 10x and 20x decrease in the amount of data to transfer (which is a GOOD THING).. So better is to let HttpClient to handle GZIP compression nicely. For example here:

http://forrst.com/posts/Enabling_GZip_compression_with_HttpClient-u0X

BTW. It seems wrong however on the server side to provide GZIP compressed data when the client does not say "Accept-Encoding: gzip", which seems to be the case... So some things have to be corrected on the server as well. The example above adds Accept-Encoding header for you.