setRequestProperty (user-agent) not active until after "a while"

I'm writing an HTTP client, which needs to parse the response from a webserver, and I have run into (another) problem.

I found that for one page I was redirected to their mobile content portal: example: www.example.com/m/public. This is not what I want.

Whe开发者_如何学运维n using a "normal" browser, this redirect did not take place.

After looking into the capture I made, I found that this could be because my user-agent is interpreted as that of a mobile handset browser (user agent was "Java/1.6.0_22").

So I changed the user agent, using this:

URL url = new URL(endpoint);
URLConnection conn = url.openConnection();
conn.setRequestProperty ( "User-agent", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 3.5.30729; InfoPath.1; .NET CLR 3.0.30618)");

To my surpise it still did not work, and I found that I was still sending user-agent "Java/1.6.0_22".

Then I looked a bit closer at my capture, and I saw that after a couple of GET requests (after the first GET I send GETs to the sources on the main page) the user-agent magically changed from java to "Mozilla...".

It seems my setRequestProperty does not become active until after a while...

Has anyone seen this? Any way to get around it?

Thanks!

This SO answer suggests setting the system property before-hand.

I had the same problem. I wrote a web crawler and web pages grabbed were mobile versions. Now I used both

System.setProperty("http.agent", "");
urlconn.setRequestProperty("User-Agent", "IE/9.0");

and it worked.