Multithreading issue with HttpClient

I'm running into a multithreading issue with HttpClient where I have the following scenario:

Thread A will issue url http://blap.com?param=2

Thread B will issue url http://blap.com?param=3

and this works about 98% of the time, but occasionally Thread A will receive the data for Thread B's url and vice-versa.

Now each thread is creating it's own HttpClient instance so I thought in theory I wouldn't need to use MultiThreadedHttpConnectionManager.

Does the behavior I'm describing seem plausible and will it be fixed by using MultiThreadedHttpConnectionManager?

I'm using java 1.6 and apache http client components 4.0.3.

Update: Here's the function in question.

public void get_url(String strDataSet) throws SQLException, MalformedURLException, IOException
{

      String query;



        query = "select * from jobs where data_set='" + strDataSet + "'开发者_运维问答";

        ResultSet rs2 = dbf.db_run_query(query);
        rs2.next();


        HttpClient httpclient = new DefaultHttpClient();
        HttpResponse response;



            String strURL;
            strURL = rs2.getString("url_static");

            if (rs2.getString("url_dynamic")!=null && !rs2.getString("url_dynamic").isEmpty())
                strURL = strURL.replace("${dynamic}", rs2.getString("url_dynamic"));

            UtilityFunctions.stdoutwriter.writeln("Retrieving URL: " + strURL,Logs.STATUS2,"DG25");

            if (!strURL.contains(":"))
                UtilityFunctions.stdoutwriter.writeln("WARNING: url is not preceeded with a protocol" + strURL,Logs.STATUS1,"DG25.5");

            //HttpGet chokes on the ^ character

            strURL = strURL.replace("^","%5E");


            HttpGet httpget = new HttpGet(strURL); 


            /*
             * The following line fixes an issue where a non-fatal error is displayed about an invalid cookie data format.
             * It turns out that some sites generate a warning with this code, and others without it.
             * I'm going to kludge this for now until I get more data on which urls throw the
             * warning and which don't.
             * 
             * warning with code: www.exchange-rates.org
             */


                if (!(strCurDataSet.contains("xrateorg") || strCurDataSet.contains("google") || strCurDataSet.contains("mwatch")))
                {
                    httpget.getParams().setParameter("http.protocol.cookie-datepatterns", 
                            Arrays.asList("EEE, dd MMM-yyyy-HH:mm:ss z", "EEE, dd MMM yyyy HH:mm:ss z"));
                }







            response = httpclient.execute(httpget);




         HttpEntity entity = response.getEntity();

          BufferedReader in = new BufferedReader(
                    new InputStreamReader(
                    entity.getContent()));



      int nTmp;         

      returned_content="";




      while ((nTmp = in.read()) != -1)
        returned_content = returned_content + (char)nTmp;


      in.close();

      httpclient.getConnectionManager().shutdown();

      UtilityFunctions.stdoutwriter.writeln("Done reading url contents",Logs.STATUS2,"DG26");



}

Update: I narrowed the problem down to the line:

response = httpclient.execute(httpget);

If I put a thread lock around that line, the problem went away. The thing is, that's the most time consuming piece and I don't want only one thread to be able to retrieve http data at a time.

Your code's not thread-safe. To fix your immediate problem, you need to declare your HttpClient as a ThreadLocal, but there's a lot more to fix.

You need to create a new HttpContext in each thread and pass it to HttpClient.execute:

HttpContext localContext = new BasicHttpClient();
response = httpclient.execute(httpget, localContext);

See the bottom of this doc (from HttpClient 4):

http://hc.apache.org/httpcomponents-client-ga/tutorial/html/statemgmt.html

Also there's a thread safe HttpContext implementation (SyncBasicHttpContext) but I'm not sure if you would need it in this case.