Unable to get Connection when doing multiple calls with jersey ApacheHttpClient

I'm having trouble with not getting connections when calling a rest api with com.sun.jersey.client.apache.ApacheHttpClient and Spring. A couple of calls get's through but suddenly I get a:

DEBUG (org.apache.commons.httpclient.MultiThreadedHttpConnectionManager:501) : - Unable to get a connection, waiting..., hostConfig=HostConfiguration[host=www.dummyapi.com]`

thats it. Nothing more!

This is my applicationContext.xml:

<bean id="customHttpClient" class="com.sun.jersey.client.apache.ApacheHttpClient">
    <constructor-arg>
        <bean class="com.sun.jersey.client.apache.ApacheHttpClientHandler" >
            <constructor-arg>
                <bean class="org.apache.commons.httpclient.HttpClient" >
                    <constructor-arg>
                        <bean class="org.apache.commons.httpclient.MultiThreadedHttpConnectionManager" >
                            <property name="params">
   开发者_开发问答                             <bean class="org.apache.commons.httpclient.params.HttpConnectionManagerParams">
                                    <property name="defaultMaxConnectionsPerHost" value="10" />
                                </bean>
                            </property>
                        </bean>
                    </constructor-arg>
                </bean>
            </constructor-arg>
        </bean>
    </constructor-arg>
    <constructor-arg ref="customClientConfiguration" />
</bean>

<bean id="customRestClient" class="com.custom.web.service.common.RestClient" >
    <constructor-arg ref="customHttpClient" />
    <constructor-arg value="${my.service.url}" />
</bean>

These are my classes:

import com.sun.jersey.client.apache.config.DefaultApacheHttpClientConfig;
import org.codehaus.jackson.jaxrs.JacksonJsonProvider;

public class CustomApacheHttpClientConfig extends DefaultApacheHttpClientConfig {

/**
 * Constructor with username and password.
 * @param username
 * @param password
 */
public CustomApacheHttpClientConfig(final String username, final String password) {
    getClasses().add(JacksonJsonProvider.class);
    getProperties().put(PROPERTY_PREEMPTIVE_AUTHENTICATION, Boolean.TRUE);
    getState().setCredentials(null, null, -1, username, password);
}
}

package com.custom.web.service.common;

import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import org.apache.log4j.Logger;
import org.codehaus.jackson.map.DeserializationConfig;
import org.codehaus.jackson.map.ObjectMapper;

import javax.ws.rs.core.MediaType;
import java.io.IOException;
import java.net.URI;

public class RestClient {

private static final Logger LOG = Logger.getLogger(RestClient.class);

private final String baseUrl;
private final Client client;
private final ObjectMapper objectMapper;

public RestClient(final String baseUrl, final Client client) {
    this.baseUrl = baseUrl;
    this.client = client;
    this.objectMapper = new ObjectMapper();
    this.objectMapper.getDeserializationConfig().set(DeserializationConfig.Feature.FAIL_ON_UNKNOWN_PROPERTIES, false);
}

public RestClient(final String baseUrl, final Client client, final ObjectMapper objectMapper) {
    this.baseUrl = baseUrl;
    this.client = client;
    this.objectMapper = objectMapper;
}

public String getBaseUrl() {
    return baseUrl;
}

public Client getClient() {
    return client;
}

public ObjectMapper getObjectMapper() {
    return objectMapper;
}
}

The call:

public void do() {
UriBuilder uriBuilder = UriBuilder.fromUri(
        String.format(
                this.customRestClient.getBaseUrl(),
               "SE")).
            path(SOME_UNIT_PATH_PARAM).
            path(id).
            path(SOME_OTHER_PATH_PARAM);

ClientResponse response = this.customRestClient.getClient().resource(uriBuilder.build()).get(ClientResponse.class);

    if (response.getClientResponseStatus().getStatusCode() != 200) {
        throw new CustomRestClientException(String.format("Something went wrong when getting stuff from rest api for id: %s %d %s ",
                id, response.getClientResponseStatus().getStatusCode(),
                " reasonPhrase: " + response.getClientResponseStatus().getReasonPhrase()));
    }

    String json = response.getEntity(String.class);

    Stuff[] stuff = null;

    try {
        stuff = this.customRestClient.getObjectMapper().readValue(json, Stuff[].class);
    } catch (IOException e) {
        throw new CustomRestClientException(
                String.format("Something went wrong when reading json value for stuff from api: %s %s %d %s ",
                        id, json, response.getClientResponseStatus().getStatusCode(), " reasonPhrase: " + e.getMessage()));
    }

    List<Stuff> stuffList = new ArrayList<Stuff>();
    stuffList.addAll(Arrays.asList(stuff));

}

As you can see I do not explicitly close the connection after each request, I leave that to the ApacheHttpClientHandler. I tried to increase defaultMaxConnectionsPerHost but no luck there. I have also browsed around the web and only found examples where people instantiate their client in code and then destroys it there as well. Please help me figure this out because I have hit a brick wall on this.

With regards Jakob!

Found the reason for my problems:

It seems that I forgot to set the soTimeout and the connectionTimeout. The documentation clearly states:
soTimout:
Defines the default socket timeout (SO_TIMEOUT) in milliseconds which is the timeout for waiting for data. A timeout value of zero is interpreted as an infinite timeout.
and
connectionTimout:
Determines the timeout until a connection is etablished. A value of zero means the timeout is not used. The default value is zero.

So all i had to do is to define this:

<bean id="customHttpClient" class="com.sun.jersey.client.apache.ApacheHttpClient">
        <constructor-arg>
            <bean class="com.sun.jersey.client.apache.ApacheHttpClientHandler" >
                <constructor-arg>
                    <bean class="org.apache.commons.httpclient.HttpClient" >
                        <constructor-arg>
                            <bean class="org.apache.commons.httpclient.MultiThreadedHttpConnectionManager" >
                                <property name="params">
                                    <bean class="org.apache.commons.httpclient.params.HttpConnectionManagerParams">
                                        <property name="defaultMaxConnectionsPerHost" value="30" />
                                        <property name="maxTotalConnections" value="30" />
                                        <property name="soTimeout" value="60000" />
                                        <property name="connectionTimeout" value="5000" />
                                    </bean>
                                </property>
                            </bean>
                        </constructor-arg>
                    </bean>
                </constructor-arg>
            </bean>
        </constructor-arg>
        <constructor-arg ref="customClientConfiguration" />
    </bean>

Voila! Everything works like a charm! Thank you skaffman you pointed me towards the right direction!

You still need to close your connections, the connection manager does not do that for you. The manager is only responsible for pooling the connections, it has no visibility of when you're finished using them.

If you don't close them, then the connection manager still thinks you're using them, and it'll quickly run out.