Web-service with attachment failed on 2-ways SSL connection

I have two applications, one that provides a Web-service (let's call it ws-provider), the other one that is the client of this Web-Service (call it ws-client). This communication is protected through a 2-ways SSL. Both servers have been correctly configured regarding this confidentiality restrictions (certificates installation, SSL configuration, Tomcat parametrization...)

The servers are running on Tomcat (5.5 for ws-provider, 6 for ws-client), Java 6 and uses Jax-WS for providing/consuming the web-service.

When the user connects to the ws-client application and executes the action that calls the ws-provider web-service, the latter action generally fails with the following error:

com.sun.xml.ws.client.ClientTransportException: The server sent HTTP status code 400: No client certificate chain in this request
        at com.sun.xml.ws.transport.http.client.HttpClientTransport.checkResponseCode(HttpClientTransport.java:218)
        at com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:137)
        at com.sun.xml.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:74)
        at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:559)
        at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:518)
        at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:503)
        at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:400)
        at com.sun.xml.ws.client.Stub.process(Stub.java:234)
        at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:120)
        at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:230)
        at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:210)
        at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:103)
        at $Proxy33.createOrRenewRequest(Unknown Source)
        at my.app.MyPushRequest.sendXMLRequest(MyPushRequest.java:29)
        at my.app.MyRequestCreation.sendRequestForDraftApproval(MyRequestCreation.java:284)

On the ws-provider, I get this error:

WARNING: Exception getting SSL attributes
java.net.SocketException: SSL Cert handshake timeout
        at org.apache.tomcat.util.net.jsse.JSSE14Support.synchronousHandshake(JSSE14Support.java:101)
        at org.apache.tomcat.util.net.jsse.JSSE14Support.handShake(JSSE14Support.java:67)
        at org.apache.tomcat.util.net.jsse.JSSESupport.getPeerCertificateChain(JSSESupport.java:121)
        at org.apache.coyote.http11.Http11Processor.action(Http11Processor.java:1127)
        at org.apache.coyote.Request.action(Request.java:349)
        ...

Now, the strange part is that I just said that it generally failed when calling the web-service. For some reasons, sometimes I get a successfull attempt when calling the web-service. This generally happens if the user launches the action in the minute that follows his login to the application (No, I don't know why!)...

The web-service request contains an attachment. This attachment is a PDF file t开发者_开发技巧hat is not bigger than 15Kb (at least during all my tests, successfull or failed, this size was never exceeded).

So after many tests, I tried to call the same web-service without attaching any PDF file, and the call was successfull.

For your information, if the security constraints are disabled on the ws-provider (i.e. we do not use 2-ways SSL connexion anymore) then the web-service calls never fail.

So I imagine that I have to configure something on my web-services or on Tomcat (or ?) to make the system working. Any idea?

Java code

Here is the Java interface that provides the method call on ws-client (as you can see, the core Java classes used for the WS are generated by Jax-WS library):

/**
 * This class was generated by the JAX-WS RI.
 * JAX-WS RI 2.1.3-b02-
 * Generated source version: 2.1
 * 
 */
@WebService(name = "PushServicePortType", targetNamespace = "http://my.app.ws/PushService")
@XmlSeeAlso({
    ObjectFactory.class
})
public interface PushServicePortType {


    @WebMethod
    @WebResult(name = "response", targetNamespace = "")
    @RequestWrapper(localName = "createOrRenewRequest", targetNamespace = "http://my.app.ws/CwfPushService", className = "my.app.CreateOrRenewRequest")
    @ResponseWrapper(localName = "createOrRenewRequestResponse", targetNamespace = "http://my.app.ws/CwfPushService", className = "my.app.CreateOrRenewRequestResponse")
    public String createOrRenewRequest(
        @WebParam(name = "xmlMessageContent", targetNamespace = "")
        String xmlMessageContent,
        @WebParam(name = "version", targetNamespace = "")
        String version,
        @WebParam(name = "attachments", targetNamespace = "")
        List<DataHandler> attachments);

}

The call on ws-client is as follow (xmlFile contains the XML for the request, datahandler is a javax.activation.DataHandler that contains the PDF attachment):

PushServicePortType pushServicePort = new PushService(new URL("url/to/wsdl"), new QName("http://my.app.ws/PushService", "PushService")).getPushServiceSOAP(new MTOMFeature());
PushRequest push = new PushRequest();
responseXML = push.sendXMLRequest(pushServicePort, xmlFile, datahandler);

There are lots of configuration files, logs or Java classes. So do not hesitate to ask for more details if needed!

EDIT 1

I've tried to force the integration of the PDF file within the SOAP message, by defining the MTOM threshold to 2Mb:

On ws-client:

PushServicePortType pushServicePort = new PushService(new URL("url/to/wsdl"), new QName("http://my.app.ws/PushService", "PushService")).getPushServiceSOAP(new MTOMFeature(2097152));

On ws-provider, I configured the MTOM annotation with @MTOM(threshold=2097152)

This correction did not solve my issue, unfortunately...

Edit 2

I've removed the PDF attachment from my web-service all, and now it also fails but with a different message:

javax.xml.ws.soap.SOAPFaultException: Failed to read a response: javax.xml.bind.UnmarshalException
 - with linked exception:
[javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,3834]
Message: XML document structures must start and end within the same entity.]
        at com.sun.xml.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:173)
        at com.sun.xml.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:102)
        at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:240)
        at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:210)
        at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:103)
        at $Proxy33.createOrRenewRequest(Unknown Source)
        at my.app.CWFPushRequest.sendXMLRequest(PushRequest.java:29)
        ...
Caused by: com.sun.xml.ws.encoding.soap.DeserializationException: Failed to read a response: javax.xml.bind.UnmarshalException
 - with linked exception:
[javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,3834]
Message: XML document structures must start and end within the same entity.]
        at com.sun.xml.ws.server.sei.EndpointMethodHandler.invoke(EndpointMethodHandler.java:235)
        at com.sun.xml.ws.server.sei.SEIInvokerTube.processRequest(SEIInvokerTube.java:74)
        at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:559)
        at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:518)
        at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:503)
        at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:400)
        at com.sun.xml.ws.server.WSEndpointImpl$2.process(WSEndpointImpl.java:229)
        at com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter.java:430)
        at com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:230)
        at com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapter.java:121)
        at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.doGet(WSServletDelegate.java:115)
        at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.doPost(WSServletDelegate.java:146)
        at com.sun.xml.ws.transport.http.servlet.WSSpringServlet.doPost(WSSpringServlet.java:52)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at sun.reflect.GeneratedMethodAccessor330.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:244)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAsPrivileged(Subject.java:517)
        at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:276)
        at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:162)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:262)
        at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:52)
        at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:171)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:167)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:525)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
        at org.apache.catalina.valves.FastCommonAccessLogValve.invoke(FastCommonAccessLogValve.java:482)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
        at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
        at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
        at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
        ... 1 more
Caused by: javax.xml.bind.UnmarshalException
        at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.handleStreamException(UnmarshallerImpl.java:397)
        at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:335)
        at com.sun.xml.bind.v2.runtime.BridgeImpl.unmarshal(BridgeImpl.java:84)
        at com.sun.xml.bind.api.Bridge.unmarshal(Bridge.java:197)
        at com.sun.xml.ws.server.sei.EndpointArgumentsBuilder$DocLit.readRequest(EndpointArgumentsBuilder.java:492)
        at com.sun.xml.ws.server.sei.EndpointMethodHandler.invoke(EndpointMethodHandler.java:233)
        ... 41 more
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,3834]
Message: XML document structures must start and end within the same entity.
        at com.sun.xml.stream.XMLReaderImpl.next(XMLReaderImpl.java:563)
        at com.sun.xml.ws.encoding.MtomCodec$MtomXMLStreamReaderEx.next(MtomCodec.java:413)
        at com.sun.xml.bind.v2.runtime.unmarshaller.StAXStreamConnector.bridge(StAXStreamConnector.java:188)
        at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:333)
        ... 45 more

I've tried to log the SOAP messages sent by ws-client, but I don't see anything wrong in it...

I may have found a solution to my problem.

As explained, the ws-provider Tomcat is giving me the following error:

java.net.SocketException: SSL Cert handshake timeout
        at org.apache.tomcat.util.net.jsse.JSSE14Support.synchronousHandshake(JSSE14Support.java:101)
        at org.apache.tomcat.util.net.jsse.JSSE14Support.handShake(JSSE14Support.java:67)
        at org.apache.tomcat.util.net.jsse.JSSESupport.getPeerCertificateChain(JSSESupport.java:121)

So I had a look on the synchronousHandshake method (Tomcat 5.5.23):

private void synchronousHandshake(SSLSocket socket) throws IOException {
    InputStream in = socket.getInputStream();
    int oldTimeout = socket.getSoTimeout();
    socket.setSoTimeout(1000);
    byte[] b = new byte[0];
    listener.reset();
    socket.startHandshake();
    int maxTries = 60; // 60 * 1000 = example 1 minute time out
    for (int i = 0; i < maxTries; i++) {
        if (logger.isTraceEnabled())
            logger.trace("Reading for try #" + i);
        try {
            int x = in.read(b);
        } catch (SSLException sslex) {
            logger.info("SSL Error getting client Certs", sslex);
            throw sslex;
        } catch (IOException e) {
            // ignore - presumably the timeout
        }
        if (listener.completed) {
            break;
        }
    }
    socket.setSoTimeout(oldTimeout);
    if (listener.completed == false) {
        throw new SocketException("SSL Cert handshake timeout");
    }
}

As you can see, the timeout idea was not a real timeout problem, at least on my case. The error seems to happen because we read data from SSLSocket, and at the end, we do not consider that the handshake was completed correctly.

Another thing is that I got another problem when removing the PDF attachment in my WS call. This error said that the XML parsing failed. So I suspected that the data received on the ws-provider side was chunked, for some reasons.

Then, I finally found an interesting parameter in the Tomcat Connector, which is maxSavePostSize:

The maximum size in bytes of the POST which will be saved/buffered by the container during FORM or CLIENT-CERT authentication. For both types of authentication, the POST will be saved/buffered before the user is authenticated. For CLIENT-CERT authentication, the POST is buffered for the duration of the SSL handshake and the buffer emptied when the request is processed. For FORM authentication the POST is saved whilst the user is re-directed to the login form and is retained until the user successfully authenticates or the session associated with the authentication request expires. The limit can be disabled by setting this attribute to -1. Setting the attribute to zero will disable the saving of POST data during authentication. If not specified, this attribute is set to 4096 (4 kilobytes).

So I decided to modify the Tomcat Connector configuration to set the maxSavePostSize="-1" (i.e. the buffer is not limited to 4Kb anymore).

I am not 100% sure that this is the correct fix to apply in my situation, but all the tests I made today (with PDF attachment) were successfull, even the tests that always failed before...