IO Completion Ports and OpenSSL

I have some legacy code that uses OpenSSL for communication. Just like any other session it does a handshake using the SSL functions and then encrypted communication over TCP. We recently changed our code to use IO completion ports. The way it works is contrary to that of OpenSSL. Basically, I'm having a hard time migrating our secure communication code from full OpenSSL usage to IOCP sockets and OpenSSL encrypti开发者_运维知识库on.

Does anyone have/anyone know of any references that might help me with such a task? How would TLS handshaking work over IOCP?

In order to use OpenSSL for encryption, but do your own socket IO, what you basically do is create a memory BIO, that you read and write socket data into as that becomes available, and attach that to the SSL context.

Each time you do a SSL_write call, you follow up with a call to the memory BIO to see if it has data in its read buffer, read that out and send it. Conversely, when data arrives on the socket via your io completion port mechanism, you write it to the BIO and call SSL_read to read the data out. SSL_read might return an error code indicating its in a handshake, which usually means its generated more data to write - which you handle by reading the memory BIO again.

To create my SSL session, I do this:

// This creates a SSL session, and an in, and an out, memory bio and
// attaches them to the ssl session.
SSL* conn = SSL_new(ctx);
BIO* bioIn = BIO_new(BIO_s_mem());
BIO* bioOut = BIO_new(BIO_s_mem());
SSL_set_bio(conn,bioIn,bioOut);
// This tells the ssl session to start the negotiation.
SSL_set_connect_state(conn);

As I receive data from the network layer:

// buf contains len bytes read from the socket.
BIO_write(bioIn,buf,len);
SendPendingHandshakeData();
TryResendBufferedData(); // see below
int cbPlainText;
while( cbPlainText = SSL_read(ssl,&plaintext,sizeof(plaintext)) >0)
{
  // Send the decoded data to the application
  ProcessPlaintext(plaintext,cbPlaintext);
}

As I receive data from the application to send - you need to be prepared for SSL_write to fail because a handshake is in progress, in which case you buffer the data, and try and send it again in the future after receiving some data.

if( SSL_write(conn,buf,len) < 0)
{
  StoreDataForSendingLater(buf,len);
}
SendPendingHandshakeData();

And SendPendingHandshakeData sends any data (handshake or ciphertext) that SSL needs to send.

while(cbPending = BIO_ctrl_pending(bioOut))
{
  int len = BIO_read(bioOut,buf,sizeof(buf));
  SendDataViaSocket(buf,len); // you fill this in here.
}

Thats the process in a nutshell. The code samples arn't complete as I had to extract them from a much larger library, but I believe they are sufficient to get one started with this use of SSL. In real code, when SSL_read/write / BIO_read/write fail, its probably better to call SSL_get_error and decide what to do based on the result: SSL_ERROR_WANT_READ is the important one and means that you could not SSL_write any more data, as it needs you to read and send the pending data in the bioOut BIO first.

You should look into Boost.Asio