Winsock: Overlapped AcceptEx indicates a new connection while no client connecting

In my program I am using the overlapped version of AcceptEx() to accept new connections. After a new connection has been accepted, the programm initiates another overlapped call to AcceptEx() for accepting more connections. This works fine and I can connect multiple clients to the server successfully.

But if I just connect one client and let the server-application call WSARecv (overlapped) on this socket, AcceptEx() magically accepts a new "ghost"-connection (There is the first client running doing nothing). When 开发者_如何学GoI call WSARecv on that, of course - it gives an error.

The program incorporates an I/O-Completion Port for all the overlapped calls.

I don´t know where the fake connection comes from. But that it seems to be a bug in my code that I am unable to find.

Things i can definetly exclude from being the errors reason: 1.The overlapped-structures i use and the parameter for casting works correct. 2. The IOCP-wrapper class.

Following is the relevant code (in my opinion) - if You need more, tell me, please :)

//schematic
main()
{
    Server.Init(...);
    Server.Start();         //Run-loop
}

CServer::Init(/*...*/)
{
    [...]


    //Create the listen socket...
    Ret = InitAcceptorSocket(strLocalAddress, strListenPort, nBacklog);
    if(Ret != Inc::INC_OK)
        return Ret;

    //...Associate it with the IOCP
    if(!m_pIOCP->AssociateHandle((HANDLE) m_pListenSocket->operator size_t(), 2))
        return Inc::INC_FATAL;

    [...]
}

CServer::InitAcceptorSocket(const std::wstring& strLocalAddress, const std::wstring& strListenPort, int nBacklog)
{
    //Create the socket
    m_pListenSocket.reset(new Inc::CSocket(AF_INET, SOCK_STREAM, IPPROTO_TCP));

    //Bind to specific port
    if(!m_pListenSocket->Bind(Inc::WStringToString(strLocalAddress), Inc::WStringToString(strListenPort)))      //Works as bind just calls getadrrinfo within itself

    //Put the socket into listen mode
    if(!m_pListenSocket->Listen(nBacklog))      //simple listen-wrapper: just calls the function and returns status indication
}

//Starts the server's work-cycle
CServer::Start(/**/)
{
    //Call accept
    DoCallAccept(m_pListenSocket.get());

    //Resume the threads
    //std::for_each(m_vecThreadHandles.begin(), m_vecThreadHandles.end(), [] (HANDLE hThread) {::ResumeThread(hThread);} );

    //TEST: Enter the Loop, too
    ServerMainWorkerThreadProc(this);

    return Inc::INC_OK;
}


//Worker thread proc
uintptr_t WINAPI ServerMainWorkerThreadProc(void* pvArgs)
{
    CServer* pServer = (CServer*)pvArgs;
    bool bLooping = true;

    try
    {
        while(bLooping)
        {
            bLooping = pServer->DoWork();
        };
    }
    catch(Inc::CException& e)
    {
        DebugBreak();
    }

    return 0;
}


bool CServer::DoWork()
{

    DWORD dwBytes = 0;
    ULONG_PTR ulKey = 0;
    OVERLAPPED* pOverlapped = nullptr;

    //Dequeue a completion packet
    if(!m_pIOCP->GetCompletionStatus(&dwBytes, &ulKey, &pOverlapped, INFINITE))
    {
        //error stuff
    }

    //Check for termination request:
    if(!dwBytes && !ulKey && !pOverlapped)
        return false;

    //Convert the Overlapped and check which work has to be done
    switch(((MYOVERLAPPED*)pOverlapped)->WorkType)
    {
    case WT_ACCEPT:                 //A new connection has been accepted
        HandleAcceptedConnection((WORK_ACCEPT*)pOverlapped);
        break;
    case WT_SEND:                   //Send data
        //HandleSendRequest((WORK_SEND*)pOverlapped);
        break;
    case WT_RECV:                   //Data has been received
        //HandleReceivedData((WORK_RECV*)pOverlapped);
        break;
    [...]

    return true;
}

    //New connection has been accepted
bool CServer::HandleAcceptedConnection(WORK_ACCEPT* pWork)
{
    //Create a new client socket object
    std::unique_ptr<Inc::CSocket> pSocket(new Inc::CSocket(pWork->SocketNewConnection));        //obtains the nescessary information (like AF_INET , etc by calls to getsockopt - works fine)

    //Associate with the IOCP
    if(!m_pIOCP->AssociateHandle((HANDLE)((SOCKET)(*(pSocket.get()))), 2))
    {
        //Report the error
    }

    //Queue a recv-packet
    if(!DoCallRecv(pSocket.get()))
    {
        //Report the error
    }

    //Release the client-socket-object
    pSocket.release();

    //Call accept another time
    DoCallAccept(pWork->pListenSocket);

    //Cleanuo
    delete pWork;

    return true;
}


//Call Recv on the socket
bool CServer::DoCallRecv(Inc::CSocket* pSocket)
{
    //Create the work object for receiving data
    std::unique_ptr<WORK_RECV> pWorkRecv(new WORK_RECV);
    memset((OVERLAPPED*)pWorkRecv.get(), 0, sizeof(OVERLAPPED));
    pWorkRecv->pSocket = pSocket;


    //Call Recv
    std::string strRecvBuffer;      //temporary receive buffer for immediate completion
    short sRet = pSocket->Recv(strRecvBuffer, pWorkRecv->pTestWSABuf, 2048, (OVERLAPPED*)pWorkRecv.get());
    [...]
    if(sRet == Inc::REMOTETRANSACTION_PENDING)
    {
        //release the work item so it is still on the heap when the overlapped operation completes
        pWorkRecv.release();
    }

    return true;
}

//Queue a call to accept
bool CServer::DoCallAccept(Inc::CSocket* pListenSocket)
{
    //Create the overlapped-structure
    std::unique_ptr<WORK_ACCEPT> pWork(new WORK_ACCEPT);
    memset((OVERLAPPED*)pWork.get(), 0, sizeof(OVERLAPPED));
    pWork->pListenSocket = pListenSocket;
    pWork->pSocket = m_pListenSocket.get();

    //Call accept
    pWork->SocketNewConnection = m_pListenSocket->Accept(nullptr, nullptr, (OVERLAPPED*)pWork.get());

    //Release the work object
    pWork.release();

    return true;
}


//The accept function for My custom socket-wrapper-class
SOCKET Inc::CSocket::Accept(sockaddr_storage* pAddr, int* pAddrLen, OVERLAPPED* pOverlapped)
{
    [...]
    else        //Overlapped
    {
        //create the client socket
        SOCKET ClientSock = socket(m_SocketAF, SOCK_STREAM, 0);
        if(ClientSock == INVALID_SOCKET)
            throw(Inc::CException(WSAGetLastError(), "Socket creation failed."));
        //address structure & size
        sockaddr_storage *ClientAddress = {0}; DWORD dwClientAddressSize = sizeof(sockaddr_storage);
        //output buffer
        //char acOutputBuffer[(2 * sizeof(sockaddr_storage)) + 32] = "";
        //received bytes
        DWORD dwBytes = 0;

        if(m_lpfnAcceptEx(m_Socket, ClientSock, (PVOID)m_acOutputBuffer, 0, (dwClientAddressSize + 16), (dwClientAddressSize + 16), &dwBytes, pOverlapped) == FALSE)
        {
            int nError = WSAGetLastError();
            if(nError != WSA_IO_PENDING)
                throw(Inc::CException(nError, "AcceptEx failed."));

            return ClientSock;
        }

        //if immidiately & successfully connected, get the client address
        [...]

        return ClientSock;
    }
}


//The receive function
short Inc::CSocket::RecvHelper(std::string& strIncomingDataBuffer, WSABUF*& pWSABuf, unsigned int nBytesToRecv, OVERLAPPED* pOverlapped)
{
    int iRet = 0;                   //ret code
    DWORD dwReceived = 0, dwFlags = 0;

    //Clear the Buffer
    strIncomingDataBuffer.clear();

    //create the receiving buffer
    std::unique_ptr<char[]> pcBuf(new char[nBytesToRecv]);
    //create the WSABUF
    std::unique_ptr<WSABUF> pWSABufBuf (new WSABUF);
    pWSABufBuf->len = nBytesToRecv;
    pWSABufBuf->buf = pcBuf.get();


    iRet = WSARecv(m_Socket, pWSABufBuf.get(), 1, pOverlapped ? NULL : (&dwReceived), &dwFlags, pOverlapped, NULL);
    if(iRet == 0)
    {
        //closed (gracefully) by the client (indicated by zero bytes returned)
        if(dwReceived == 0 && (!pOverlapped))
            return REMOTECONNECTION_CLOSED;     //return

        //successfull received
        strIncomingDataBuffer.assign(pWSABufBuf->buf, dwReceived);

        return SUCCESS;
    }
    if(iRet == SOCKET_ERROR)
    {
        int nError = WSAGetLastError();

        //Overlapped transaction initiated successfully
        //waiting for completion
        if(nError == WSA_IO_PENDING)
        {
            //release the buffers
            pcBuf.release();
            pWSABuf = pWSABufBuf.release();     //hand it over to the user

            return REMOTETRANSACTION_PENDING;   //return "transaction pending"-status
        }

        //forced closure(program forced to exit)
        if(nError == WSAECONNRESET)
        {
        [...]
}

EDIT: Wrote a test-server which works just fine

//Accept a new connection
        ACCEPTLAPPED* pOverAccept = new ACCEPTLAPPED;
        pOverAccept->pSockListen = &SockListen;
        pOverAccept->pSockClient = new Inc::CSocket(SockListen.Accept(nullptr, nullptr, pOverAccept));

        //Main loop
        DWORD dwBytes = 0, dwFlags = 0;
        ULONG_PTR ulKey = 0;
        OVERLAPPED* pOverlapped = nullptr;
        while(true)
        {
            dwBytes = 0; dwFlags = 0; ulKey = 0; pOverlapped = nullptr;

            //Dequeue a packet
            pIOCP->GetCompletionStatus(&dwBytes, &ulKey, &pOverlapped, INFINITE);

            switch(((BASELAPPED*)pOverlapped)->Type)
            {
            case 1:     //Accept
                {
                    //ASsociate handle
                    ACCEPTLAPPED* pOld = (ACCEPTLAPPED*)pOverlapped;
                    pIOCP->AssociateHandle((HANDLE)(pOld->pSockClient)->operator SOCKET(),2);
                    //call recv
                    RECVLAPPED* pRecvLapped = new RECVLAPPED;
                    pRecvLapped->pSockClient = pOld->pSockClient;
                    short sRet = (pRecvLapped->pSockClient)->Recv(pRecvLapped->strBuf, pRecvLapped->pBuf, 10, pRecvLapped);

                    //Call accept again
                    ACCEPTLAPPED* pNewAccLapp = new ACCEPTLAPPED;
                    pNewAccLapp->pSockListen = ((ACCEPTLAPPED*)pOverlapped)->pSockListen;
                    pNewAccLapp->pSockClient = new Inc::CSocket((pNewAccLapp->pSockListen)->Accept(nullptr, nullptr, pNewAccLapp));

                    delete pOverlapped;
                };
                break;
            case 2:     //Recv
                {
                    RECVLAPPED* pOld = (RECVLAPPED*)pOverlapped;
                    if(!pOverlapped->InternalHigh)
                    {
                        delete pOld->pSockClient;
                        Inc::CSocket::freewsabufpointer(&(pOld->pBuf));
                        delete pOld;
                        break;
                    };
                    cout << std::string(pOld->pBuf->buf, pOld->pBuf->len) <<endl;

I"ve been working with AcceptEx and IOCP, and I never-ever saw such a problem.

About your code. It's hard to say what's wrong exactly in it, since it's not complete. But I'm pretty sure the problem is there.

One problem that I see is that the 3rd parameter you supply to AcceptEx is a local buffer. This is wrong, because this buffer should remain valid for the duration of the accept operation. What you did may easily lead to the stack memory corruption of whatever.

But your "spurious accept" problem is probably caused by something else. And I suspect that I know what's the problem. Let me guess:

1. You use the same IOCP for both listening and accepted (client) socket. This is reasonable, there's no need to have more than 1 IOCP.
2. When you dequeue a completion from the IOCP - you automatically cast it to WORK_ACCEPT and call HandleAcceptedConnection. Don't you?

If so - the problem is obvious. You call the WSARecv on the client socket. It completes, and the completion is queued the the IOCP. You fetch it, however you treat it as a completed accept. You cast it to WORK_ACCEPT, which looks junky (simply becase it is not a WORK_ACCEPT structure).

If this is the case - you must add a way to distinguish different completion types. For instance you may declare a base struct (from which all the completions would inherit), which would have a type member, which will identify the completion type.

I expect you have a bug in your code but it's hard to say where as you don't show the actual calls to the underlying API; I've no idea what your wrapper code is doing...

You might find it useful to look at a working AcceptEx() based server. There's one available in my free IOCP server framework: http://www.serverframework.com/products---the-free-framework.html

Did clean up and rewrite the issued part of the code: Works now....

But the funny thing is: while comparing the two "codes" with each other... the the calls etc are still the same...