epoll architecture problem with bottle neck issue

while(m_severRun){

    printf("ServerManager::eventAcceptLoop, epoll_wait\n");
    int event_cnt = epoll_wait(m_epfd, m_events, EPOLL_SIZE, -1);
    if(event_cnt == -1){
        perror("epoll_wait error \n");
        break;
    }

    for(int i=0; i<event_cnt; i++){

        SocketClient *conn = reinterpret_cast<SocketClient *>(m_events[i].data.ptr);


        if(conn->getFd() == m_serverSocket->getFd()){

            printf("ServerManager::eventAcceptLoop, A Client has been connected \n");

            struct sockaddr_in clnt_adr;
            socklen_t adr_sz = sizeof(clnt_adr);
            int clnt_sock = accept(m_serverSocket->getFd(), (struct sockaddr*)&clnt_adr, &adr_sz);

            SocketClient* client = new SocketClient(clnt_sock);
            if(!addClient(client))
                break;
        }
        else{

            if(m_events[i].events & EPOLLRDHUP){
                printf("ServerManager::eventAcceptLoop, EPOLLRDHUP \n");
                removeClient(conn);
                close(conn->getFd());
                continue;
            }

            if(m_events[i].events & EPOLLIN){

                printf("ServerManager::eventAcceptLoop, EPOLLIN \n");
                int recv = conn->recv();

                if(recv <= 0){
                    removeClient(conn);
                    close(conn->getFd());
                }
                else{
                    printf("ServerManager::eventAcceptLoop, A message has been received \n");
                    vector<char> data = conn->getData();
                    addWork(conn, data);
                }
            }

            if(m_events[i].events & EPOLLERR)
                printf("ServerManager::eventAcceptLoop, EPOLLERR \n");
        }
    }//for loop end
}//while loop end

I am working on a network programming(tcp) and I have this code. This is my first time using epoll so I am not sure this design is correct. Also, I am using a thread pool (5 child threads) and whenever I read data from epoll I put that on the queue in the thread pool. The problem is that in the read function I can see the bottle neck problem.

In the read function, it calls ObserveSocket

int SocketClient::ObserveSock(int sock, int timeout){

    printf("SocketClient::ObserveSock called\n");

    fd_set reads;
    int fd_max;
    struct timeval _timeout;

    FD_ZERO(&reads);
    FD_SET(sock, &reads);
    fd_max = sock + 1;

    _timeout.tv_sec = timeout;
    _timeout.tv_usec = 0;

    return select(fd_max, &reads, 0, 0, &_timeout);
}

It watches the socket and if there is no signal for a certain amount of time then it returns 0 to close the socket. I thought I need this code to detect unexpected user di开发者_Go百科sconnection or data corruption (client sent 100 bytes but server received 90 bytes then the server will wait for the last 10 bytes which won't arrive).

I will be very appreciated if you guys can tell me how I can fix the bottle neck problem and any architecture issues on this.

I will also look for any good tutorial that using epoll and covers exception handling in detail.

Thanks in advance.

EDIT

Inside of recv() function it just calls read function and before I read I call the ObserveSocket

It watches the socket and if there is no signal for a certain amount of time then it returns 0 to close the socket.

Why?

I thought I need this code to detectever unexpected user disconnection or data corruption (client sent 100 bytes but server received 90 bytes then the server will wait for the last 10 bytes which won't arrive).

You don't. You will get another read event where the read will return 0 indicating EOS, or an error event.

If you want to implement an inactivity timeout you have to implement it in the main select loop. That is, keep track of the last activity time for each socket, and if it gets too long close the socket or do whatever you have to do. Put this testing at the bottom of the select() loop, before you iterate and call select() again.

At the moment every read event is blocking all other select() events for the timeout duration. So your entire server is blocked.