Best way to create a hightraffic temporary database, in-memory or SQL?

i this project, that will be create a "NAT-T server"

It is a Server listing in some port, where clients send their ID, IP, Port and the server keep their last-contact time, for purge olders clients.

The ID is UNIQUE and is how the clients will be located.

The problem is that it have to handle 15k transactions per second. The database don´t need to be stored.

The average tracked clients should be 200k.

How could be the best way to implement it, with a SQL server or开发者_C百科 with a data-structure ? Witch data structure ?

So you need the ability to insert items, locate, update, and delete them by ID. You also need to purge clients that have not been accessed for some period of time. Let's say that the time is defined by a constant: OldClientRemovalInterval.

The client data structure would contain the ID, IP, Port, and last access time.

Here's one way to do it:

Maintain an associative array of client records, with the key being the ID. The C++ Standard Template Library has several different associative array data structures (map, hash_map, etc.). A hash_map gives very fast access time. You can insert, update, or remove items very quickly.

For the removal, maintain a FIFO queue that contains the client IDs and expected removal times. Whenever a client record is inserted or updated, create a second structure { ClientID, RemovalTime }, with the RemovalTime being computed by adding the OldClientRemovalInterval to the current time.

Items in the queue are ordered by increasing removal time. Periodically check the queue, removing any items whose removal time is expired.

Note, however, that a client might have been updated since it was added to the queue. So when you take something out of the queue, you have to look up that client in the associative array and determine if it really needs to be removed. Something like:

ClientID = RemoveItemFromQueue();
Client = GetClientRecord(ClientID);
ClientRemovalTime = Client.LastAccess + ClientRemovalInterval;
if (ClientRemovalTime <= CurrentTime)
    RemoveClient(ClientId);

This will actually require that the data structure support 30K transactions per second (for each insert/update, there will be a corresponding lookup and possible removal). But a hash_map should handle that without trouble.

How you check the queue is up to you. You could do it on a timer or on a second thread, but then you have issues with locking access to the queue and the associative array. Perhaps a better idea is to have the removal algorithm run after every insert/update, removing any item whose time has expired. Or, to avoid spending too much time in the removal, have it remove up to two old clients for each insert/update. That is:

InsertOrUpdateClient();
// now remove up to 2 items
for (int i = 0; i < 2; ++i)
{
    if (queue.Peek().RemovalTime >= CurrentTime)
    {
        // Remove item from queue,
        // and potentially remove client from associative array.
    }
    else
    {
        break;
    }
}

That will avoid spending too much time in removal, but will, on average, keep the old clients from clogging the data structure. The only disadvantage is that no removal takes place if there are no inserts/updates. So after a long period of inactivity the table could be full of old clients. If that's important, you could have a timer that periodically sends a null request (have a special client ID of -1, for example) that won't update the associative array, but will do the removal.