MultiThread or Multi Lists?

as I seen topic which not recommending more than 200 threads for server machine,

I am trying to implement Listener class which listens 1000 devices, I mean 1000 devices sending different type of messages to that application,

I tried 2 different way 1. create thread for each device runtime and dynamic list which hold the messages for that device and start thread for processing those messages from the list

but my machine not creating thread more than 50 :), and I agree its bad idea...

2. I created 10 different lists which holds the messages 10 different type of messages and I created 10 processor thread for those list, which go to its relevant list and process the message and then delete it.

but here is the problem, let say I received 50 messages from 50 devices in List 1 by the time its list1's processor thread will go to last message (50th) its time will be expired which is 10 second

any idea to for best architecture that talk to more than 500 devices and process their different type of messages with in 10 seconds.

I am working in C#, my application connected with the server as a client using tcp/ip, that server further connects with online devices, sending messages to server with device id and message data and message typ thn further I receiving messages from that server and then reply back through that server using device id,

I think you need to partition the system differently. The listeners should be high priority but only enqueue the requests. The queue should then be processed using a pool of workers. You could add prioritisation and other optimisations on the dequeuing side. In terms of getting every process done in 10s you will really be getting the second half of the system optimised.

Think of the traditional queuing system. You have a queue of work requests to process. Each request has a series of attributes. Lets same Name (string) and Priority (int). Once a the work request has been queued, other worker (thread/processes etc) can interrogate the queue to pull out items based on priority and process them.

To get the 10s I'd say as soon a worker has started processing the request a timer comes in to play and will mark that request as timed out in 10s unless the worker completes the task. Other workers can watch for results of the work in the queue and then handle the response behaviours.

use other Highly-concurrent programming models other than threaded, though threaded is one of the highly-concurrent models too.

if socket/tcpip/network messaging, please use epoll on Linux 2.6x and completion port on win/msvc.

see the docoument named EffoNetMsg.pdf at http://code.google.com/p/effonetmsg/downloads/list to learn more about highly-concurrent programming models. we only use 2 or 3 threads for multi-listeners and >1000 clients.