Is there a good threading model for many simultaneous http requests?

My .Net and Java application needs to make many simultaneous requests to a web service which takes a notable length of time to reply. Consequently there may be thousands of outstanding HTTP requests at any one time. To implement this I could have a single thread generate many asynchronous HTTP requests and handle the responses but I want to understand any alternatives which allow synchronous HTTP requests to be made without requiring thousands of threads to be used because of the associated resource cost. Any solution must avoid threads being blocked waiting for HTTP responses.

My question has two parts:

1) Can Threadpools be used to do this? I know I can queue my jobs in the threadpool queue and the threadpool will pro开发者_运维问答cess them as and when it is able to but can the threadpool worker threads avoid being blocked waiting for the http response? Can they go and process other jobs while they wait?

2) Is there some other way I can achieve this while retaining the simplicity of the HTTP synchronous API call.

Regards

Can Threadpools be used to do this?

By themselves, no.

Is there some other way I can achieve this while retaining the simplicity of the HTTP synchronous API call.

Are you talking about the client side HTTP call API? If so, there is a way to do this, by using an asynchronous server-side implementation.

• The Servlet 3.0 specification supports asynchronous request processing: read this Javaworld article.
• Read the answers to Fast NIO, asynchronous HTTP Server for Java .

Tasks do not spawn extra threads, so you save that performance hit, and they do not block the calling thread so I believe this will meet both those needs.

string[] urls = GetUrls();
var requests = new Dictionary<string, Task<string>>()
var responses = new Dictionary<string, string>();

// send off each request and store the on-going task in a dictionary
foreach (string url in urls)
    requests.Add(url, Task.Factory.StarNew(() => return WebClient.DownloadString(url))

// get the results asynchronously
Task.Factory.StartNew(() => {
    foreach (var request in requests)
        responses.Add(request.key, request.Result)
 }

Now when you get the response you can do it a couple of ways. The above will get the result of each task one by one. When you call result it will wait for the task to finish, blocking the calling method and will throw exceptions at this point if the task faults. You can also wait the entire batch at once with Task.WaitAll(); Since calling .Result blocks the calling method you will have to retrieve the results from inside a task as well.

The example assumes your sending HTTP-GETs and want the result as a string, but regardless of the HTTP method or result you want, the way you make the call and get the result will be the same. You also want to add exception handling to catch any exceptions that may be thrown. You can ready about task exception handling here.