开发者

Executing web-service calls in parallel

I saw these questions Recommendations for executing .NET HttpWebRequests in parallel in ASP.NET and Async.Parallel or Array.Parallel.Map from about a year ago and was wondering if the advice from them still stands or if there's now a better approa开发者_开发知识库ch with Task Parallel Library.

My case is closer to the second question, except I only need to hit 10(ish) different web-services instead of the same one 3000 times. As this is a web application, does that kind of request amplification change the suggested approach? Is there a better approach to take? How well can I expect something like this to scale as the number of users increases?

A few notes:

  • Caching results will be the next implementation detail, but I would like to get the base functionality working first
  • Each web-service returns very little data (under a kb), longest one takes no more than 3 seconds to return and I believe most of that is latency (generating results not computationally expensive)
  • Calling web-services from the client won't work (I don't think) as some are in different domains
  • Either C# of F# implementation would work as ideally this will be abstracted away to a method call that takes a string and returns a collection of strings


You might find this discussion relevant:Async instead of Parallel


Web requests are not suitable for TPL, since they are IO, not processing. You already had all the necessary infrastructure in place since .Net 2.0: BeginGetResponse and BeginGetResponseStream. Keep a lid on the number of outstanding requests (~200 is a good consumer OS, ~2000 is for server flavours, after accordingly tweaking the TCP parameters, specially TcpNumConnections) and make sure you adjust the ServicePointManager.DefaultConnectionLimit.

Trying to achieve high scale and high bandwidth/throughput by simply throwing synchronous IO calls on tens and hundreds of tasks via TPL will get you nowhere fast.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜