Web service with large result

We're building a web service and anticipate that there could be substantial data being returned.

How will ASP.Net and IIS 6 handle this? What kind of things can we do to make this less resource intensive?

For example, we currently have a seri开发者_运维百科alizable class, which has property which is an array of another serializable class type. When the web service is called, it queries a large set of records from the database, instantiates the class, populates the array property, then returns the class which gets serialized out.

How is this getting handled under the hood? I imagine that the entire DataSet object is stored in memory, then the entire serializable object is stored in member, then serialized and the entire xml is in memory, and finally streamed to the caller.

Are there things I can do to reduce memory consumption such as streaming out the results as they are being read from the database? Is this possible?

I'm not sure simply swapping to using WCF is the answer as the question is really about the load on requesting the data.

There are several things to consider even before choosing your prefered service method:

Can you cache the data? If it is possible to store the data in-memory then this would prevent there being the draw each time the data is requested. However if the data is prohibitive in size, you could consider writing the data from the db to file and serving it from there.

You could consider something like Redis with its VM mode. This would allow you to cache the data but if it isn't accessed frequently, the data could be written to file to prevent excessive use of memory.

Is there any way, the call to the database can be broken into several more efficient calls?

You could consider calling the data asynchronously if the time taken to execute the call is lengthy.

Finally, is it essential that the data is served as a webservice? Can the data be served as an XML document?