GC Pauses the Full Application Activity

I am building a web crawl开发者_StackOverflow社区er in .Net which executes approx 500 httpwebrequests at a time. Everything runs fine but the problem arise at some points; Looks like garbage collection thread takes over the whole application pause for a few seconds. Is there anyway that reduce the delay generated by GC.

Guidance on how to avoid GC problems in .Net

To get the best out of the allocator you should consider practices such as the following:

• Allocate all of the memory (or as much as possible) to be used with a given data structure at the same time.
• Remove temporary allocations that can be avoided with little penalty in complexity.
• Minimize the number of times object pointers get written, especially those writes made to older objects.
• Reduce the density of pointers in your data structures.
• Make limited use of finalizers, and then only on "leaf" objects, as much as possible. Break objects if necessary to help with this.

A regular practice of reviewing your key data structures and conducting memory usage profiles with tools like Allocation Profiler will go a long way to keeping your memory usage effective and having the garbage collector working its best for you.

You can request a GC at some more convenient time which may reduce the pauses later

Also look at the related StackOverflow questions to the right - there are some relevant ones there ----->