How can I increase the I/O queue depth to maximize performance

Disk benchmarks typically have a chart that shows the I/O throughput increasing as the queue depth increases. I'm assuming 开发者_开发问答that what's going on at the hardware level is that the disk controller is reordering the requests to coincide with where the magnetic head happens to be. But how can I increase the queue depth in my own I/O-heavy application? I'll be using C#, but language-agnostic answers will probably also be helpful.

One (untested) method that occurred to me was to split my disk access into many files and issue the I/Os in many threads. But surely there is a way to do it without using a thread per parallel I/O. And I'm also assuming that splitting to many files is not needed since a DB is generally one file.

You don't need to use a thread per I/O operation, but you do need to parallelize your I/O. Servers that perform large-scale I/O (like databases) typically use I/O completion ports (IOCP). Completion ports allow multiple I/O requests to be pending in parallel - and can maximize the underlying hardware's ability to reorder queued requests. IOCP use callbacks to a thread-pool to notify an application when pending I/O operations have completed.

Unfortunately, C# has limited (built-in) support for completion ports - however, there are some open-source libraries that attempt to bridge this gap.

Beware, IOCP is not for the weak of heart - and the performance benefit only accrues if you are truly performing a massive amount of I/O.