Efficient file reading techniques in C#

I am looking for the best way to read a CSV file line by line. I want to know what are the most efficient ways of doing the same. I am particulary concerned when the size of the file is big. Are the file reading utilities avilable in the .NET classes the most effient?

(PS: I have searched for the word 'Ef开发者_如何学运维ficient' to know if someone has already posted similar question before posting this.)

You can try this: fast CSV Parser.

Here's the benchmark result:

To give more down to earth numbers, with a 45 MB CSV file containing 145 fields and 50,000 records, the reader was processing about 30 MB/sec. So all in all, it took 1.5 seconds! The machine specs were P4 3.0 GHz, 1024 MB.

The file handling in .NET is fine. If you want to read a line at a time in a comfortable way using foreach, I have a simple LineReader class you might want to look at (with a more general purpose version in MiscUtil).

With that you can use:

foreach (string line in new LineReader(file))
{
    // Do the conversion or whatever
}

It's also easily usable with LINQ. You may or may not find this helpful - if the accepted solution works for you, that may be a better "off the shelf" solution.