Avoiding BinaryReader.ReadString() in C#?

Good morning,

At the startup of the application I am writing I need to read about 1,600,000 entries from a file to a Dictionary<Tuple<String, String>, Int32>. It is taking about 4-5 seconds to build the whole structure using a BinaryReader (using a FileReader takes about the same time). I profiled the code and found that the function doing the most work in this process is BinaryReader.ReadString(). Although this process needs to be run only once and at startup, I would like to make it as quick as possible. Is there any way I can avoid BinaryRead开发者_运维技巧er.ReadString() and make this process faster?

Thank you very much.

Are you sure that you absolutely have to do this before continuing?

I would examine the possibility of hiving off the task to a separate thread which sets a flag when finished. Then your startup code simply kicks off that thread and continues on its merry way, pausing only when both:

• the flag is not yet set; and
• no more work can be done without the data.

Often, the illusion of speed is good enough, as anyone who has coded up a splash screen will tell you.

Another possibility, if you control the data, is to store it in a more binary form so you can just blat it all in with one hit (i.e., no interpretation of the data, just read in the whole thing). That, of course, makes it harder to edit the data from outside your application but you haven't stated that as a requirement.

If it is a requirement or you don't control the data, I'd still look into my first suggestion above.

If you think that reading the file line by line is the bottleneck, and depending on its size, you can try to read it all at once:

// read the entire file at once
string entireFile = System.IO.File.ReadAllText(path);

It this doesn't help, you can try to add a separate thread with a semaphore, which would start reading in background immediately when the program is started, but block the requesting thread at the moment you try to access the data.

This is called a Future, and you have an implementation in Jon Skeet's miscutil library.

You call it like this at the app startup:

// following line invokes "DoTheActualWork" method on a background thread.
// DoTheActualWork returns an instance of MyData when it's done
Future<MyData> calculation = new Future<MyData>(() => DoTheActualWork(path));

And then, some time later, you can access the value in the main thread:

// following line blocks the calling thread until
// the background thread completes
MyData result = calculation.Value;

If you look at the Future's Value property, you can see that it blocks at the AsyncWaitHandle if the thread is still running:

public TResult Value
{
    get
    {
        if (!IsCompleted)
        {
            _asyncResult.AsyncWaitHandle.WaitOne();
            _lock.WaitOne();
        }
        return _value;
    }
}

If strings are repeated inside tuples you could reorganize your file to have all different involving strings at the start, and have references to those strings (integers) in the body of the file. Your main Dictionary does not have to change, but you would need a temporary Dictionary during startup with all different strings (values) and their references (keys).