开发者

Most efficient way to create and write multiple 10 KB text files?

We're using the following approach to writing out text files:

public void WriteFile(string filePath, string contents)
{
    if (string.IsNullOrEmpty(filePath)) { throw new ArgumentNullException("filePath"); }
    if (contents == null) { throw new ArgumentNullException("contents"); }

    if (File.Exists(filePath))
    {
        throw new InvalidOperationException(string.Format("The file '{0}' already exists", filePath));
    }

    StreamWriter sw = File.CreateText(filePath); // slow

    sw.Write(contents);
    sw.Close(); // slow
}

We are calling this function a large number of times, so performance is key. The two lines commented slow are where our application is spending most of its time.

The parameter contents contains, on average, about 10 KB of text.

Are there any other approaches in .NET or using the Win32 API that are known to have significantly better performance?

We have already tried the following:

TextWriter tw = new StreamWriter(filePath);

tw.Write(contents);
tw.Close();

But have found the performance to be similar to our initial approach.

Edit

Based on suggestions we also tried:

File.WriteAllText(filePath, conten开发者_JAVA技巧ts);

However the performance of this is similar to the other approached listed above.


Sounds like a job for a backgroundworker. Offloads the writing to a new thread, and has events to notify you/the user of progress as well as completion.


The most straight forward way of writing the file would be:

File.WriteAllText(filePath, contents);

If you have performance problem with that, you should look at your requirements. As there really isn't any part of the operation that can be done more efficiently, you should determine if any part of the operation could be skipped or done later.

For example, do you need the file to be completely written and closed when you exit the method? Otherwise you could put the tasks in a queue and let a background thread write file after file at it's own pace.


Forget about you code. The performance problems when working with HDD is the problems of File System, Operation System and of course the hardware quality.

The only thing I can suggest is to redesign your application. For example you can write huge file instead of numerous small files. Just use .tar archiving for all of them, as it does not compress anything but simply stores file streams one by one.


You could to create another thread to write your file asynchronously in 8k data blocks (internal buffer size).


What about File.WriteAllText?


sw.Write... sw.Flush(); // insert new line sw.Close()...

If you insert the new line above you will probably see your "slow" move to it as the data is written to disk.

Are you running virus scanning software that you can disable? Can you enable write-caching in your battery-backup disk controller?

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜