how to write a super fast ascii file in C# if it contains 100000 lines of data?

i have a code which generates random strings and numbers from random class in C#. and then writes it to a text file. how can i improve the performance? the code is as follows:

    int i = 0;
    Record[] rec = new Record[100000];
    Class1 cl = new Class1();
    Random random = new Random();
    while (i < 100000)
    {
        rec[i].num1 = random.Next();
        rec[i].num2 = random.Next();
        rec[i].mult = rec[i].num1 * rec[i].num2;
        rec[i].rel = true;
        rec[i].name = cl.generateRandomString(1);
        rec[i].var_set = cl.generateRandomString(2);          
        using (StreamWriter writer = new StreamWriter("important.txt", true))
        {

            writer.Write(rec[i].name);
            writer.Write("   ");
            writer.Write(rec[i].var_set);
            writer.Write("   ");
            writer.Write(rec[i].num1);
            wr开发者_如何转开发iter.Write("   ");
            writer.Write(rec[i].num2);
            writer.Write("   ");
            writer.Write(rec[i].mult);
            writer.Write("   "); 
            writer.WriteLine(rec[i].rel);
        }
        i++;

Based on the code you supplied in the comment for @Ben, I got this code that you are using.

Record[] rec = new Record[100000]; 
Class1 cl = new Class1(); 
Random random = new Random(); 
while (i < 100000) 
{ 
    rec[i].num1 = random.Next();
    rec[i].var_set = cl.generateRandomString(2);
    using (StreamWriter writer = new StreamWriter("important.txt", true)) 
    { 
        writer.Write(rec[i].name); 
        writer.Write(" "); 
    } 
    i++;
}

The biggest problem I see is that you are opening the streamwriter for each iteration of the loop. You probably want to change your code to something like this so that the file is only opened once.

int i = 0;
Record[] rec = new Record[100000]; 
Class1 cl = new Class1(); 
Random random = new Random(); 
StreamWriter sr = new StreamWriter("important.txt", true);

try
{
    while (i < 100000) 
    { 
        rec[i].num1 = random.Next();
        rec[i].var_set = cl.generateRandomString(2);
        sr.Write(rec[i].name); 
        sr.Write(" "); 
        i++;
    }
}
finally
{
    sr.Close();
}

what happens when you do this:

Record[] rec = new Record[100000]; 
Class1 cl = new Class1(); 
Random random = new Random(); 
int i = 0;
while (i < 100000) 
{ rec[i].num1 = random.Next(); rec[i].var_set = cl.generateRandomString(2); i++; };

i = 0;
using (StreamWriter writer = new StreamWriter("important.txt", true))
    {

        while ( i < 100000)
        {
        writer.Write(rec[i].name);
        writer.Write("   ");
        writer.Write(rec[i].var_set);
        writer.Write("   ");
        writer.Write(rec[i].num1);
        writer.Write("   ");
        writer.Write(rec[i].num2);
        writer.Write("   ");
        writer.Write(rec[i].mult);
        writer.Write("   "); 
        writer.WriteLine(rec[i].rel);

        i++;
        };
    }

EDIT - another option:

Record[] rec = new Record[100000]; 
Class1 cl = new Class1(); 
Random random = new Random(); 
int i = 0;
while (i < 100000) 
{ rec[i].num1 = random.Next(); rec[i].var_set = cl.generateRandomString(2); i++; };

File.WriteAllLines ( "important.txt", (from r in rec select r.name + "   " + 
r.var_set  + "   " + r.num1 + "   " + r.num2 + "   " + r.mult + 
"   " + r.rel).ToArray());

System.IO.MemoryMappedFiles.MemoryMappedFile is incredibly fast.

But probably your performance problems are caused by formatting your output, and not actually writing the file (file writes are very fast when disk write cache is enabled).

You really should take the time to edit your question and include the code from the comment.

So that's your current code as from the comment:

Record[] rec = new Record[100000]; 
Class1 cl = new Class1(); 
Random random = new Random(); 
while (i < 100000) 
{ 
    rec[i].num1 = random.Next();
    rec[i].var_set = cl.generateRandomString(2);
    using (StreamWriter writer = new StreamWriter("important.txt", true))
    { 
        writer.Write(rec[i].name); 
        writer.Write(" "); 
    } 
    i++; 
}

What is wrong with it is the fact that you're opening, writing and closing the file 100000 times for each iteration of the outer loop.

Put the open/close logic outside the loop:

Record[] rec = new Record[100000]; 
Class1 cl = new Class1(); 
Random random = new Random(); 
using (StreamWriter writer = new StreamWriter("important.txt", true))
{ 
    while (i < 100000) 
    { 
        rec[i].num1 = random.Next();
        rec[i].var_set = cl.generateRandomString(2);
        writer.Write(rec[i].name); 
        writer.Write(" "); 
        i++; 
    } 
}

The performance is always application specific, only you can really define what is "superfast". By the way one of the solutions + already offered by @Ben is to have a big chunk of data, like a 20.000 lines per chunk and write them in 5 shots, pr one big chunk of data of 100.000 lines if can afford them in your app, why not? Could be good solution too, but it's very your context specific, so how says Rico Mariani: "measure, measure, measure".

Regards.