Truncate the beginning of a log file in .NET

I have an VB.NET app that writes the status to a log file in text format. Over time, the file is getting large and I wanted to know if there is an efficient way to truncate the beginning of the file.

To make things easier, I am looking to specify a file size (say 2-3 mb) and I am writing the log using 开发者_如何学编程a StreamWriter:

Using strm As New IO.StreamWriter(filelocation.log, True)
    strm.WriteLine("msg to write")
    strm.Close()
End Using

I thought about using the strm.BaseStream.Length to determine how much of the file to cut off, but by using the .SetLength it would cut from the end - not the desired result.

I would highly suggest looking at log4net to accomplish your logging. It's extremely powerful and flexible and has a built in way of rolling your logs based on a size you can specify. It's not the answer you're looking for here, but it's definitely worth looking into. Here's a sample config for what we do for logging during debug:

<log4net>
    <appender name="GeneralLog" type="log4net.Appender.RollingFileAppender">
        <file value="ClientTools.log"/>
        <appendToFile value="true"/>
        <maximumFileSize value="3000KB"/>
        <rollingStyle value="Size"/>
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%d{HH:mm:ss} [%t] %-5p %c - %m%n"/>
        </layout>
    </appender>
    <root>
        <level value="DEBUG"/>
        <appender-ref ref="GeneralLog"/>
    </root>
    <logger name="NHibernate" additivity="false">
        <level value="DEBUG"/>
        <appender-ref ref="GeneralLog"/>
    </logger>
</log4net>

This code in the app.config file will create a log file called ClientTools.log in the application folder, write it in a specific format including the date and time, and roll the log at 3MB.

To use the logger, we do this in Init() of the web page: public ILog log;

public void InitiateLogging()
{
    log4net.Config.XmlConfigurator.Configure();
    log = LogManager.GetLogger("MyApplication");
}

And then when you want to log something, do this:

log.Info("No resources available.");
// or
log.Fatal(exception.Message);
// or
log.Warn("Something bad may happen here.");

You don't have to worry about creating a stream object, closing the stream, disposing the stream, etc. And it's very DRY.

Why not check if the file's length in bytes is greater than 3MB, if it is, overwrite it and write to it afresh. Something like this, (I'm a c# guy):

System.IO.FileInfo f = new FileInfo(file_name);
if (f.Length > (1024 * 1024 * 3)){ 
    // File is over 3MB
    //
    // Perhaps back it up?
    // Then...
    using (StreamWriter sw = new StreamWriter(file_name, false))
    {
        // Overwrite the contents....
    }
}else{
    // Open as normal for append mode
}

Hope this helps, Best regards, Tom.

One approach could be

1. Read original file(A) line by line
2. Skip not required lines
3. Write required lines to another file (B).
4. save file B.
5. Delete file A.
6. Rename file B to file A.

You could check the File-Length and truncate it the hard (and dirty) way:

  If New FileInfo("yourFilePathHere").Length > (2 ^ 21) Then
        File.WriteAllText("yourFilePathHere", _
        File.ReadAllText("yourFilePathHere").Substring(ToInt32(2 ^ 21)))
  End If

Here are a couple functions that I use for this:

private void ShrinkLogFile()
{
    var file = new FileInfo(Filename);
    if (file.Exists && file.Length > MaxFileSize)
    {
        MoveLinesToBeginningStartingAt(file.Length - (MaxFileSize / 2));
    }
}
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2202:Do not dispose objects multiple times")]
private void MoveLinesToBeginningStartingAt(long offsetToKeep)
{
    // Open the file twice.  We'll read from the end and write to the beginning.
    using (var fileReader = new FileStream(Filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
    using (var fileWriter = new FileStream(Filename, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite))
    {
        // Find the end of the first line so we start at the beginning of a new line.
        fileReader.Position = offsetToKeep;
        using (var reader = new StreamReader(fileReader, Encoding.UTF8, true, 512, true))   // Note that we leave fileReader open...
        {
            offsetToKeep += reader.ReadLine().Length;       // Advance offset past the (probably) partial first line
            offsetToKeep += Environment.NewLine.Length;     // Advance past the newline
        }

        // Go to new position and copy the rest of the file to the beginning of the same file.
        fileReader.Position = offsetToKeep;
        fileReader.CopyTo(fileWriter);

        // Truncate the file
        var fileInfo = new FileInfo(Filename);
        fileWriter.SetLength(fileInfo.Length - offsetToKeep);
    }
}

There are a couple other things to consider:

1. You need to run these inside a lock (xxx) so there is NO possibility that another message will attempt to write to the file while the "Shrink" is happening.
2. Shrinking a file can take a bit of time, so your writes to the log should happen on a background thread.

Quick and easy method I use. This example keeps the most recent 60 lines of the log file.

dim LogPath as string = "YourLogPath"    
Dim lineCount = File.ReadAllLines(LogPath).Length

        If lineCount > 60 Then ' because I want my log to be 60 lines long
            Dim delLineCount As Integer = lineCount - 60
            Dim lines As List(Of String) = New List(Of String)(File.ReadAllLines(LogPath))
            lines.RemoveRange(0, delLineCount)
            File.WriteAllLines(LogPath, lines.ToArray())
        End If