Extracting specific fields from a log file and export it to csv file using C#
This is the code that I have written in C# which shows the contains in the sample.log file which is 110MB in size.
using System;
using System.IO;
using System.Text;
class Program
{
static void Main(string[] args)
{
Fil开发者_Python百科eStream fs = new FileStream("sample.log", FileMode.Open, FileAccess.Read);
StreamReader sr = new StreamReader(fs);
string StrFromFile;
StringBuilder ResultStr = new StringBuilder();
while ((StrFromFile = sr.ReadLine()) != null)
{
// your separator char seems to be char 1
string[] SplitStrs = StrFromFile.Split(new char[] {(char)1});
for (int i = 0; i < SplitStrs.Length; i++)
{
if (SplitStrs[i].StartsWith("52="))
{
ResultStr.Append(SplitStrs[i] + " ");
}
else if (SplitStrs[i].StartsWith("55="))
{
ResultStr.Append(SplitStrs[i] + " ");
}
else if (SplitStrs[i].StartsWith("132="))
{
ResultStr.Append(SplitStrs[i] + " ");
}
else if (SplitStrs[i].StartsWith("133="))
{
ResultStr.Append(SplitStrs[i] + " ");
}
else if (SplitStrs[i].StartsWith("35="))
{
ResultStr.Append(SplitStrs[i] + " ");
}
}
Console.WriteLine(ResultStr);
ResultStr.Length = 0;
}
sr.Close();
fs.Close();
Console.ReadKey();
}
}
I am getting this output,
Output:
35=5 52=20101219-18:05:01.522
35=A 52=20101219-18:06:01.504
35=A 52=20101219-18:06:02
35=1 52=20101219-18:06:02
35=R 52=20101219-18:06:01.847 55=EUR/USD
Now the problem is how do I write this output in csv file? Any idea.
Rather than write your own: Log Parser 2.2:
Log parser is a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows® operating system such as the Event Log, the Registry, the file system, and Active Directory®. You tell Log Parser what information you need and how you want it processed. The results of your query can be custom-formatted in text based output, or they can be persisted to more specialty targets like SQL, SYSLOG, or a chart.
Log Parser Examples
Coding Horror: Microsoft LogParser
Forensic Log Parsing with Microsoft's LogParser
Log Parser's built-in Input Formats can retrieve data from the following sources:
- IIS log files (W3C, IIS, NCSA, Centralized Binary Logs, HTTP Error logs, URLScan logs, ODBC logs)
- Windows Event Log
- Generic XML, CSV, TSV and W3C - formatted text files (e.g. Exchange Tracking log files, Personal Firewall log files, Windows Media® Services log files, FTP log files, SMTP log files, etc.)
- Windows registry
- Active Directory Objects
- File and Directory information
- NetMon .cap capture files
- Extended/Combined NCSA log files
For Linux you could use AWStats:
AWStats can analyze a lot of log formats: Apache NCSA combined log files (XLF/ELF) or common (CLF), IIS log files (W3C), WebStar native log files and other web, proxy, wap or streaming servers log files (but also ftp or mail log files).
AWStats logfile analyzer 7.0 Documentation: FAQ
Also: Logparser (Microsoft's one) or similar for Unix?
Have a look at filehelpers: www.filehelpers.com, an open source .Net library for parsing and saving CSV files.
Very easy and very powerful
精彩评论