How to unwrap wrapped lines in text file, reformat text file
I need help finding a Python solution to reformat the wrapped lines / rewrite the log file so there are no line breaks as described. That will allow me to continue to find on unbroken lines.
Every entry in the *.log is time stamped. Lines that are too long are wrapped as expected, however: The wrapped part is also time stamped. ">" (Greater than) is the only indication that a line has wrapped - happens on position 37. > The log is from a *nix machine.
I don't know how to begin...
开发者_高级运维2011-223-18:31:11.737 VWR:tao abc exec /home/abcd/abcd9.94/bin/set_specb.tcl -s DL 2242.500000 5
2011-223-18:31:11.737 > -20.000000 10
###needs to be rewritten as:
2011-223-18:31:11.737 VWR:tao abc exec /home/abcd/abcd9.94/bin/set_specb.tcl -s DL 2242.500000 5 -20.000000 10
And another
2011-223-17:40:07.039 EVT:703 agc_drift_cal.tcl: out of tolerance drift of 5.3080163871 detected! Downlink Alignmen
2011-223-17:40:07.039 >t check required.
###these lines deleted and consolodated as one:
2011-223-17:40:07.039 EVT:703 agc_drift_cal.tcl: out of tolerance drift of 5.3080163871 detected! Downlink Alignment check required.
I don't know how to begin, other than...
for filename in validfilelist:
logfile = open(filename, 'r')
logfile_list = logfile.readlines()
logfile.close
for line in logfile_list:
#!/usr/bin/python
import re
#2011-223-18:31:11.737 > -20.000000 10
ptn_wrp = re.compile(r"^\d+-\d+-\d+:\d+:\d+.\d+\s+>(.*)$")
validfilelist = ["log1.txt", "log2.txt"]
for filename in validfilelist:
logfile = open(filename, 'r')
logfile_new = open("%s.new" % filename, 'w')
for line in logfile:
line = line.rstrip('\n')
m = ptn_wrp.match(line)
if m:
logfile_new.write(m.group(1))
else:
logfile_new.write("\n")
logfile_new.write(line)
logfile_new.write("\n")
logfile.close()
logfile_new.close()
write new line when the line is not a wrap line. the only side effect is an empty line in the beginning. should not be a problem for log analysis. new file is the processed result.
This would do the trick if you wrap it in a filecontext:
f = [
"2011-223-18:31:11.737 VWR:tao abc exec /home/abcd/abcd9.94/bin/set_specb.tcl -s DL 2242.500000 5",
"2011-223-18:31:11.737 > -20.000000 10",
"2011-223-17:40:07.039 EVT:703 agc_drift_cal.tcl: out of tolerance drift of 5.3080163871 detected! Downlink Alignmen",
"2011-223-17:40:07.039 >t check required.",
]
import re
wrapped_line = "\d{4}-\d{3}-\d{2}:\d{2}:\d{2}\.\d{3} *>(.*$)"
result = [""]
for line in f:
thematch = re.match(wrapped_line,line)
if thematch:
result[-1] += thematch.group(1)
else:
result.append(line)
print result
for filename in validfilelist:
logfile = open(filename, 'r')
logfile_list = logfile.readlines()
logfile.close()
for line in logfile_list:
if(line[21:].strip()[0] == '>'):
#line_is_broken
else:
#line_is_not_broken
精彩评论