Why doesnt "tail" work to truncate log files?

I'm trying to manage my log file size using a cron script. I basically want to delete all but the last 2000 lines of the log file 开发者_JS百科every night. I'm trying to run this command, but it seems to be emptying the entire file instead of doing what I want:

tail -2000 logfile.txt > logfile.txt

Does anyone know why this isn't working and/or how to accomplish what I want? Thanks!

You are overwriting the file before tail even starts to read it. The shell processes the > redirect operator, by clearing out the file first. Then it runs tail which has no data to read.

You can solve this by using a temporary file:

tail -2000 logfile.txt >logfile.tmp
mv logfile.tmp logfile.txt

There is a problem with the accepted solution if the process keeps the log file open; you basically need to reuse the i-node. Mmrobins' response is good, logrotate should do the right thing.

To use tail, you can do something (similar to Pantonza's & Greg's idea), but retain the original file by truncating the original file in-place:

tail -2000 logfile.txt >logfile.tmp
cat logfile.tmp > logfile.txt
rm logfile.tmp

To avoid a temp file, you could read into a variable, then stuff it back:

bash -c 'X=$(tail -2000 logfile.txt);echo "$X">logfile.txt'

In all cases, there's the possibility of a race condition between your truncation and the process appending to the file. Not sure if logrotate handles that, none of the tail solutions here do.

Rather than doing this with your own cron file you might want to look into using logrotate for a more robust solution. You can rotate the logs, control how long to keep them around, email them, compress old logs, and run scripts afterwards if you want to.

See the man page here or type man logrotate from the command line

Create a new logrotate file in your /etc/logrotate.d/ directory. Here's an example:

/var/logs/myapp/logfile.txt {
  # keep the 5 latest rotations of the log 
  rotate 5
  # rotate once the file is bigger than 2k
  size 2k
  # don't error if the file isn't there
  missingok
  # compress rotated (old) log files
  compress
  # set ownership and permissions on the new log file
  create 0640 myuser myuser
}

Here is another solution, without dealing with tmp files:

echo "`tail -2000 logfile.txt`" > logfile.txt

Greg Hewgill is right, logfile.txt is being truncated before tail can work on it.

try:

tail -2000 logfile.txt > logfile2.txt; rm -f logfile.txt; mv logfile2.txt logfile.txt

If you're a vim user then another option is to use vi instead of tail, to operate on the file directly:

vi "+execute \"normal! G2000kO\<esc>dgg\"" "+w" "+q" logfile.txt

The string G2000kO\<esc>dgg is simply the keyboard input you would use to do this with vi (the escape button must be escaped). It is very intuitive to use and covers a huge range of use cases.

vi also handles large (10+ GB) textfiles without even slowing down, so this method is very robust.