开发者

LInux sort/uniq apache log

I have small file (100) lines of web request(apache std format) there are multiple request from clients. I want to ONLY have a list of request(lines) in my file that comes from a UNIQUE IP and is the latest entry

开发者_JS百科

I have so far /home/$: cat all.txt | awk '{ print $1}' | sort -u | "{print the whole line ??}"

The above gives me the IP's(bout 30 which is right) now i need to have the rest of the line(request) as well.


Use an associative array to keep track of which IPs you've found already:

awk '{  
  if (!found[$1]) {
    print;
    found[$1]=1;
  }
}' all.txt

This will print the first line for each IP. If you want the last one then:

awk '
     { found[$1] = $0 }
     END {
       for (ip in found)
         print found[ip]
     }
' all.txt


I hate that unique doesn't come with the same options as sort, or that sort cannot do what it says, I reckon this should work[1],

tac access.log | sort -fb -k1V -u

but alas, it doesn't;

Therefore, it seems we're stuck at doing something silly like

cat all.txt | awk '{ print $1}' | sort -u | while read ip
do
    tac all.txt | grep "^$ip" -h | head -1
done

Which is really inefficient, but 'works' (haven't tested it: module typo's then)

[1] according to the man-page


The following should work:

tac access.log | sort -f -k1,1 -us

This takes the file in reverse order and does a stable sort using the first field, keeping only unique items.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜