Monitor a application/server in real-time through its log [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.

---

Cl开发者_开发百科osed 8 years ago.

• Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
• This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.

Improve this question

Is there any tool that can analyze in real-time a log (a custom server application) and output custom metrics over the log? Possibly some graphs on a web interface?

My typical use case is the following: I have a socket server receiving incoming requests. Each request is of a specific "type". I would like to see pieces of information like: how many requests occurred in the last N minutes? The average time of response for the requests (each logged request has its response time); How many requests of a single type has occurred in the last N minutes? And others.

I'm aware that specific information contained in a log item cannot be automatically extracted and I would probably need some coding to instruct the tool how to parse and understand the log, and how it could gather information out of it.

Is there a tool/software/system/monitor that could help with that?

---

Send Jabber/XMPP message to user@jabberserver.com when new line is appended to the log:

# tail --follow=name --retry /var/log/logfile.log | sendxmpp -i -u username -p password -j jabberserver.com user@jabberserver.com

username/password must be created on jabberserver.com. user@jabberserver.com must be loged in via Jabber/XMPP client that you write on your own (I prefer perl + Net::Jabber). Then you can do whatever you want with the incoming XMPP messages (store on DB, send e-mail, ...).

---

Tail is the most common tool I know of and if your on windows there is wintail.

---

At Ustream we needed a tool just like that, so we wrote Yolo. It is a general log parser and processor that can send data to StatsD, which aggregates the data you need (it supports counters, timing data and gauges) and sends to Graphite.

In a nutshell you just have to write a config file, setup a regexp parser to match your log data and setup the statsd keys you want to use. You don't have to write any code.

You can find an example config on Github.