Generic Log Parser Algorithm

My Application when running is writing logs. Now I need to check whether indexing is completed or not by checking for a status message as to whether it's written in logs or not (note that logging is going on dynamically and the process is running). My application is not sending a signal as to when it has completed the process of indexing, just logs it and goes doing other stuff. Should I poll the logs continuously to check whether status has been written in logs but that would be kind of anti-pattern or bad design. I cant even have a busy-waiting or a do nothing loop and then check, another bad design. How can I check for the en开发者_运维问答tered entry in logs in the best way without querying logs repetedly for that and with consuming less CPU cycles?

Polling is the usual solution. Other solutions require the collaboration of the generating process in some way; if this is possible, it's obviously a preferable solution, but if the generating process is to remain unaware of the listener (in the sense of not knowing about its existance), then polling is about the only valid solution. (Depending on the logging facilities, you might be able to arrange for the log to go into a named pipe, and read that.)

Note that polling isn't necessarily that expensive, if you aren't doing it too often.

If you control both of the programs (i.e. reading and writing the log), then the easiest solution is to have the writer notify all listeners when it is done using some form of inter-process communication (e.g. signals).

Only if IPC is not possible, you should look at smarter methods of waiting of polling for changes. Most operating systems let you register a callback for when a file or directory is modified. Take a look at this question for some suggestions.

Assuming that log parsing is the only alternative you have, the idiom you are looking for has the following high-level representation (UNIX CLI style)

# tail -f logfile.txt | grep STATUS_PATTERN

There (1) "tail -f" prints out any new lines that are appended to logfile.txt and passes them to (2) "grep" which performs the actual pattern matching.

Both (1) and (2) functionality is trivial to be implemented in Java/C++ as a separate thread/process and provide more lightweight load than the periodic polling. You will also need a little bit of extra functionality to detect log rotation conditions.