Python Client/Server to stream logfile updates in realtime?

We have applications on Solaris 10 servers that generate text logfiles. We need to stream these logfiles in realtime to a central server for monitoring of new events.

Ideally we'd have a NFS-mount, and all our system would write their logs to there, and the monitoring server could just pull them up from there. Unfortunately, for technical and non-technical reasons that's not an option here.

At the moment, we're using a backgrounded tail -f to pipe the data over an SSH tunnel.

However, we were looking at whether it's worth putting together something a bit more robust.

I was thinking of writing a simple Python client/server with Twistedb (or something similar - recommendations?) to stream the log data. Is this something that's easily achievable? Any existing libraries/tools I could look to for id开发者_如何学JAVAeas? Any issues I should be aware of?

Also, this is Solaris 10, so I'm not familiar with the state of filesystem monitors. I do know Gamin is available via OpenCSW. however, are there any other choices out there?

Check out Python's logging module. http://docs.python.org/library/logging.html

It contains the capability to log to files, streams, syslog, networked servers, and more. The cookbook contains examples or logging over the network. http://docs.python.org/howto/logging-cookbook.html#logging-cookbook

The module is fairly easy to extend also.

Do consider zeromq, instead of raw sockets. Its no message broker server, its a library to let you write your message passing system. It enables easy cross-platform and language-agnostic communication over TCP, IPC (yes,inter-process!), multicast and other protocols. Simply swap a python component with a C component and it just works as before.

Its the perfect software to satisfy your "..excuse to learn.." ;-)

But the show-stopper would be building it in your Solaris setup. Python bindings are at http://pypi.python.org/pypi/pyzmq/2.1.9