Queue System Farm questions

Until now, we haven't really needed to implement a queue system in our infrastructure, but now we need some features that cannot be done thru cron jobs (too slow).

Our infrastructure is LAMP (PHP) with some NoSQL.

I have looked at Gearman and it seems to be perfect for our needs. I know Digg uses it, as well as Yahoo so it must be reliable.

• Is there any management tools that will automatically start a worker when it dies, or when new tasks are created?

• How do you handle multiple projects under one same infrastructure (project A has 5 tasks, project B has 10, project C has 1)

• Is the开发者_高级运维re a way to do periodic tasks easily?

I've seen Celery and it looks good but It seems like it's only for AMQP (Or for RabbbitMQ only) but it's in python, and I haven't found much info on whether you can use non-python tasks.

Thank you

Celery supports webhook tasks, which enables it to call out to other languages via HTTP:

http://celeryq.org/docs/userguide/remote-tasks.html

Celery also supports Redis, MongoDB, MySQL/PostgreSQL/Oracle and beanstalkd: http://celeryq.org/docs/tutorials/otherqueues.html

RabbitMQ is the most reliable choice though.