Starting multiple celery daemons automatically using Jenkins

I have an Ubuntu server set up with 5 different django sites running on it. These are used for testing, so each developer has their own site and database as well as one site for integrated code which is only updated when features are ready. Jenkins is used to update each site from Github whenever changes are pushed to the repository.

We recently added Django-Celery to our dependencies so that we can do some processing on uploaded files asynchronously. Each site now needs its own celery queue that uses the correct settings (database, upload directory, etc.) for that particular site.

I want to restart each celery server whenever code changes so it can get the latest changes automatically. We have an update script within our git repository that Jenkins runs whenever it updates a site. When I try to start a celery daemon within this script, celery starts, but shuts down again at the end of the script.

Here's a copy of my update script:

#!/bin/bash

# Delete all *.pyc files
find $WORKSPACE -name '*.pyc' | xargs rm

# Update the database
[…]

# Run automated tests
python code/manage.py test <project> --noinput
TEST_STATUS=$?

# Refresh this repo's public website
touch $WORKSPACE/apache/wsgi.py

# Restart our celery daemon for this installation
/sbin/start-stop-daemon --stop -p $WORKSPACE/../celery.pid
echo 'Starting Celery Server'

# When run on the command line, this line starts a daemon just fine
/sbin/start-stop-daemon --start --background --quiet --oknodo -p $WORKSPACE/../celery.pid -m --exec $WORKSPACE/code/manage.py -- celeryd --logfile=$WORKSPACE/../celery.log

echo 'Celery Server Status: '$?

exit $TEST_STATUS

Here's a copy of the celery log during the execution of this script:

[2011-05-10 20:45:41,286: WARNING/MainProcess] -------------- celery@ip-10-227-139-6 v2.2.6
---- **** -----
--- * ***  * -- [Configuration]
-- * - **** ---   . broker:      djkombu.transport.DatabaseTransport://guest@localhost:5672/
- ** ----------   . loader:      djcelery.loaders.DjangoLoader
- ** ----------   . logfile:     /var/lib/jenkins/jobs/mpdaugherty-farmforce/workspace/../celery.log@WARNING
- ** ----------   . concurrency: 1
- ** ----------   . events:      OFF
- *** --- * ---   . beat:        OFF
-- ******* ----
--- ***** ----- [Queues]
 --------------   . celery:      exchange:celery (direct) binding:celery
[2011-05-10 20:45:41,333: WARNING/MainProcess] celery@ip-10-227-139-6 has 开发者_如何学JAVAstarted.
[2011-05-10 20:46:28,481: WARNING/MainProcess] celeryd: Warm shutdown (MainProcess)

Any advice for how I can get the celery daemons started by Jenkins to not shut down? Thanks a lot!

One of my coworkers finally made this work. Instead of starting the Celery daemon directly, we use at to schedule it immediately and disconnect from the current shell.

Instead of:

# Restart our celery daemon for this installation
/sbin/start-stop-daemon --stop -p $WORKSPACE/../celery.pid
echo 'Starting Celery Server'

# When run on the command line, this line starts a daemon just fine
/sbin/start-stop-daemon --start --background --quiet --oknodo -p $WORKSPACE/../celery.pid -m --exec $WORKSPACE/code/manage.py -- celeryd --logfile=$WORKSPACE/../celery.log

Change to:

# Restart our celery daemon for this installation
echo "/sbin/start-stop-daemon --stop -p $WORKSPACE/../celery.pid
echo 'Starting Celery Server'" | at now

# When run on the command line, this line starts a daemon just fine
echo "/sbin/start-stop-daemon --start --background --quiet --oknodo -p $WORKSPACE/../celery.pid -m --exec $WORKSPACE/code/manage.py -- celeryd --logfile=$WORKSPACE/../celery.log" | at now