How should I handle working with celeryd_multi from code?
So far, I've been working only with python manage.py celeryd
, starting it like开发者_C百科 this:
python manage.py celeryd -l info --settings=settings
The code from my view, does this:
BinaryExecTask.delay(request.POST["binary_path"])
And the code from my settings.py
, is this:
import djcelery
djcelery.setup_loader()
BROKER_BACKEND = "djkombu.transport.DatabaseTransport"
#celery
BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "guest"
BROKER_PASSWORD = "guest"
BROKER_VHOST = "/"
and it will execute some binaries in the background. The thing is, some of the binaries take pretty short time to run, while others may take up to half an hour. Working with celeryd
, all my tasks are blocked until the current one finishes it's execution. I saw here some examples of starting celeryd_multi, but running:
python manage.py celeryd_multi start 3 --settings=settings -l info
gives this error:
celeryd-multi v2.3.1
> Starting nodes...
> celery1.x: Traceback (most recent call last):
File "manage.py", line 14, in <module>
execute_manager(settings)
File "c:\code\python27\lib\site-packages\django-1.3-py2.7.egg\django\core\management\_
line 438, in execute_manager
utility.execute()
File "c:\code\python27\lib\site-packages\django-1.3-py2.7.egg\django\core\management\_
line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "c:\code\python27\lib\site-packages\django_celery-2.3.3-py2.7.egg\djcelery\manage
s\celeryd_multi.py", line 22, in run_from_argv
["%s %s" % (argv[0], argv[1])] + argv[2:])
File "c:\code\python27\lib\site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_mul
172, in execute_from_commandline
self.commands[argv[0]](argv[1:], cmd)
File "c:\code\python27\lib\site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_mul
205, in start
retcode = self.waitexec(argv)
File "c:\code\python27\lib\site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_mul
354, in waitexec
pipe = Popen(argstr, env=self.env)
File "c:\code\python27\lib\subprocess.py", line 672, in __init__
errread, errwrite)
File "c:\code\python27\lib\subprocess.py", line 882, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
The celeryd-multi start 3 -c 3
throws the same error. What should I do so that I could succesfully start a celery instance that will allow me to run more tasks in parallel? Also, would I need to do something different in my view?
EDIT: some debugging led me here ( site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_multi.py(354)waitexec )
351 def waitexec(self, argv, path=sys.executable):
352 args = " ".join([path] + list(argv))
353 -> argstr = shlex.split(args.encode("utf-8"))
354 pipe = Popen(argstr, env=self.env)
(Pdb) p argstr
['c:codepython27python.exe', 'manage.py', 'celeryd_detach', '-l', 'info', '--pidfile=celeryd@1.pid',
'-n', 'celery1.x', '--logfile=celeryd@1.log']
(Pdb) p Popen(argstr, env=self.env)
*** WindowsError: WindowsError(2, 'The system cannot find the file specified')
(Pdb)
So, as we can see, the path to Python gets destroyed :). What should I do next?
EDIT2: I opened an issue here
Looks like smth Windows-specific... Did you try to provide full path to python executable in command line like
C:\code\python27\bin\python.exe manage.py celeryd_multi start 3 --settings=settings -l info
Also, use full paths for python file can be usefull
C:\code\python27\bin\python.exe C:\path\to\your\project\manage.py celeryd_multi start 3 --settings=settings -l info
精彩评论