开发者

Dynamically add/remove threads to the worker pool in celery

How do I add more threads (and remove threads) to the current multiprocessing pool, from within a task (i.e. celeryd was run with CELERYD_CONCURRENCY = 10 but I want to change it on-the-fly to CELERYD_CONCURRENCY = 15)?

There is a function called celery.concurrency.pro开发者_如何学JAVAcesses.TaskPool.Pool.grow but I have no idea how to call that from a running task or whether it is the correct function to do that.


Read the source:

https://github.com/ask/celery/blob/master/celery/concurrency/processes/__init__.py

there's both grow() and shrink(), although the latter seems a tad fishy.

you'd need to keep a reference to the pool somewhere, if you have only one pool, keep it global.

caveat poster: if multiprocessing actually means running multiple separate processes, you might already be in a child process when you try to shrink or grow, and obviously that won't work.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜