开发者

How to map a job among multiple workers using Redis Pub/Sub?

I have a Redis client which needs to split/map a job among say 3 workers. Suppose the job consists of 3 tasks. I want to map them in such a way that 3 of them runs in parallel and the job takes only (approx) 1/3 of execution time. Is there any way to do this with Redis itself?开发者_如何学编程 I didn't find any.


You can use redis to create a task queue. For example, you can have a master process pushing tasks to a task queue and then have workers polling the task queue constantly for new work.

Master psuedo code:

while(1)
  if some_condition
    redis.rpush "tasks", "task1"
    redis.rpush "tasks", "task2"
    redis.rpush "tasks", "task3"
  end
  sleep 5
end

Worker pseudo code:

while(1)
  # blpop blocks until there is an element in "tasks" 
  task = redis.blpop("tasks", 0)[1]
  perform task
end
0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜