How to map a job among multiple workers using Redis Pub/Sub?
I have a Redis client which needs to split/map a job among say 3 workers. Suppose the job consists of 3 tasks. I want to map them in such a way that 3 of them runs in parallel and the job takes only (approx) 1/3 of execution time. Is there any way to do this with Redis itself?开发者_如何学编程 I didn't find any.
You can use redis to create a task queue. For example, you can have a master process pushing tasks to a task queue and then have workers polling the task queue constantly for new work.
Master psuedo code:
while(1)
if some_condition
redis.rpush "tasks", "task1"
redis.rpush "tasks", "task2"
redis.rpush "tasks", "task3"
end
sleep 5
end
Worker pseudo code:
while(1)
# blpop blocks until there is an element in "tasks"
task = redis.blpop("tasks", 0)[1]
perform task
end
精彩评论