How to implement a distributed semaphore?

I have a limited set of resources that I want to share between multiple processes than run on multiple servers. I guess what I need could be called a distributed semaphore.

I have found an old Perl implementation (based on memcached) that could be adapted. I have not fully investigated it yet.

Is t开发者_如何学Chere a library/component/software that already does that? An algorithm, perhaps? How about failover?

The algorithm for this is called Paxos. There are other algorithms, but they all reduce down to Paxos (or are incorrect). The most popular implementation of it is Apache Zookeeper. The Zookeeper servers run Paxos amongst themselves. Clients refer to named objects and can lock them etc.

I want to point out that from the use case you describe there is probably little need for the semaphore itself to be distributed. So you do not necessarily want a distributed semaphore, but a semaphore a for distributed system.

Throttle might fit this bill: https://github.com/pacman82/throttle

In order to implement a distributed semaphore, you need to define an authority amongst a cluster of node, then once it is selected, you need to centralize requests to it. Handling lock requests is a piece of cake, selecting the leader is more complicated.

Paxos will solve that for you. I have just updated the wikipedia page, because the algorithm description was incomplete and misleading.

there is a simpler algorithm similar to paxos, it's called raft: http://raftconsensus.github.io/

I'd like to suggest implement distributed semaphore based on consul sessions. A session in Consul represents a contract that has very specific semantics.

The contract that Consul provides is that under any of the following situations, the session will be invalidated:

1. Node is deregistered
2. Any of the health checks are deregistered
3. Any of the health checks go to the critical state
4. Session is explicitly destroyed
5. TTL expires, if applicable

When a session is invalidated, it is destroyed and can no longer be used.

Detailed guide: https://www.consul.io/docs/internals/sessions.html