Semaphores and Locks in Ubuntu
I have a backup server which receives many rsync
connections every hour. Since having too many open rsync
instances might crash it, I want to limit the number of concurrent instances using a Semaphore. What I have in mind is something like:
ssh root@backup_server "get_semaphore" #Will hold until semaphore released
rsync开发者_如何学C -avzrL --super --delete local_directory root@backup_server:`localhost`
Any ideas?
In my opinion, you should be looking at limiting the number of concurrent connections directly on the server side, so that you don't have to do anything extreme on the client side:
(1) If you run rsync in daemon mode, there is a max connections
option for rsyncd.conf.
(2) On Linux, you can use iptables
and its connlimit
module to limit the number of concurrent connections to a port from one or more remote hosts. You may have to force any rsync clients to use a different port than your normal ssh users and have an sshd instance listen to that as well.
(3) Replace your rsync binary on the server with a wrapper script that will catch the --server parameter that is used internally by rsync and stall until a slot is available. Said wrapper script should make sure that no more than N instances of itself are executed at the same time. E.g.
#!/bin/bash
N=5
mutex_hold() {
while ! mkdir /var/lock/rsync/mutex 2>/dev/null; do
sleep 1
done
}
mutex_release() {
rmdir /var/lock/rsync/mutex
}
if [[ "$1" = "--server" ]]; then
shopt -s nullglob
while mutex_hold && A=(/var/lock/rsync/[0-9]*) && [[ "${#A[@]}" -ge "$N" ]] && mutex_release; do
sleep 1
done
touch /var/lock/rsync/$$
mutex_release
rsync.bin "$@"
rm -f /var/lock/rsync/$$
else
rsync.bin "$@"
fi
Please note that this script is mostly untested and that it lacks the necessary trap code to remove the lockfile even if interrupted. It also does not take care of any stale lock files etc or creating the lock directory.
If you are set on making your own semaphore implementation, you might be interested in the mutex I used above, in order to avoid the race condition between counting the number of lockfiles and creating a new lockfile, which might allow two (or more) instances to run in a single slot.
In all of these cases you should make sure that your clients can handle connection timeouts or rejected connections gracefully.
If you really want to use a semaphore-style system as you proposed, the script above might serve with a few modifications.
精彩评论