PHP flock() alternative
PHP's documentation page for flock() indicates that it's not safe to use under IIS. If I can't开发者_开发知识库 rely on flock under all circumstances, is there another way I could safely achieve the same thing?
There is no alternative available to safely achieve the same under all imaginary possible circumstances. That's by design of computer systems and the job is not trivial for cross-platform code.
If you need to make safe use of flock(), document the requirements for your application instead.
Alternatively you can create your own locking mechanism, however you must ensure it's atomic. That means, you must test for the lock and if it does not exists, establish the lock while you need to ensure that nothing else can acquire the lock in-between.
This can be done by creating a lock-file representing the lock but only if it does not exists. Unfortunately, PHP does not offer such a function to create a file in such a way.
Alternatively you can create a directory with mkdir() and work with the result because it will return true when the directory was created and false if it already existed.
You can implement a filelock - unlock pattern around your read/write operations based on mkdir, since that is atomic and pretty fast. I've stress tested this and unlike mgutt did not find a bottleneck. You have to take care of deadlock situations though, which is probably what mgutt experienced. A dead lock is when two lock attempts keep waiting on each other. It can be remedied by a random interval on the lock attempts. Like so:
// call this always before reading or writing to your filepath in concurrent situations
function lockFile($filepath){
clearstatcache();
$lockname=$filepath.".lock";
// if the lock already exists, get its age:
$life=@filectime($lockname);
// attempt to lock, this is the really important atomic action:
while (!@mkdir($lockname)){
if ($life)
if ((time()-$life)>120){
//release old locks
rmdir($lockname);
$life=false;
}
usleep(rand(50000,200000));//wait random time before trying again
}
}
Then work on your file in filepath and when you're done, call:
function unlockFile($filepath){
$unlockname= $filepath.".lock";
return @rmdir($unlockname);
}
I've chosen to remove old locks, well after the maximum PHP execution time in case a script exits before it has unlocked. A better way would be to remove locks always when the script fails. There is a neat way for this, but I have forgotten.
My proposal is to use mkdir() instead of flock(). This is a real-world example for reading/writing caches showing the differences:
$data = false;
$cache_file = 'cache/first_last123.inc';
$lock_dir = 'cache/first_last123_lock';
// read data from cache if no writing process is running
if (!file_exists($lock_dir)) {
// we suppress error messages as the cache file exists in 99,999% of all requests
$data = @include $cache_file;
}
// cache file not found
if ($data === false) {
// get data from database
$data = mysqli_fetch_assoc(mysqli_query($link, "SELECT first, last FROM users WHERE id = 123"));
// write data to cache if no writing process is running (race condition safe)
// we suppress E_WARNING of mkdir() because it is possible in 0,001% of all requests that the dir already exists after calling file_exists()
if (!file_exists($lock_dir) && @mkdir($lock_dir)) {
file_put_contents($cache_file, '<?php return ' . var_export($data, true) . '; ?' . '>')) {
// remove lock
rmdir($lock_dir);
}
}
Now, we try to achieve the same with flock():
$data = false;
$cache_file = 'cache/first_last123.inc';
// we suppress error messages as the cache file exists in 99,999% of all requests
$fp = @fopen($cache_file, "r");
// read data from cache if no writing process is running
if ($fp !== false && flock($fp, LOCK_EX | LOCK_NB)) {
// we suppress error messages as the cache file exists in 99,999% of all requests
$data = @include $cache_file;
flock($fp, LOCK_UN);
}
// cache file not found
if (!is_array($data)) {
// get data from database
$data = mysqli_fetch_assoc(mysqli_query($link, "SELECT first, last FROM users WHERE id = 123"));
// write data to cache if no writing process is running (race condition safe)
$fp = fopen($cache_file, "c");
if (flock($fp, LOCK_EX | LOCK_NB)) {
ftruncate($fp, 0);
fwrite($fp, '<?php return ' . var_export($data, true) . '; ?' . '>');
flock($fp, LOCK_UN);
}
}
The important part is LOCK_NB to avoid blocking all consecutive requests:
It is also possible to add LOCK_NB as a bitmask to one of the above operations if you don't want flock() to block while locking.
Without it, the code would produce a huge bottleneck!
An additional important part is if (!is_array($data)) {. This is because $data could contain:
array()as a result of the db queryfalseof the failinginclude- or an empty string (race condition)
The race condition happens if the first visitor executes this line:
$fp = fopen($cache_file, "c");
and another visitor executes this line one millisecond later:
if ($fp !== false && flock($fp, LOCK_EX | LOCK_NB)) {
This means the first visitor creates the empty file, but the second visitor creates the lock and so include returns an empty string.
So you saw many pitfalls that can be avoided through using mkdir() and its 7x faster, too:
$filename = 'index.html';
$loops = 10000;
$start = microtime(true);
for ($i = 0; $i < $loops; $i++) {
file_exists($filename);
}
echo __LINE__ . ': ' . round(microtime(true) - $start, 5) . PHP_EOL;
$start = microtime(true);
for ($i = 0; $i < $loops; $i++) {
$fp = @fopen($filename, "r");
flock($fp, LOCK_EX | LOCK_NB);
}
echo __LINE__ . ': ' . round(microtime(true) - $start, 5) . PHP_EOL;
result:
file_exists: 0.00949
fopen/flock: 0.06401
P.S. as you can see I use file_exists() in front of mkdir(). This is because my tests (German) resulted bottlenecks using mkdir() alone.
Here is my "PHP flock() alternative" - build on
mkdir().
The idea to do it with mkdir() came from here and here.
My version
- checks if I already have got lock-access. It also prevents blocking myself if I create and use the class multiple times for the same basedir.name
- checks if my locking-file, with which I am asking for lock-access, was created
- lets me get lock-access in the order I came to ask for it
- stops waiting and looping if it could not get lock-access in the time I specified
- removes dead lock-files (= files where the SID of the PID does not exist any more)
You can use the PHP-class like this:
//$dir (string) = base-directory for the lock-files (with 'files' I mean directories => mode 0644)
// 2 (float/int) = time to wait for lock-access before returning unsuccessful (default is 0 <= try once and return)
//'.my_lock' (string) = the way you want to name your locking-dirs (default is '.fLock')
$lock = new FileLock($dir, 2, '.my_lock');
//start lock - a locking directory will be created looking like this:
//$dir/.my_lock-1536166146.4997-22796
if ($lock->lock()) {
//open your file - modify it - write it back
} else { /* write alert-email to admin */ }
//check if I had locked before
if ($lock->is_locked) { /* do something else with your locked file */ }
//unlock - the created dir will be removed (rmdir)
$lock->unlock();
Here is the working class:
//build a file-locking class
define('LOCKFILE_NONE', 0);
define('LOCKFILE_LOCKED', 1);
define('LOCKFILE_ALREADY_LOCKED', 2);
define('LOCKFILE_ALREADY_LOCKED_IN_OTHER_CLASS', 3);
define('LOCKFILE_FAILED_TO_OBTAIN_LOCK', false);
define('LOCKFILE_FAILED_TO_OBTAIN_LOCK_BY_TIMEOUT', '');
class FileLock {
//FileLock assumes that there are no other directories or files in the
//lock-base-directory named "$name-(float)-(int)"
//FileLock uses mkdir() to lock. Why?
//- mkdir() is atomic, so the lock is atomic and faster then saving files.
// Apparently it is faster than flock(), that requires several calls to the
// file system.
//- flock() depends on the system, mkdir() works everywhere.
private static $locked_memory = array();
public function __construct($lockbasedir, $wait_sec=0, $name='.fLock') {
$this->lockbasedir = (string)$lockbasedir;
$this->wait = (float)$wait_sec;
$this->name = (string)$name;
$this->pid = (int)getmypid();
//if this basedir.name was locked before and is still locked don't try to lock again
$this->is_locked = empty(self::$locked_memory[$this->lockbasedir . $this->name]) ? LOCKFILE_NONE : LOCKFILE_ALREADY_LOCKED;
}
public function lock() {
if ($this->is_locked) return $this->is_locked;
$break_time = microtime(true);
//create the directory as lock-file NOW
$this->lockdir = "{$this->name}-" . number_format($break_time, 4, '.', '') . "-{$this->pid}";
@mkdir("{$this->lockbasedir}/{$this->lockdir}", 0644);
$break_time += $this->wait;
//try to get locked
while ($this->wait == 0 || microtime(true) < $break_time) {
//get all locks with $this->name
$files = preg_grep("/^{$this->name}-\d+\.\d+-\d+$/", scandir($this->lockbasedir));
//since scandir() is sorted asc by default
//$first_file is the next directory to obtain lock
$first_file = reset($files);
if (!$first_file) {
//no lock-files at all
return $this->is_locked = LOCKFILE_FAILED_TO_OBTAIN_LOCK;
} elseif ($first_file == $this->lockdir) {
//Its me!! I'm getting locked :)
self::$locked_memory[$this->lockbasedir . $this->name] = 1;
return $this->is_locked = LOCKFILE_LOCKED;
} elseif (preg_match("/^{$this->name}-\d+\.\d+-{$this->pid}$/", $first_file)) {
//my process-ID already locked $this->name in another class before
rmdir("{$this->lockbasedir}/{$this->lockdir}");
$this->lockdir = $first_file;
self::$locked_memory[$this->lockbasedir . $this->name] = 1;
return $this->is_locked = LOCKFILE_ALREADY_LOCKED_IN_OTHER_CLASS;
}
//missing lock-file for this job
if (array_search($this->lockdir, $files) === false) return LOCKFILE_FAILED_TO_OBTAIN_LOCK;
//run only once
if ($this->wait == 0) break;
//check if process at first place has died
if (!posix_getsid(explode('-', $first_file)[2])) {
//remove dead lock
@rmdir("{$this->lockbasedir}/$first_file");
} else {
//wait and try again after 0.1 seconds
usleep(100000);
}
}
return $this->is_locked = LOCKFILE_FAILED_TO_OBTAIN_LOCK_BY_TIMEOUT;
}
public function unlock($force=false) {
if ($force || $this->is_locked == 1) {
rmdir("{$this->lockbasedir}/{$this->lockdir}");
self::$locked_memory[$this->lockbasedir . $this->name] = $this->is_locked = LOCKFILE_NONE;
}
}
}
I appreciate this question is a few years old but I kinda felt that a working example/replacement for flock might be worth building. I've based this on the other answers but for someone who is purely looking to replace the flock functionality (rather than write a file at the same time (although this does reflect the PHP manual flock example)) I believe the following will suffice
function my_flock ($path,$release = false){
if ($release){
@rmdir($path);
} else {
return !file_exists($path) && @mkdir($path);
}
}
Based on mkdir:
// call this always before reading or writing to your filepath in concurrent situations
function lockFile($filepath){
clearstatcache();
$lockname=$filepath.".lock";
// if the lock already exists, get its age:
$life=@filectime($lockname);
// attempt to lock, this is the really important atomic action:
while (!@mkdir($lockname)){
if ($life)
if ((time()-$life)>120){
//release old locks
rmdir($lockname);
}else $life=@filectime($lockname);
usleep(rand(50000,200000));//wait random time before trying again
}
}
To avoid deadlock when one script in case a script exits before it has unlocked and one (or more scripts) at the same time have no result on $life=@filectime($lockname); because all scripts starts at the same time and then directory isn't created yet. To unlock then call:
function unlockFile($filepath){
$unlockname= $filepath.".lock";
return @rmdir($unlockname);
}
None of these methods are fully atomic.
I have made some tests, confirming this.
The code for T7, using 7 files named by their size in kB:
clearstatcache();
$_DEBUG_ = false;
echo "Lock and flush tester.".time()."<br>";
$time_constant = 1570787996;
die; // Remove this line when you set time_constant
while ( time()<$time_constant )
{
usleep(500);
}
function test($n, $p, $_DEBUG_){
// $delay_multiplier = $n*2.5;
$sname = "$n"; // source
$tname = "$n.txt";// target
echo "<h4>$n at ".time()."</h4>";
for ($i = 0; $i<50; $i++ ){
$start = microtime(true);
clearstatcache(); // needed for filesize and touch
$st = stat("$sname");
$original_size = $st['size'];
if ( $_DEBUG_ )
echo "; 1) prevAccess by ".$st['mtime']." fsize ".$st['size']."; ";
$fsize = filesize($sname);
if ( $original_size <> $fsize )
die("; fsize total FAILTURE; ");
if ($fsize === 0)
echo "! <b>The fsize is 0</b>: stat(): ".$st['size']." ;";
else
{
// READ OPERATION AND LOCK FOR SHARE
$locked = false;
for ($c = 0; !$locked; $c++):
if ( $c > 400)
break;
$fp = fopen($sname, "r");
$locked = flock($fp, LOCK_SH);
if ($locked)
break;
else
{
echo "failed to get LOCK_SH;<br>";
usleep(5000);
}
endfor;
$s = fread($fp, $fsize );
$success = flock($fp, LOCK_UN);
if ( $success === false )
die("; r flock release failed; ");
$success = fclose($fp);
if ( $success === false )
die("; fclose failed; ");
// 10 - loaded data , $p - broser
if ( $success )
{
$result = touch("$sname",strlen($s),$p);
if ( $_DEBUG_ )
echo "; TOUCH: $result;";
}
else
die("fclose FAIL.");
if ( strlen($s)<60 )
echo "*$s LENGTH:".strlen($s)."<br>";
}
clearstatcache();
$st = stat("$tname");
if ( $_DEBUG_ )
echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; ";
// WRITE OPERATION WITH LOC_EX
$fp = fopen($tname, "w");
$locked = false;
/*
// TOTO NEMÁ VLIV NA ZAMKNUTÍ
for ($c = 0; !$locked; $c++ ):
$c++;
if ( $c > 400)
break;
$locked = flock($fp, LOCK_EX);
if ($locked)
break;
else
{
echo "failed to get LOCK_EX;<br>";
usleep(5000);
}
endfor;
*/
$locked = flock($fp, LOCK_EX);
if ( $locked ) { // acquire an exclusive lock
$success = fwrite($fp, $s);
if ( $success === false)
echo "; w FAILED;";
else
if ( $_DEBUG_ )
echo " $success B written; ";
$success = fflush($fp);// flush output before releasing the lock
if ( $success === false )
echo "; flush FAILED; ";
$success = flock($fp, LOCK_UN); // release the lock
if ( $success === false )
echo "; release FAILED; ";
$success = fclose($fp);
if ( $success === false )
echo "; fclose FAILED; ";
clearstatcache(); // needed for filesize and touch
$fsize = filesize($tname);
if ($original_size>$fsize)
{
echo "; <b>WRITE FAILED, restoring</b>;";
$original_fname = "$n";
$result = copy($original_fname, $tname);
if ($result == false )
die(" <b>TOTAL FAILTURE: copy failed.</b>");
else
echo " <b>RESTORED</b>;";
}
else
{
if ($fsize === 0)
echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp<br>";
if ( $success )
touch("$tname",$fsize,$p);
}
} else {
echo "Couldn't get the lock!";
}
$time_elapsed_secs = microtime(true) - $start;
//usleep( $delay_multiplier + $n*rand(2,6) );
if ( $time_elapsed_secs === 0 )
echo " FAILED ";
echo "time: $time_elapsed_secs s<br>";
}
}
// headers to identify originator of the request
switch ( $_SERVER['HTTP_USER_AGENT'] ):
// FF 1:
case "Mozilla/5.0 (Windows NT 5.1;) Gecko":
$p = 1; break;
// Chrome:
case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit Chrome Safari":
$p = 2; break;
// OPERA:
case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit Chrome Safari":
$p = 3; break;
endswitch;
copy("523","523.txt");
copy("948","948.txt");
copy("1371","1371.txt");
copy("1913","1913.txt");
copy("2701","2701.txt");
copy("4495","4495.txt");
copy("6758","6758.txt");
test("523",$p,$_DEBUG_);
test("948",$p,$_DEBUG_);
test("1371",$p,$_DEBUG_);
test("1913",$p,$_DEBUG_);
test("2701",$p,$_DEBUG_);
test("4495",$p,$_DEBUG_);
test("6758",$p,$_DEBUG_);
The code for T8 (mkdir lock test):
clearstatcache();
$_DEBUG_ = false;
echo "Atomicity tester.".time()."<br>";
$time_constant = 1570787996;
die; // Remove this line when you set time_constant
while ( time()<$time_constant )
{
usleep(500);
}
/*
c is counter for optimalization
first call must have c = 0;
*/
function atomicFuse($n, $c, $disableDelay = false){
$start = false;
if ( !file_exists("$n.t") )
$start = mkdir("$n.t");
if ( !$disableDelay ){
if ( $start == false )
{
$n = $n*30;
switch($c): // Delay example increase:
case 0: break; // 0,01569 total
case 1: break; // 0,03138 total
case 2: $n = $n*2; break; // 0,06276 total
case 3: $n = $n*4; break; // 0,12552 total
// case 4: You need at least *6 or *8 to get out of problems with extrem times
case 4: $n = $n*8; break; // 0,25104 t.(upper limit)
// In case of heavy traffic:
case 5: $n = $n*8; break; // 0,36087 total extrem
case 6: $n = $n*10; break; // 0,51777 total extrem
case 7: $n = $n*20; break; // 1,03554 total extrem
default: $n = $n*8; break;
endswitch;
usleep($n);
echo ($n)."<br>";
}
}
return $start;
}
function test($n, $p, $_DEBUG_){
$fp = null;
$sname = "$n"; // source
$tname = "$n.txt";// target
echo "<h4>$n at ".time()."</h4>";
for ($i = 0; $i<50; $i++ ){
$start_time = microtime(true);
{
$start = atomicFuse($n,0);
if (!$start) $start = atomicFuse($n,1);
if (!$start) $start = atomicFuse($n,2);
if (!$start) $start = atomicFuse($n,3);
if (!$start) $start = atomicFuse($n,4);
if (!$start) $start = atomicFuse($n,5);
if (!$start) $start = atomicFuse($n,6);
if (!$start) $start = atomicFuse($n,7);
if (!$start) $start = atomicFuse($n, false);
if (!$start) echo "<b>Atomicity failed.</b> ";
if ( $start )
{
echo "<b>Atomicity OK.</b> ";
/////////////////////////////
// CHECK FILESIZE VALIDITY //
/////////////////////////////
clearstatcache(); // needed for filesize and touch
$st = stat("$sname");
$original_size = $st['size'];
if ( $_DEBUG_ )
echo "; 1) prevAccess by ".$st['mtime']." fsize ".$st['size']."; ";
$fsize = filesize($sname);
if ( $original_size <> $fsize )
die("; fsize total FAILTURE; ");
if ($fsize === 0)
echo "! <b>The fsize is 0</b>: stat(): ".$st['size']." ;";
///////////////////
// OPEN THE FILE //
///////////////////
$fp = fopen($sname, "r");
$s = fread($fp, $fsize );
$success = fclose($fp);
if ( $success === false )
die("; fclose failed; ");
// 10 - loaded data, $p - browser
if ( $success )
{
$result = touch("$sname",strlen($s),$p);
if ( $_DEBUG_ )
echo "; TOUCH: $result;";
}
else
die("fclose FAIL.");
if ( strlen($s)<60 )
echo "*$s LENGTH:".strlen($s)."<br>";
}
}
if ( $start )
{
clearstatcache();
$st = stat("$tname");
if ( $_DEBUG_ )
echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; ";
// WRITE OPERATION WITH LOC_EX
$fp = fopen($tname, "w");
if ( true ) { // acquire an exclusive lock
$success = fwrite($fp, $s);
if ( $success === false)
echo "; w FAILED;";
else
if ( $_DEBUG_ )
echo " $success B written; ";
$success = fflush($fp);// flush output before releasing the lock
if ( $success === false )
echo "; flush FAILED; ";
if ( $success === false )
echo "; release FAILED; ";
$success = fclose($fp);
if ( $success === false )
echo "; fclose FAILED; ";
clearstatcache(); // needed for filesize and touch
$fsize = filesize($tname);
if ($original_size>$fsize)
{
echo "; <b>WRITE FAILED, restoring</b>;";
$original_fname = "$n";
$result = copy($original_fname, $tname);
if ($result == false )
die(" <b>TOTAL FAILTURE: copy failed.</b>");
else
echo " <b>RESTORED</b>;";
}
else
{
if ($fsize === 0)
echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp<br>";
if ( $success )
touch("$tname",$fsize,$p);
}
} else {
echo "Couldn't get the lock!";
}
$success = rmdir("$n.t"); // remove atomic fuse
if ( $success )
echo "<h4>DIR REMOVED</h4>";
else
echo "<h4>DIR NOT REMOVED</h4>";
} // start
else
echo "skipped";
$time_elapsed_secs = microtime(true) - $start_time;
if ( $time_elapsed_secs === 0 )
echo " FAILED ";
echo "time: $time_elapsed_secs s<br>";
} // for
}
switch ( $_SERVER['HTTP_USER_AGENT'] ):
case "":
$p = 1; break;
case "":
$p = 2; break;
case "":
$p = 3; break;
endswitch;
copy("523","523.txt");
copy("948","948.txt");
copy("1371","1371.txt");
copy("1913","1913.txt");
copy("2701","2701.txt");
copy("4495","4495.txt");
copy("6758","6758.txt");
test("523",$p,$_DEBUG_);
test("948",$p,$_DEBUG_);
test("1371",$p,$_DEBUG_);
test("1913",$p,$_DEBUG_);
test("2701",$p,$_DEBUG_);
test("4495",$p,$_DEBUG_);
test("6758",$p,$_DEBUG_);
Note: T5-T7 - I did not determinated whether the file damages were made by fflush or fwrite, but it was in these tests, where these error occures.
Note: T8 - Specific problem with this test is, that it often waits too long on begin of a testing block (on the begin of the testing function). There even delays like 7 seconds waiting. But I also tried to remove these numbers and the avarage does not change too much, so the curve of T8 would stay the same after this change. The problem here is that using the delay in a loop is not ideal solution of the problem, it makes the probability of failture even higher. Note, that by "failture" I do not really mean file corruption but skipping of the given atomic task because time out.
加载中,请稍侯......
精彩评论