Is there a faster alternative to Perl's stat?
I'm traversing an entire partition, stat()
'ing each file and then checking the returned values for mtime, size and uid against hashed values. stat()
however is far too slow in Perl and I'm wondering if there's any qui开发者_StackOverflow社区cker alternatives I may be overlooking.
When you call stat
you're querying the filesystem and will be limited by its performance. For large numbers of files this will be slow; it's not really a Perl issue.
Before you go off optimizing stat, use Devel::NYTProf to see where the real slow-down is.
Also, investigate the details of how you've mounted the filesystem. Is everything local, or have you mounted something over NFS or something similar? There are many things that can be the problem, as other answers have pointed out. Don't spend too much time focussing on any potential problem until you know it's the problem.
Good luck,
stat
is doing IO on each file which can't be avoided if you're wanting to read those data. So that'll be the limit on speed and can't be worked around any other way that I can think of.
If you're repeatedly stat
-ing the same file(s) then consider using Memoize
.
use Memoize();
sub fileStat {
my ($filename) = @_;
return stat($filename);
}
Memoize::memoize('fileStat');
You've seen that stat
is slow enough as it is, so don't call it more than once on the same file.
The perlfunc documentation on -X (the shell-ish file test operators) describes a nice cache for stat
:
If any of the file tests (or either the
stat
orlstat
operators) are given the special filehandle consisting of a solitary underline, then the stat structure of the previous file test (or stat operator) is used, saving a system call. (This doesn't work with-t
, and you need to remember thatlstat
and-l
will leave values in the stat structure for the symbolic link, not the real file.) (Also, if the stat buffer was filled by anlstat
call,-T
and-B
will reset it with the results ofstat _
). Example:print "Can do.\n" if -r $a || -w _ || -x _; stat($filename); print "Readable\n" if -r _; print "Writable\n" if -w _; print "Executable\n" if -x _; print "Setuid\n" if -u _; print "Setgid\n" if -g _; print "Sticky\n" if -k _; print "Text\n" if -T _; print "Binary\n" if -B _;
Consider File::Find module.
- If you are on *NIX, you can just use
ls
and parse the output, I should think. - As Ether mentioned,
find
is possibly a good alternative if you just want to make decisions on what you stat. - But size, date, and uid should all be available from
ls
output. - While date and size are available from the
dir
command on a Windows platform.
精彩评论