Getting nfsstat-like statistics for ext3 and other filesystems

For those not familiar with nfsstat: it basically just counts the number o开发者_开发问答f calls to specific functions. This way one can get statistics about NFS I/O operations and thus use them to analyze performance.

I would like to do something like that in a more general fashion. I'd like to somehow count the number of e.g. "stat" calls. If possible a count per mount point. I am however not sure where to start. I have sufficient programming skills using C against the libc library, but lacking knowlegde of the Linux kernel.

I feel this is supposed to be done via a loadable kernel module but I'm not sure the kernel API provides the possibilities to hook into those kinds of system calls. Or should I use inotify for this (doesn't monitor stat calls)?

Where to start?

To watch all the system calls happening on the system, you can use the Linux audit subsystem to watch for a specific syscall. You can set up filters such as restricting the watch to a directory tree. The documentation is rather sparse; start with the auditctl man page, or perhaps this tutorial. Most recent distributions ship an auditd package. Install it and make sure the auditd daemon is running, then do

auditctl -A exit,always -F dir=/mount/point -S stat

and watch the calls get logged in /var/log/audit/audit.log (or wherever your distribution has set this up).

At the other extreme, if you're only interested in the system calls made by a particular process (and optionally its subprocesses), use strace.

strace -s9999 -estrace -f mycommand

In between, if you want to watch some of the filesystem accesses of a bunch of programs, make them access the files you're concerned about through loggedfs. This is a stackable filesystem: it provides an alternate view of an existing directory tree. It can log every operation, has rich filters and comes with reasonable documentation.