Is there a build tool based on inotify-like mechanism

In relatively big projects which are using plain old make, even building the project when nothing has changed takes a few tens of seconds. Especially with many executions of make -C, which have the new process overhea开发者_运维百科d.

The obvious solution to this problem is a build tool based on inotify-like feature of the OS. It would look out when a certain file is changed, and based on that list it would compile this file alone.

Is there such machinery out there? Bonus points for open source projects.

You mean like Tup:

From the home page:

"Tup is a file-based build system - it inputs a list of file changes and a directed acyclic graph (DAG), then processes the DAG to execute the appropriate commands required to update dependent files. The DAG is stored in an SQLite database. By default, the list of file changes is generated by scanning the filesystem. Alternatively, the list can be provided up front by running the included file monitor daemon."

I am just wondering if it is stat()ing the files that takes so long. To check this here is a small systemtap script I wrote to measure the time it takes to stat() files:

# call-counts.stp

global calls, times

probe kernel.function(@1) {
    times[probefunc()] = gettimeofday_ns()
}

probe kernel.function(@1).return {
    now = gettimeofday_ns()
    delta = now - times[probefunc()]
    calls[probefunc()] <<< delta
}

And then use it like this:

$ stap -c "make -rC ~/src/prj -j8 -k" ~/tmp/count-calls.stp sys_newstat
make: Entering directory `/home/user/src/prj'
make: Nothing to be done for `all'.
make: Leaving directory `/home/user/src/prj'
calls["sys_newstat"] @count=8318 @min=684 @max=910667 @sum=26952500 @avg=3240

The project I ran it upon has 4593 source files and it takes ~27msec (26952500nsec above) for make to stat all the files along with the corresponding .d files. I am using non-recursive make though.

If you're using OSX, you can use fswatch

https://github.com/alandipert/fswatch

Here's how to use fswatch to for changes to a file and then run make if it detects any

fswatch -o anyFile | xargs -n1 -I{} make

You can run fswatch from inside a makefile like this:

watch: $(FILE)
  fswatch -o $^ | xargs -n1 -I{} make

(Of course, $(FILE) is defined inside the makefile.) make can now watch for changes in the file like this:

> make watch

You can watch another file like this:

> make watch anotherFile

Install inotify-tools and write a few lines of bash to invoke make when certain directories are updated.

As a side note, recursive make scales badly and is error prone. Prefer non-recursive make.

The change-dependency you describe is already part of Make, but Make is flexible enough that it can be used in an inefficient way. If the slowness really is caused by the recursion (make -C commands) -- which it probably is -- then you should reduce the recursion. (You could try putting in your own conditional logic to decide whether to execute make -C, but that would be a very inelegant solution.)

Roughly speaking, if your makefiles look like this

# main makefile

foo:
    make -C bar baz

and this

# makefile in bar/

baz: quartz
    do something

you can change them to this:

# main makefile

foo: bar/quartz
    cd bar && do something

There are many details to get right, but now if bar/quartz has not been changed, the foo rule will not run.