parsing multithreading make's output (-j N)

I have a lot of source directories in common directory. When I start make by issuing command:

make -j 4

I receive a lot of strings from make's threads along with invoked gcc compiler instances. For parsing errors I have to run make twice, second time with one thread:

make -j 1

so I can correctly parse make's output.

Is there a way for running multithreaded make one time and correctly decide which error related to which project (source directory)?

Thank you!

If you are using recursive make (naughty boy) together with -j, then you can wrap Make with a shell script which prefixes each line of output with a unique per-make-invocation string.

$ cat M
#!/bin/bash
PREFIX=$$:
exec -a $0 make "$@" 2>&1 | sed "s/^/$PREFIX/"

Now, assuming your makefiles correctly use ${MAKE} to indicate recursion, we can use ./M instead of make.

$ ./M -j --no-print-directory target
28720:/home/user/M -fa.mak
28720:/home/user/M -fb.mak
28720:/home/user/M -fc.mak
28720:/home/user/M -fd.mak
28720:/home/user/M -fe.mak
28720:32484:gcc blah...
28720:31936:/home/user/M -fanother.mak
28720:32484:gcc blah...
28720:31936:gcc blah...
28720:31936:gcc blah...
28720:31936:56781:echo blah...
∶

In this case, each line is prefixed with a list of process IDs (good for debugging recursive make). For your use case, you may prefer M to mangle source file names so that they become absolute pathnames in error messages.

May be it is possible to switch to pmake?

PMake is set up to handle the output from multiple jobs in a graceful fashion (source)

If you're using some kind of meta-build system (eg CMake) try using Ninja to actually run the build.

It solves this problem as well as being quite a lot faster.