Thursday, July 1, 2010

Makefiles and concurrency

Makefiles offer an option for parallel execution, but there is a stumbling block: it is possible to create race conditions that can lead to corrupt output files if two Makefiles want to make the same target.

For example:

Makefile:
all: a b

a:
        $(MAKE) -f Makefile.a

b:
        $(MAKE) -f Makefile.b
Makefile.a:
a: x
        echo A
        cat x > a

x:
        echo A makes X!
        sleep 3
        echo 'A' >> x

Makefile.b:
b: x
        echo B
        cat x > b

x:
        echo B makes X!
        sleep 2
        echo 'B' >> x

When run single-threaded, the file x just contains 'A'. However, in multi-threaded mode, it will contain both lines: 'B' and 'A'. So there is no "stack" of commands to be executed that gets checked for duplicated commands to make the same file. Hence, files that are prerequisites for the parallel parts need to be made first.