12. Portable Make Programming

Writing portable makefiles is an art. Since a makefile's commands are executed by the shell, you must consider the shell portability issues already mentioned. However, other issues are specific to make itself.

12.1 $< in Ordinary Make Rules

Posix says that the `$<' construct in makefiles can be used only in inference rules and in the `.DEFAULT' rule; its meaning in ordinary rules is unspecified. Solaris make for instance replaces it with the empty string. OpenBSD (3.0 and later) make diagnoses these uses and errors out.

12.2 Failure in Make Rules

Posix 2008 requires that make must invoke each command with the equivalent of a `sh -e -c' subshell, which causes the subshell to exit immediately if a subsidiary simple-command fails, although not all make implementations have historically followed this rule. For example, the command `touch T; rm -f U' may attempt to remove `U' even if the touch fails, although this is not permitted with Posix make. One way to work around failures in simple commands is to reword them so that they always succeed, e.g., `touch T || :; rm -f U'. However, even this approach can run into common bugs in BSD implementations of the `-e' option of sh and set (see Limitations of Shell Builtins), so if you are worried about porting to buggy BSD shells it may be simpler to migrate complicated make actions into separate scripts.

12.3 Special Characters in Make Macro Names

Posix limits macro names to nonempty strings containing only ASCII letters and digits, `.', and `_'. Many make implementations allow a wider variety of characters, but portable makefiles should avoid them. It is portable to start a name with a special character, e.g., `$(.FOO)'.

Some ancient make implementations don't support leading underscores in macro names. An example is NEWS-OS 4.2R.

$ cat Makefile
_am_include = #
_am_quote =
all:; @echo this is test
$ make
Make: Must be a separator on rules line 2.  Stop.
$ cat Makefile2
am_include = #
am_quote =
all:; @echo this is test
$ make -f Makefile2
this is test

However, this problem is no longer of practical concern.

12.4 Backslash-Newline-Newline in Make Macro Values

On some versions of HP-UX, make reads multiple newlines following a backslash, continuing to the next non-empty line. For example,

FOO = one \

BAR = two

test:
        : FOO is "$(FOO)"
        : BAR is "$(BAR)"

shows FOO equal to one BAR = two. Other implementations sensibly let a backslash continue only to the immediately following line.

12.5 Backslash-Newline in Make Comments

According to Posix, Make comments start with # and continue until an unescaped newline is reached.

$ cat Makefile
# A = foo \
      bar \
      baz

all:
        @echo ok
$ make   # GNU make
ok

However this is not always the case. Some implementations discard everything from # through the end of the line, ignoring any trailing backslash.

$ pmake  # BSD make
"Makefile", line 3: Need an operator
Fatal errors encountered -- cannot continue

Therefore, if you want to comment out a multi-line definition, prefix each line with #, not only the first.

# A = foo \
#     bar \
#     baz

12.6 Long Lines in Makefiles

Tru64 5.1's make has been reported to crash when given a makefile with lines longer than around 20 kB. Earlier versions are reported to exit with Line too long diagnostics.

12.7 make macro=value and Submakes

A command-line variable definition such as foo=bar overrides any definition of foo in a makefile. Some make implementations (such as GNU make) propagate this override to subsidiary invocations of make. Some other implementations do not pass the substitution along to submakes.

$ cat Makefile
foo = foo
one:
        @echo $(foo)
        $(MAKE) two
two:
        @echo $(foo)
$ make foo=bar            # GNU make 3.79.1
bar
make two
make[1]: Entering directory `/home/adl'
bar
make[1]: Leaving directory `/home/adl'
$ pmake foo=bar           # BSD make
bar
pmake two
foo

You have a few possibilities if you do want the foo=bar override to propagate to submakes. One is to use the `-e' option, which causes all environment variables to have precedence over the makefile macro definitions, and declare foo as an environment variable:

$ env foo=bar make -e

The `-e' option is propagated to submakes automatically, and since the environment is inherited between make invocations, the foo macro is overridden in submakes as expected.

This syntax (foo=bar make -e) is portable only when used outside of a makefile, for instance from a script or from the command line. When run inside a make rule, GNU make 3.80 and prior versions forget to propagate the `-e' option to submakes.

Moreover, using `-e' could have unexpected side effects if your environment contains some other macros usually defined by the makefile. (See also the note about make -e and SHELL below.)

Another way to propagate overrides to submakes is to do it manually, from your makefile:

foo = foo
one:
        @echo $(foo)
        $(MAKE) foo=$(foo) two
two:
        @echo $(foo)

You need to foresee all macros that a user might want to override if you do that.

Makefiles generated by automake expand $(AM_MAKEFLAGS) on the command line of submakes, which can be used for propagated overrides (see (automake)Subdirectories section `Automake' in GNU Automake).

12.8 The Make Macro MAKEFLAGS

Posix requires make to use MAKEFLAGS to affect the current and recursive invocations of make, but allows implementations several formats for the variable. It is tricky to parse $MAKEFLAGS to determine whether `-s' for silent execution or `-k' for continued execution are in effect. For example, you cannot assume that the first space-separated word in $MAKEFLAGS contains single-letter options, since in the Cygwin version of GNU make it is either `--unix' or `--win32' with the second word containing single-letter options.

$ cat Makefile
all:
        @echo MAKEFLAGS = $(MAKEFLAGS)
$ make
MAKEFLAGS = --unix
$ make -k
MAKEFLAGS = --unix -k

12.9 The Make Macro SHELL

Posix-compliant make internally uses the $(SHELL) macro to spawn shell processes and execute Make rules. This is a builtin macro supplied by make, but it can be modified by a makefile or by a command-line argument.

Not all make implementations define this SHELL macro. Tru64 make is an example; this implementation always uses /bin/sh. So it's a good idea to always define SHELL in your makefiles. If you use Autoconf, do

SHELL = @SHELL@

If you use Automake, this is done for you.

Do not force SHELL = /bin/sh because that is not correct everywhere. Remember, `/bin/sh' is not Posix compliant on many systems, such as FreeBSD 4, NetBSD 3, AIX 3, Solaris 10, or Tru64. Additionally, DJGPP lacks /bin/sh, and when its GNU make port sees such a setting it enters a special emulation mode where features like pipes and redirections are emulated on top of DOS's command.com. Unfortunately this emulation is incomplete; for instance it does not handle command substitutions. Using @SHELL@ means that your makefile will benefit from the same improved shell, such as bash or ksh, that was discovered during configure, so that you aren't fighting two different sets of shell bugs between the two contexts.

Posix-compliant make should never acquire the value of $(SHELL) from the environment, even when make -e is used (otherwise, think about what would happen to your rules if SHELL=/bin/tcsh).

However not all make implementations have this exception. For instance it's not surprising that Tru64 make doesn't protect SHELL, since it doesn't use it.

$ cat Makefile
SHELL = /bin/sh
FOO = foo
all:
        @echo $(SHELL)
        @echo $(FOO)
$ env SHELL=/bin/tcsh FOO=bar make -e   # Tru64 Make
/bin/tcsh
bar
$ env SHELL=/bin/tcsh FOO=bar gmake -e  # GNU make
/bin/sh
bar

Conversely, make is not supposed to export any changes to the macro SHELL to child processes. Again, many implementations break this rule:

$ cat Makefile
all:
        @echo $(SHELL)
        @printenv SHELL
$ env SHELL=sh make -e SHELL=/bin/ksh   # BSD Make, GNU make 3.80
/bin/ksh
/bin/ksh
$ env SHELL=sh gmake -e SHELL=/bin/ksh  # GNU make 3.81
/bin/ksh
sh

12.10 Parallel Make

Support for parallel execution in make implementation varies. Generally, using GNU make is your best bet. When NetBSD make is invoked with `-jN', it will reuse the same shell for multiple commands within one recipe. This can have unexpected consequences.(5) For example, change of directories or variables persist between commands:

all:
        @var=value; cd /; pwd; echo $$var; echo $$$$
        @pwd; echo $$var; echo $$$$

may output the following with make -j1:

--- all ---
/
value
32235
/
value
32235

while without `-j1', or with `-B', the output looks less surprising:

/
value
32238
/tmp

32239

Another consequence of this is that, if one command in a recipe uses exit 0 to indicate a successful exit, the shell will be gone and the remaining commands of this recipe will not be executed.

The above example also shows additional status output NetBSD make produces in parallel mode for targets being updated.

Furthermore, parallel NetBSD make will route standard error from commands that it spawns into its own standard output, and may remove leading whitespace from output lines.

You can avoid these issues by using the `-B' option to enable compatibility semantics. However, that will effectively also disable all parallelism as that will cause prerequisites to be updated in the order they are listed in a rule.

12.11 Comments in Make Rules

Never put comments in a rule.

Some make treat anything starting with a tab as a command for the current rule, even if the tab is immediately followed by a #. The make from Tru64 Unix V5.1 is one of them. The following makefile runs # foo through the shell.

all:
        # foo

12.12 The `obj/' Subdirectory and Make

Never name one of your subdirectories `obj/' if you don't like surprises.

If an `obj/' directory exists, BSD make enters it before reading the makefile. Hence the makefile in the current directory is not read.

$ cat Makefile
all:
        echo Hello
$ cat obj/Makefile
all:
        echo World
$ make      # GNU make
echo Hello
Hello
$ pmake     # BSD make
echo World
World

12.13 Exit Status of make -k

Do not rely on the exit status of make -k. Some implementations reflect whether they encountered an error in their exit status; other implementations always succeed.

$ cat Makefile
all:
        false
$ make -k; echo exit status: $?    # GNU make
false
make: *** [all] Error 1
exit status: 2
$ pmake -k; echo exit status: $?   # BSD make
false
*** Error code 1 (continuing)
exit status: 0

12.14 VPATH and Make

Posix does not specify the semantics of VPATH. Typically, make supports VPATH, but its implementation is not consistent.

Autoconf and Automake support makefiles whose usages of VPATH are portable to recent-enough popular implementations of make, but to keep the resulting makefiles portable, a package's makefile prototypes must take the following issues into account. These issues are complicated and are often poorly understood, and installers who use VPATH should expect to find many bugs in this area. If you use VPATH, the simplest way to avoid these portability bugs is to stick with GNU make, since it is the most commonly-used make among Autoconf users.

Here are some known issues with some VPATH implementations.

12.14.1 Variables listed in VPATH

Do not set VPATH to the value of another variable, for example `VPATH = $(srcdir)', because some ancient versions of make do not do variable substitutions on the value of VPATH. For example, use this

srcdir = @srcdir@
VPATH = @srcdir@

rather than `VPATH = $(srcdir)'. Note that with GNU Automake, there is no need to set this yourself.

12.14.2 VPATH and Double-colon Rules

With ancient versions of Sun make, any assignment to VPATH causes make to execute only the first set of double-colon rules. However, this problem is no longer of practical concern.

12.14.3 $< Not Supported in Explicit Rules

Using $< in explicit rules is not portable. The prerequisite file must be named explicitly in the rule. If you want to find the prerequisite via a VPATH search, you have to code the whole thing manually. See section Build Directories.

12.14.4 Automatic Rule Rewriting

Some make implementations, such as Solaris and Tru64, search for prerequisites in VPATH and then rewrite each occurrence as a plain word in the rule. For instance:

# This isn't portable to GNU make.
VPATH = ../pkg/src
f.c: if.c
        cp if.c f.c

executes cp ../pkg/src/if.c f.c if `if.c' is found in `../pkg/src'.

However, this rule leads to real problems in practice. For example, if the source directory contains an ordinary file named `test' that is used in a dependency, Solaris make rewrites commands like `if test -r foo; …' to `if ../pkg/src/test -r foo; …', which is typically undesirable. To avoid this problem, portable makefiles should never mention a source file whose name is that of a shell keyword like `until' or a shell command like cat or gcc or test.

Because of these problems GNU make and many other make implementations do not rewrite commands, so portable makefiles should search VPATH manually. It is tempting to write this:

# This isn't portable to Solaris make.
VPATH = ../pkg/src
f.c: if.c
        cp `test -f if.c || echo $(VPATH)/`if.c f.c

However, the "prerequisite rewriting" still applies here. So if `if.c' is in `../pkg/src', Solaris and Tru64 make execute

cp `test -f ../pkg/src/if.c || echo ../pkg/src/`if.c f.c

which reduces to

cp if.c f.c

and thus fails. Oops.

A simple workaround, and good practice anyway, is to use `$?' and `$@' when possible:

VPATH = ../pkg/src
f.c: if.c
        cp $? $@

but this does not generalize well to commands with multiple prerequisites. A more general workaround is to rewrite the rule so that the prerequisite `if.c' never appears as a plain word. For example, these three rules would be safe, assuming `if.c' is in `../pkg/src' and the other files are in the working directory:

VPATH = ../pkg/src
f.c: if.c f1.c
        cat `test -f ./if.c || echo $(VPATH)/`if.c f1.c >$@
g.c: if.c g1.c
        cat `test -f 'if.c' || echo $(VPATH)/`if.c g1.c >$@
h.c: if.c h1.c
        cat `test -f "if.c" || echo $(VPATH)/`if.c h1.c >$@

Things get worse when your prerequisites are in a macro.

VPATH = ../pkg/src
HEADERS = f.h g.h h.h
install-HEADERS: $(HEADERS)
        for i in $(HEADERS); do \
          $(INSTALL) -m 644 \
            `test -f $$i || echo $(VPATH)/`$$i \
            $(DESTDIR)$(includedir)/$$i; \
        done

The above install-HEADERS rule is not Solaris-proof because for i in $(HEADERS); is expanded to for i in f.h g.h h.h; where f.h and g.h are plain words and are hence subject to VPATH adjustments.

If the three files are in `../pkg/src', the rule is run as:

for i in ../pkg/src/f.h ../pkg/src/g.h h.h; do \
  install -m 644 \
     `test -f $i || echo ../pkg/src/`$i \
     /usr/local/include/$i; \
done

where the two first install calls fail. For instance, consider the f.h installation:

install -m 644 \
  `test -f ../pkg/src/f.h || \
    echo ../pkg/src/ \
  `../pkg/src/f.h \
  /usr/local/include/../pkg/src/f.h;

It reduces to:

install -m 644 \
  ../pkg/src/f.h \
  /usr/local/include/../pkg/src/f.h;

Note that the manual VPATH search did not cause any problems here; however this command installs `f.h' in an incorrect directory.

Trying to quote $(HEADERS) in some way, as we did for foo.c a few makefiles ago, does not help:

install-HEADERS: $(HEADERS)
        headers='$(HEADERS)'; \
        for i in $$headers; do \
          $(INSTALL) -m 644 \
            `test -f $$i || echo $(VPATH)/`$$i \
            $(DESTDIR)$(includedir)/$$i; \
        done

Now, headers='$(HEADERS)' macro-expands to:

headers='f.h g.h h.h'

but g.h is still a plain word. (As an aside, the idiom headers='$(HEADERS)'; for i in $$headers; is a good idea if $(HEADERS) can be empty, because some shells diagnose a syntax error on for i in;.)

One workaround is to strip this unwanted `../pkg/src/' prefix manually:

VPATH = ../pkg/src
HEADERS = f.h g.h h.h
install-HEADERS: $(HEADERS)
        headers='$(HEADERS)'; \
        for i in $$headers; do \
          i=`expr "$$i" : '$(VPATH)/\(.*\)'`;
          $(INSTALL) -m 644 \
            `test -f $$i || echo $(VPATH)/`$$i \
            $(DESTDIR)$(includedir)/$$i; \
        done

Automake does something similar. However the above hack works only if the files listed in HEADERS are in the current directory or a subdirectory; they should not be in an enclosing directory. If we had HEADERS = ../f.h, the above fragment would fail in a VPATH build with Tru64 make. The reason is that not only does Tru64 make rewrite dependencies, but it also simplifies them. Hence ../f.h becomes ../pkg/f.h instead of ../pkg/src/../f.h. This obviously defeats any attempt to strip a leading `../pkg/src/' component.

The following example makes the behavior of Tru64 make more apparent.

$ cat Makefile
VPATH = sub
all: ../foo
        echo ../foo
$ ls
Makefile foo
$ make
echo foo
foo

Dependency `../foo' was found in `sub/../foo', but Tru64 make simplified it as `foo'. (Note that the `sub/' directory does not even exist, this just means that the simplification occurred before the file was checked for.)

For the record here is how SunOS 4 make behaves on this example.

$ make
make: Fatal error: Don't know how to make target `../foo'
$ mkdir sub
$ make
echo sub/../foo
sub/../foo

12.14.5 Tru64 make Creates Prerequisite Directories Magically

When a prerequisite is a subdirectory of VPATH, Tru64 make creates it in the current directory.

$ mkdir -p foo/bar build
$ cd build
$ cat >Makefile <<END
VPATH = ..
all: foo/bar
END
$ make
mkdir foo
mkdir foo/bar

This can yield unexpected results if a rule uses a manual VPATH search as presented before.

VPATH = ..
all : foo/bar
        command `test -d foo/bar || echo ../`foo/bar

The above command is run on the empty `foo/bar' directory that was created in the current directory.

12.14.6 Make Target Lookup

GNU make uses a complex algorithm to decide when it should use files found via a VPATH search. See (make)Search Algorithm section `How Directory Searches are Performed' in The GNU Make Manual.

If a target needs to be rebuilt, GNU make discards the file name found during the VPATH search for this target, and builds the file locally using the file name given in the makefile. If a target does not need to be rebuilt, GNU make uses the file name found during the VPATH search.

Other make implementations, like NetBSD make, are easier to describe: the file name found during the VPATH search is used whether the target needs to be rebuilt or not. Therefore new files are created locally, but existing files are updated at their VPATH location.

OpenBSD and FreeBSD make, however, never perform a VPATH search for a dependency that has an explicit rule. This is extremely annoying.

When attempting a VPATH build for an autoconfiscated package (e.g., mkdir build && cd build && ../configure), this means GNU make builds everything locally in the `build' directory, while BSD make builds new files locally and updates existing files in the source directory.

$ cat Makefile
VPATH = ..
all: foo.x bar.x
foo.x bar.x: newer.x
        @echo Building $@
$ touch ../bar.x
$ touch ../newer.x
$ make        # GNU make
Building foo.x
Building bar.x
$ pmake       # NetBSD make
Building foo.x
Building ../bar.x
$ fmake       # FreeBSD make, OpenBSD make
Building foo.x
Building bar.x
$ tmake       # Tru64 make
Building foo.x
Building bar.x
$ touch ../bar.x
$ make        # GNU make
Building foo.x
$ pmake       # NetBSD make
Building foo.x
$ fmake       # FreeBSD make, OpenBSD make
Building foo.x
Building bar.x
$ tmake       # Tru64 make
Building foo.x
Building bar.x

Note how NetBSD make updates `../bar.x' in its VPATH location, and how FreeBSD, OpenBSD, and Tru64 make always update `bar.x', even when `../bar.x' is up to date.

Another point worth mentioning is that once GNU make has decided to ignore a VPATH file name (e.g., it ignored `../bar.x' in the above example) it continues to ignore it when the target occurs as a prerequisite of another rule.

The following example shows that GNU make does not look up `bar.x' in VPATH before performing the .x.y rule, because it ignored the VPATH result of `bar.x' while running the bar.x: newer.x rule.

$ cat Makefile
VPATH = ..
all: bar.y
bar.x: newer.x
        @echo Building $@
.SUFFIXES: .x .y
.x.y:
        cp $< $@
$ touch ../bar.x
$ touch ../newer.x
$ make        # GNU make
Building bar.x
cp bar.x bar.y
cp: cannot stat `bar.x': No such file or directory
make: *** [bar.y] Error 1
$ pmake       # NetBSD make
Building ../bar.x
cp ../bar.x bar.y
$ rm bar.y
$ fmake       # FreeBSD make, OpenBSD make
echo Building bar.x
cp bar.x bar.y
cp: cannot stat `bar.x': No such file or directory
*** Error code 1
$ tmake       # Tru64 make
Building bar.x
cp: bar.x: No such file or directory
*** Exit 1

Note that if you drop away the command from the bar.x: newer.x rule, GNU make magically starts to work: it knows that bar.x hasn't been updated, therefore it doesn't discard the result from VPATH (`../bar.x') in succeeding uses. Tru64 also works, but FreeBSD and OpenBSD still don't.

$ cat Makefile
VPATH = ..
all: bar.y
bar.x: newer.x
.SUFFIXES: .x .y
.x.y:
        cp $< $@
$ touch ../bar.x
$ touch ../newer.x
$ make        # GNU make
cp ../bar.x bar.y
$ rm bar.y
$ pmake       # NetBSD make
cp ../bar.x bar.y
$ rm bar.y
$ fmake       # FreeBSD make, OpenBSD make
cp bar.x bar.y
cp: cannot stat `bar.x': No such file or directory
*** Error code 1
$ tmake       # Tru64 make
cp ../bar.x bar.y

It seems the sole solution that would please every make implementation is to never rely on VPATH searches for targets. In other words, VPATH should be reserved to unbuilt sources.

12.15 Single Suffix Rules and Separated Dependencies

A Single Suffix Rule is basically a usual suffix (inference) rule (`.from.to:'), but which destination suffix is empty (`.from:').

Separated dependencies simply refers to listing the prerequisite of a target, without defining a rule. Usually one can list on the one hand side, the rules, and on the other hand side, the dependencies.

Solaris make does not support separated dependencies for targets defined by single suffix rules:

$ cat Makefile
.SUFFIXES: .in
foo: foo.in
.in:
        cp $< $@
$ touch foo.in
$ make
$ ls
Makefile  foo.in

while GNU Make does:

$ gmake
cp foo.in foo
$ ls
Makefile  foo       foo.in

Note it works without the `foo: foo.in' dependency.

$ cat Makefile
.SUFFIXES: .in
.in:
        cp $< $@
$ make foo
cp foo.in foo

and it works with double suffix inference rules:

$ cat Makefile
foo.out: foo.in
.SUFFIXES: .in .out
.in.out:
        cp $< $@
$ make
cp foo.in foo.out

As a result, in such a case, you have to write target rules.

12.16 Timestamp Resolution and Make

Traditionally, file timestamps had 1-second resolution, and make used those timestamps to determine whether one file was newer than the other. However, many modern file systems have timestamps with 1-nanosecond resolution. Some make implementations look at the entire timestamp; others ignore the fractional part, which can lead to incorrect results. Normally this is not a problem, but in some extreme cases you may need to use tricks like `sleep 1' to work around timestamp truncation bugs.

Commands like `cp -p' and `touch -r' typically do not copy file timestamps to their full resolutions (see Limitations of Usual Tools). Hence you should be wary of rules like this:

dest: src
        cp -p src dest

as `dest' often appears to be older than `src' after the timestamp is truncated, and this can cause make to do needless rework the next time it is invoked. To work around this problem, you can use a timestamp file, e.g.:

dest-stamp: src
        cp -p src dest
        date >dest-stamp

This document was generated on January, 20 2010 using texi2html 1.76.