20. Frequent Autoconf Questions, with answers

Several questions about Autoconf come up occasionally. Here some of them are addressed.

20.1 Distributing configure Scripts

What are the restrictions on distributing configure
scripts that Autoconf generates?  How does that affect my
programs that use them?

There are no restrictions on how the configuration scripts that Autoconf produces may be distributed or used. In Autoconf version 1, they were covered by the GNU General Public License. We still encourage software authors to distribute their work under terms like those of the GPL, but doing so is not required to use Autoconf.

Of the other files that might be used with configure, `config.h.in' is under whatever copyright you use for your `configure.ac'. `config.sub' and `config.guess' have an exception to the GPL when they are used with an Autoconf-generated configure script, which permits you to distribute them under the same terms as the rest of your package. `install-sh' is from the X Consortium and is not copyrighted.

20.2 Why Require GNU M4?

Why does Autoconf require GNU M4?

Many M4 implementations have hard-coded limitations on the size and number of macros that Autoconf exceeds. They also lack several builtin macros that it would be difficult to get along without in a sophisticated application like Autoconf, including:

m4_builtin
m4_indir
m4_bpatsubst
__file__
__line__

Autoconf requires version 1.4.6 or later of GNU M4.

Since only software maintainers need to use Autoconf, and since GNU M4 is simple to configure and install, it seems reasonable to require GNU M4 to be installed also. Many maintainers of GNU and other free software already have most of the GNU utilities installed, since they prefer them.

20.3 How Can I Bootstrap?

If Autoconf requires GNU M4 and GNU M4 has an Autoconf
configure script, how do I bootstrap?  It seems like a chicken
and egg problem!

This is a misunderstanding. Although GNU M4 does come with a configure script produced by Autoconf, Autoconf is not required in order to run the script and install GNU M4. Autoconf is only required if you want to change the M4 configure script, which few people have to do (mainly its maintainer).

20.4 Why Not Imake?

Why not use Imake instead of configure scripts?

Several people have written addressing this question, so I include adaptations of their explanations here.

The following answer is based on one written by Richard Pixley:

Autoconf generated scripts frequently work on machines that it has never been set up to handle before. That is, it does a good job of inferring a configuration for a new system. Imake cannot do this.

Imake uses a common database of host specific data. For X11, this makes sense because the distribution is made as a collection of tools, by one central authority who has control over the database.

GNU tools are not released this way. Each GNU tool has a maintainer; these maintainers are scattered across the world. Using a common database would be a maintenance nightmare. Autoconf may appear to be this kind of database, but in fact it is not. Instead of listing host dependencies, it lists program requirements.

If you view the GNU suite as a collection of native tools, then the problems are similar. But the GNU development tools can be configured as cross tools in almost any host+target permutation. All of these configurations can be installed concurrently. They can even be configured to share host independent files across hosts. Imake doesn't address these issues.

Imake templates are a form of standardization. The GNU coding standards address the same issues without necessarily imposing the same restrictions.

Here is some further explanation, written by Per Bothner:

One of the advantages of Imake is that it easy to generate large makefiles using the `#include' and macro mechanisms of cpp. However, cpp is not programmable: it has limited conditional facilities, and no looping. And cpp cannot inspect its environment.

All of these problems are solved by using sh instead of cpp. The shell is fully programmable, has macro substitution, can execute (or source) other shell scripts, and can inspect its environment.

Paul Eggert elaborates more:

With Autoconf, installers need not assume that Imake itself is already installed and working well. This may not seem like much of an advantage to people who are accustomed to Imake. But on many hosts Imake is not installed or the default installation is not working well, and requiring Imake to install a package hinders the acceptance of that package on those hosts. For example, the Imake template and configuration files might not be installed properly on a host, or the Imake build procedure might wrongly assume that all source files are in one big directory tree, or the Imake configuration might assume one compiler whereas the package or the installer needs to use another, or there might be a version mismatch between the Imake expected by the package and the Imake supported by the host. These problems are much rarer with Autoconf, where each package comes with its own independent configuration processor.

Also, Imake often suffers from unexpected interactions between make and the installer's C preprocessor. The fundamental problem here is that the C preprocessor was designed to preprocess C programs, not makefiles. This is much less of a problem with Autoconf, which uses the general-purpose preprocessor M4, and where the package's author (rather than the installer) does the preprocessing in a standard way.

Finally, Mark Eichin notes:

Imake isn't all that extensible, either. In order to add new features to Imake, you need to provide your own project template, and duplicate most of the features of the existing one. This means that for a sophisticated project, using the vendor-provided Imake templates fails to provide any leverage--since they don't cover anything that your own project needs (unless it is an X11 program).

On the other side, though:

The one advantage that Imake has over configure: `Imakefile' files tend to be much shorter (likewise, less redundant) than `Makefile.in' files. There is a fix to this, however--at least for the Kerberos V5 tree, we've modified things to call in common `post.in' and `pre.in' makefile fragments for the entire tree. This means that a lot of common things don't have to be duplicated, even though they normally are in configure setups.

20.5 How Do I #define Installation Directories?

My program needs library files, installed in datadir and
similar.  If I use

AC_DEFINE_UNQUOTED([DATADIR], [$datadir],
  [Define to the read-only architecture-independent
   data directory.])

I get

#define DATADIR "${prefix}/share"

As already explained, this behavior is on purpose, mandated by the GNU Coding Standards, see Installation Directory Variables. There are several means to achieve a similar goal:

• - Do not use AC_DEFINE but use your makefile to pass the actual value of datadir via compilation flags. See section Installation Directory Variables, for the details.
• - This solution can be simplified when compiling a program: you may either extend the CPPFLAGS:

  CPPFLAGS = -DDATADIR='"$(datadir)"' @CPPFLAGS@

  If you are using Automake, you should use AM_CPPFLAGS instead:

  AM_CPPFLAGS = -DDATADIR='"$(datadir)"'

  Alternatively, create a dedicated header file:

  DISTCLEANFILES = myprog-paths.h myprog-paths.h: Makefile echo '#define DATADIR "$(datadir)"' >$@

• - Use AC_DEFINE but have configure compute the literal value of datadir and others. Many people have wrapped macros to automate this task; for an example, see the macro AC_DEFINE_DIR from the Autoconf Macro Archive.

  This solution does not conform to the GNU Coding Standards.

• - Note that all the previous solutions hard wire the absolute name of these directories in the executables, which is not a good property. You may try to compute the names relative to prefix, and try to find prefix at runtime, this way your package is relocatable.

20.6 What is `autom4te.cache'?

What is this directory `autom4te.cache'?  Can I safely remove it?

In the GNU Build System, `configure.ac' plays a central role and is read by many tools: autoconf to create `configure', autoheader to create `config.h.in', automake to create `Makefile.in', autoscan to check the completeness of `configure.ac', autoreconf to check the GNU Build System components that are used. To "read `configure.ac'" actually means to compile it with M4, which can be a long process for complex `configure.ac'.

This is why all these tools, instead of running directly M4, invoke autom4te (see section Invoking autom4te) which, while answering to a specific demand, stores additional information in `autom4te.cache' for future runs. For instance, if you run autoconf, behind the scenes, autom4te also stores information for the other tools, so that when you invoke autoheader or automake etc., reprocessing `configure.ac' is not needed. The speed up is frequently 30%, and is increasing with the size of `configure.ac'.

But it is and remains being simply a cache: you can safely remove it.

Can I permanently get rid of it?

The creation of this cache can be disabled from `~/.autom4te.cfg', see Customizing autom4te, for more details. You should be aware that disabling the cache slows down the Autoconf test suite by 40%. The more GNU Build System components are used, the more the cache is useful; for instance running `autoreconf -f' on the Core Utilities is twice slower without the cache although `--force' implies that the cache is not fully exploited, and eight times slower than without `--force'.

20.7 Header Present But Cannot Be Compiled

The most important guideline to bear in mind when checking for features is to mimic as much as possible the intended use. Unfortunately, old versions of AC_CHECK_HEADER and AC_CHECK_HEADERS failed to follow this idea, and called the preprocessor, instead of the compiler, to check for headers. As a result, incompatibilities between headers went unnoticed during configuration, and maintainers finally had to deal with this issue elsewhere.

The transition began with Autoconf 2.56. As of Autoconf 2.64 both checks are performed, and configure complains loudly if the compiler and the preprocessor do not agree. However, only the compiler result is considered.

Consider the following example:

$ cat number.h
typedef int number;
$ cat pi.h
const number pi = 3;
$ cat configure.ac
AC_INIT([Example], [1.0], [bug-example@example.org])
AC_CHECK_HEADERS([pi.h])
$ autoconf -Wall
$ ./configure
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... grep
checking for egrep... grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking pi.h usability... no
checking pi.h presence... yes
configure: WARNING: pi.h: present but cannot be compiled
configure: WARNING: pi.h:     check for missing prerequisite headers?
configure: WARNING: pi.h: see the Autoconf documentation
configure: WARNING: pi.h:     section "Present But Cannot Be Compiled"
configure: WARNING: pi.h: proceeding with the compiler's result
configure: WARNING:     ## -------------------------------------- ##
configure: WARNING:     ## Report this to bug-example@example.org ##
configure: WARNING:     ## -------------------------------------- ##
checking for pi.h... yes

The proper way the handle this case is using the fourth argument (see section Generic Header Checks):

$ cat configure.ac
AC_INIT([Example], [1.0], [bug-example@example.org])
AC_CHECK_HEADERS([number.h pi.h], [], [],
[[#ifdef HAVE_NUMBER_H
# include <number.h>
#endif
]])
$ autoconf -Wall
$ ./configure
checking for gcc... gcc
checking for C compiler default output... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for number.h... yes
checking for pi.h... yes

See Particular Header Checks, for a list of headers with their prerequisites.

20.8 Expanded Before Required

Older versions of Autoconf silently built files with incorrect ordering between dependent macros if an outer macro first expanded, then later indirectly required, an inner macro. Starting with Autoconf 2.64, this situation no longer generates out-of-order code, but results in duplicate output and a syntax warning:

$ cat configure.ac
⇒AC_DEFUN([TESTA], [[echo in A
⇒if test -n "$SEEN_A" ; then echo duplicate ; fi
⇒SEEN_A=:]])
⇒AC_DEFUN([TESTB], [AC_REQUIRE([TESTA])[echo in B
⇒if test -z "$SEEN_A" ; then echo bug ; fi]])
⇒AC_DEFUN([TESTC], [AC_REQUIRE([TESTB])[echo in C]])
⇒AC_DEFUN([OUTER], [[echo in OUTER]
⇒TESTA
⇒TESTC])
⇒AC_INIT
⇒OUTER
⇒AC_OUTPUT
$ autoconf
⇒configure.ac:11: warning: AC_REQUIRE:
⇒ `TESTA' was expanded before it was required
⇒configure.ac:4: TESTB is expanded from...
⇒configure.ac:6: TESTC is expanded from...
⇒configure.ac:7: OUTER is expanded from...
⇒configure.ac:11: the top level

To avoid this warning, decide what purpose the macro in question serves. If it only needs to be expanded once (for example, if it provides initialization text used by later macros), then the simplest fix is to change the macro to be declared with AC_DEFUN_ONCE (see section One-Shot Macros), although this only works in Autoconf 2.64 and newer. A more portable fix is to change all instances of direct calls to instead go through AC_REQUIRE (see section Prerequisite Macros). If, instead, the macro is parameterized by arguments or by the current definition of other macros in the m4 environment, then the macro should always be directly expanded instead of required.

For another case study, consider this example trimmed down from an actual package. Originally, the package contained shell code and multiple macro invocations at the top level of `configure.ac':

AC_DEFUN([FOO], [AC_COMPILE_IFELSE([…])])
foobar=
AC_PROG_CC
FOO

but that was getting complex, so the author wanted to offload some of the text into a new macro in another file included via `aclocal.m4'. The naïve approach merely wraps the text in a new macro:

AC_DEFUN([FOO], [AC_COMPILE_IFELSE([…])])
AC_DEFUN([BAR], [
foobar=
AC_PROG_CC
FOO
])
BAR

With older versions of Autoconf, the setting of `foobar=' occurs before the single compiler check, as the author intended. But with Autoconf 2.64, this issues the "expanded before it was required" warning for AC_PROG_CC, and outputs two copies of the compiler check, one before `foobar=', and one after. To understand why this is happening, remember that the use of AC_COMPILE_IFELSE includes a call to AC_REQUIRE([AC_PROG_CC]) under the hood. According to the documented semantics of AC_REQUIRE, this means that AC_PROG_CC must occur before the body of the outermost AC_DEFUN, which in this case is BAR, thus preceeding the use of `foobar='. The older versions of Autoconf were broken with regards to the rules of AC_REQUIRE, which explains why the code changed from one over to two copies of AC_PROG_CC when upgrading autoconf. In other words, the author was unknowingly relying on a bug exploit to get the desired results, and that exploit broke once the bug was fixed.

So, what recourse does the author have, to restore their intended semantics of setting `foobar=' prior to a single compiler check, regardless of whether Autoconf 2.63 or 2.64 is used? One idea is to remember that only AC_DEFUN is impacted by AC_REQUIRE; there is always the possibility of using the lower-level m4_define:

AC_DEFUN([FOO], [AC_COMPILE_IFELSE([…])])
m4_define([BAR], [
foobar=
AC_PROG_CC
FOO
])
BAR

This works great if everything is in the same file. However, it does not help in the case where the author wants to have aclocal find the definition of BAR from its own file, since aclocal requires the use of AC_DEFUN. In this case, a better fix is to recognize that if BAR also uses AC_REQUIRE, then there will no longer be direct expansion prior to a subsequent require. Then, by creating yet another helper macro, the author can once again guarantee a single invocation of AC_PROG_CC, which will still occur after foobar=. The author can also use AC_BEFORE to make sure no other macro appearing before BAR has triggered an unwanted expansion of AC_PROG_CC.

AC_DEFUN([FOO], [AC_COMPILE_IFELSE([…])])
AC_DEFUN([BEFORE_CC], [
foobar=
])
AC_DEFUN([BAR], [
AC_BEFORE([$0], [AC_PROG_CC])dnl
AC_REQUIRE([BEFORE_CC])dnl
AC_REQUIRE([AC_PROG_CC])dnl
FOO
])
BAR

20.9 Debugging configure scripts

While in general, configure scripts generated by Autoconf strive to be fairly portable to various systems, compilers, shells, and other tools, it may still be necessary to debug a failing test, broken script or makefile, or fix or override an incomplete, faulty, or erroneous test, especially during macro development. Failures can occur at all levels, in M4 syntax or semantics, shell script issues, or due to bugs in the test or the tools invoked by configure. Together with the rather arcane error message that m4 and make may produce when their input contains syntax errors, this can make debugging rather painful.

Nevertheless, here is a list of hints and strategies that may help:

• When autoconf fails, common causes for error include:
  • mismatched or unbalanced parentheses or braces (see section Dealing with unbalanced parentheses),
  • under- or overquoted macro arguments (see section The Autoconf Language, see section Quoting and Parameters, see section Quotation and Nested Macros),
  • spaces between macro name and opening parenthesis (see section The Autoconf Language).

  Typically, it helps to go back to the last working version of the input and compare the differences for each of these errors. Another possibility is to sprinkle pairs of m4_traceon and m4_traceoff judiciously in the code, either without a parameter or listing some macro names and watch m4 expand its input verbosely (see section Debugging via autom4te).

• Sometimes autoconf succeeds but the generated configure script has invalid shell syntax. You can detect this case by running `bash -n configure' or `sh -n configure'. If this command fails, the same tips apply, as if autoconf had failed.
• Debugging configure script execution may be done by sprinkling pairs of set -x and set +x into the shell script before and after the region that contains a bug. Running the whole script with `shell ./configure -vx 2>&1 | tee log-file' with a decent shell may work, but produces lots of output. Here, it can help to search for markers like `checking for' a particular test in the log-file.
• When configure tests produce invalid results for your system, it may be necessary to override them:
  • For programs, tools or libraries variables, preprocessor, compiler, or linker flags, it is often sufficient to override them at make run time with some care (see section make macro=value and Submakes). Since this normally won't cause configure to be run again with these changed settings, it may fail if the changed variable would have caused different test results from configure, so this may work only for simple differences.
  • Most tests which produce their result in a substituted variable allow to override the test by setting the variable on the configure command line (see section Compilers and Options, see section Defining Variables, see section Particular systems).
  • Many tests store their result in a cache variable (see section Caching Results). This lets you override them either on the configure command line as above, or through a primed cache or site file (see section Cache Files, see section Setting Site Defaults). The name of a cache variable is documented with a test macro or may be inferred from Cache Variable Names; the precise semantics of undocumented variables are often internal details, subject to change.

Conversely, as macro author, you can make it easier for users of your macro:

• by minimizing dependencies between tests and between test results as far as possible,
• by using make variables to factorize and allow override of settings at make run time,
• by honoring the GNU Coding Standards and not overriding flags reserved for the user except temporarily during configure tests,
• by not requiring users of your macro to use the cache variables. Instead, expose the result of the test via run-if-true and run-if-false parameters. If the result is not a boolean, then provide it through documented shell variables.

This document was generated on January, 20 2010 using texi2html 1.76.