[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28. History of Automake

This chapter presents various aspects of the history of Automake. The exhausted reader can safely skip it; this will be more of interest to nostalgic people, or to those curious to learn about the evolution of Automake.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28.1 Timeline

1994-09-19 First CVS commit.

If we can trust the CVS repository, David J.@tie{}MacKenzie (djm) started working on Automake (or AutoMake, as it was spelt then) this Monday.

The first version of the automake script looks as follows.

 
#!/bin/sh

status=0

for makefile
do
  if test ! -f ${makefile}.am; then
    echo "automake: ${makefile}.am: No such honkin' file"
    status=1
    continue
  fi

  exec 4> ${makefile}.in

done

From this you can already see that Automake will be about reading `*.am' file and producing `*.in' files. You cannot see anything else, but if you also know that David is the one who created Autoconf two years before you can guess the rest.

Several commits follow, and by the end of the day Automake is reported to work for GNU fileutils and GNU m4.

The modus operandi is the one that is still used today: variable assignments in `Makefile.am' files trigger injections of precanned `Makefile' fragments into the generated `Makefile.in'. The use of `Makefile' fragments was inspired by the 4.4BSD make and include files, however Automake aims to be portable and to conform to the GNU standards for `Makefile' variables and targets.

At this point, the most recent release of Autoconf is version 1.11, and David is preparing to release Autoconf 2.0 in late October. As a matter of fact, he will barely touch Automake after September.

1994-11-05 David MacKenzie's last commit.

At this point Automake is a 200 line portable shell script, plus 332 lines of `Makefile' fragments. In the `README', David states his ambivalence between "portable shell" and "more appropriate language":

I wrote it keeping in mind the possibility of it becoming an Autoconf macro, so it would run at configure-time. That would slow configuration down a bit, but allow users to modify the Makefile.am without needing to fetch the AutoMake package. And, the Makefile.in files wouldn't need to be distributed. But all of AutoMake would. So I might reimplement AutoMake in Perl, m4, or some other more appropriate language.

Automake is described as "an experimental Makefile generator". There is no documentation. Adventurous users are referred to the examples and patches needed to use Automake with GNU m4 1.3, fileutils 3.9, time 1.6, and development versions of find and indent.

These examples seem to have been lost. However at the time of writing (10 years later in September, 2004) the FSF still distributes a package that uses this version of Automake: check out GNU termutils 2.0.

1995-11-12 Tom Tromey's first commit.

After one year of inactivity, Tom Tromey takes over the package. Tom was working on GNU cpio back then, and doing this just for fun, having trouble finding a project to contribute to. So while hacking he wanted to bring the `Makefile.in' up to GNU standards. This was hard, and one day he saw Automake on ftp://alpha.gnu.org/, grabbed it and tried it out.

Tom didn't talk to djm about it until later, just to make sure he didn't mind if he made a release. He did a bunch of early releases to the Gnits folks.

Gnits was (and still is) totally informal, just a few GNU friends who Fran@m_cedillacois Pinard knew, who were all interested in making a common infrastructure for GNU projects, and shared a similar outlook on how to do it. So they were able to make some progress. It came along with Autoconf and extensions thereof, and then Automake from David and Tom (who were both gnitsians). One of their ideas was to write a document paralleling the GNU standards, that was more strict in some ways and more detailed. They never finished the GNITS standards, but the ideas mostly made their way into Automake.

1995-11-23 Automake 0.20

Besides introducing automatic dependency tracking (see section Dependency Tracking in Automake), this version also supplies a 9-page manual.

At this time aclocal and AM_INIT_AUTOMAKE did not exist, so many things had to be done by hand. For instance, here is what a configure.in (this is the former name of the `configure.ac' we use today) must contain in order to use Automake 0.20:

 
PACKAGE=cpio
VERSION=2.3.911
AC_DEFINE_UNQUOTED(PACKAGE, "$PACKAGE")
AC_DEFINE_UNQUOTED(VERSION, "$VERSION")
AC_SUBST(PACKAGE)
AC_SUBST(VERSION)
AC_ARG_PROGRAM
AC_PROG_INSTALL

(Today all of the above is achieved by AC_INIT and AM_INIT_AUTOMAKE.)

Here is how programs are specified in `Makefile.am':

 
PROGRAMS = hello
hello_SOURCES = hello.c

This looks pretty much like what we do today, except the PROGRAMS variable has no directory prefix specifying where `hello' should be installed: all programs are installed in `$(bindir)'. LIBPROGRAMS can be used to specify programs that must be built but not installed (it is called noinst_PROGRAMS nowadays).

Programs can be built conditionally using AC_SUBSTitutions:

 
PROGRAMS = @progs@
AM_PROGRAMS = foo bar baz

(AM_PROGRAMS has since then been renamed to EXTRA_PROGRAMS.)

Similarly scripts, static libraries, and data can be built and installed using the LIBRARIES, SCRIPTS, and DATA variables. However LIBRARIES were treated a bit specially in that Automake did automatically supply the `lib' and `.a' prefixes. Therefore to build `libcpio.a', one had to write

 
LIBRARIES = cpio
cpio_SOURCES = ...

Extra files to distribute must be listed in DIST_OTHER (the ancestor of EXTRA_DIST). Also extra directories that are to be distributed should appear in DIST_SUBDIRS, but the manual describes this as a temporary ugly hack (today extra directories should also be listed in EXTRA_DIST, and DIST_SUBDIRS is used for another purpose, see section Conditional Subdirectories).

1995-11-26 Automake 0.21

In less time than it takes to cook a frozen pizza, Tom rewrites Automake using Perl. At this time Perl 5 is only one year old, and Perl 4.036 is in use at many sites. Supporting several Perl versions has been a source of problems through the whole history of Automake.

If you never used Perl 4, imagine Perl 5 without objects, without `my' variables (only dynamically scoped `local' variables), without function prototypes, with function calls that needs to be prefixed with `&', etc. Traces of this old style can still be found in today's automake.

1995-11-28 Automake 0.22
1995-11-29 Automake 0.23

Bug fixes.

1995-12-08 Automake 0.24
1995-12-10 Automake 0.25

Releases are raining. 0.24 introduces the uniform naming scheme we use today, i.e., bin_PROGRAMS instead of PROGRAMS, noinst_LIBRARIES instead of LIBLIBRARIES, etc. (However EXTRA_PROGRAMS does not exist yet, AM_PROGRAMS is still in use; and TEXINFOS and MANS still have no directory prefixes.) Adding support for prefixes like that was one of the major ideas in automake; it has lasted pretty well.

AutoMake is renamed to Automake (Tom seems to recall it was Fran@m_cedillacois Pinard's doing).

0.25 fixes a Perl 4 portability bug.

1995-12-18 Jim Meyering starts using Automake in GNU Textutils.
1995-12-31 Fran@m_cedillacois Pinard starts using Automake in GNU tar.
1996-01-03 Automake 0.26
1996-01-03 Automake 0.27

Of the many changes and suggestions sent by Fran@m_cedillacois Pinard and included in 0.26, perhaps the most important is the advice that to ease customization a user rule or variable definition should always override an Automake rule or definition.

Gordon Matzigkeit and Jim Meyering are two other early contributors that have been sending fixes.

0.27 fixes yet another Perl 4 portability bug.

1996-01-13 Automake 0.28

Automake starts scanning `configure.in' for LIBOBJS support. This is an important step because until this version Automake only knew about the `Makefile.am's it processed. `configure.in' was Autoconf's world and the link between Autoconf and Automake had to be done by the `Makefile.am' author. For instance, if `config.h' was generated by `configure', it was the package maintainer's responsibility to define the CONFIG_HEADER variable in each `Makefile.am'.

Succeeding releases will rely more and more on scanning `configure.in' to better automate the Autoconf integration.

0.28 also introduces the AUTOMAKE_OPTIONS variable and the `--gnu' and `--gnits' options, the latter being stricter.

1996-02-07 Automake 0.29

Thanks to `configure.in' scanning, CONFIG_HEADER is gone, and rebuild rules for `configure'-generated file are automatically output.

TEXINFOS and MANS converted to the uniform naming scheme.

1996-02-24 Automake 0.30

The test suite is born. It contains 9 tests. From now on test cases will be added pretty regularly (see section Release Statistics), and this proved to be really helpful later on.

EXTRA_PROGRAMS finally replaces AM_PROGRAMS.

All the third-party Autoconf macros, written mostly by Fran@m_cedillacois Pinard (and later Jim Meyering), are distributed in Automake's hand-written `aclocal.m4' file. Package maintainers are expected to extract the necessary macros from this file. (In previous versions you had to copy and paste them from the manual...)

1996-03-11 Automake 0.31

The test suite in 0.30 was run via a long check-local rule. Upon Ulrich Drepper's suggestion, 0.31 makes it an Automake rule output whenever the TESTS variable is defined.

DIST_OTHER is renamed to EXTRA_DIST, and the check_ prefix is introduced. The syntax is now the same as today.

1996-03-15 Gordon Matzigkeit starts writing libtool.
1996-04-27 Automake 0.32

-hook targets are introduced; an idea from Dieter Baron.

`*.info' files, which were output in the build directory are now built in the source directory, because they are distributed. It seems these files like to move back and forth as that will happen again in future versions.

1996-05-18 Automake 0.33

Gord Matzigkeit's main two contributions:

Although they were very basic at this point, these are probably among the top features for Automake today.

Jim Meyering also provides the infamous jm_MAINTAINER_MODE, since then renamed to AM_MAINTAINER_MODE and abandoned by its author (see section missing and AM_MAINTAINER_MODE).

1996-05-28 Automake 1.0

After only six months of heavy development, the automake script is 3134 lines long, plus 973 lines of `Makefile' fragments. The package has 30 pages of documentation, and 38 test cases. `aclocal.m4' contains 4 macros.

From now on and until version 1.4, new releases will occur at a rate of about one a year. 1.1 did not exist, actually 1.1b to 1.1p have been the name of beta releases for 1.2. This is the first time Automake uses suffix letters to designate beta releases, a habit that lasts.

1996-10-10 Kevin Dalley packages Automake 1.0 for Debian GNU/Linux.
1996-11-26 David J.@tie{}MacKenzie releases Autoconf 2.12.

Between June and October, the Autoconf development is almost stalled. Roland McGrath has been working at the beginning of the year. David comes back in November to release 2.12, but he won't touch Autoconf anymore after this year, and Autoconf then really stagnates. The desolate Autoconf `ChangeLog' for 1997 lists only 7 commits.

1997-02-28 automake@gnu.ai.mit.edu list alive

The mailing list is announced as follows:

 
I've created the "automake" mailing list.  It is
"automake@gnu.ai.mit.edu".  Administrivia, as always, to
automake-request@gnu.ai.mit.edu.

The charter of this list is discussion of automake, autoconf, and
other configuration/portability tools (e.g., libtool).  It is expected
that discussion will range from pleas for help all the way up to
patches.

This list is archived on the FSF machines.  Offhand I don't know if
you can get the archive without an account there.

This list is open to anybody who wants to join.  Tell all your
friends!
-- Tom Tromey

Before that people were discussing Automake privately, on the Gnits mailing list (which is not public either), and less frequently on gnu.misc.discuss.

gnu.ai.mit.edu is now gnu.org, in case you never noticed. The archives of the early years of the automake@gnu.org list have been lost, so today it is almost impossible to find traces of discussions that occurred before 1999. This has been annoying more than once, as such discussions can be useful to understand the rationale behind a piece of uncommented code that was introduced back then.

1997-06-22 Automake 1.2

Automake developments continues, and more and more new Autoconf macros are required. Distributing them in `aclocal.m4' and requiring people to browse this file to extract the relevant macros becomes uncomfortable. Ideally, some of them should be contributed to Autoconf so that they can be used directly, however Autoconf is currently inactive. Automake 1.2 consequently introduces aclocal (aclocal was actually started on 1996-07-28), a tool that automatically constructs an `aclocal.m4' file from a repository of third-party macros. Because Autoconf has stalled, Automake also becomes a kind of repository for such third-party macros, even macros completely unrelated to Automake (for instance macros that fix broken Autoconf macros).

The 1.2 release contains 20 macros, including the AM_INIT_AUTOMAKE macro that simplifies the creation of `configure.in'.

Libtool is fully supported using *_LTLIBRARIES.

The missing script is introduced by Fran@m_cedillacois Pinard; it is meant to be a better solution than AM_MAINTAINER_MODE (see section missing and AM_MAINTAINER_MODE).

Conditionals support was implemented by Ian Lance Taylor. At the time, Tom and Ian were working on an internal project at Cygnus. They were using ILU, which is pretty similar to CORBA. They wanted to integrate ILU into their build, which was all `configure'-based, and Ian thought that adding conditionals to automake was simpler than doing all the work in `configure' (which was the standard at the time). So this was actually funded by Cygnus.

This very useful but tricky feature will take a lot of time to stabilize. (At the time this text is written, there are still primaries that have not been updated to support conditional definitions in Automake 1.9.)

The automake script has almost doubled: 6089 lines of Perl, plus 1294 lines of `Makefile' fragments.

1997-07-08 Gordon Matzigkeit releases Libtool 1.0.
1998-04-05 Automake 1.3

This is a small advance compared to 1.2. It adds support for assembly, and preliminary support for Java.

Perl 5.004_04 is out, but fixes to support Perl 4 are still regularly submitted whenever Automake breaks it.

1998-09-06 sourceware.cygnus.com is on-line.

Sourceware was setup by Jason Molenda to host open source projects.

1998-09-19 Automake CVS repository moved to sourceware.cygnus.com
1998-10-26 sourceware.cygnus.com announces it hosts Automake:

Automake is now hosted on sourceware.cygnus.com. It has a publicly accessible CVS repository. This CVS repository is a copy of the one Tom was using on his machine, which in turn is based on a copy of the CVS repository of David MacKenzie. This is why we still have to full source history. (Automake was on Sourceware until 2007-10-29, when it moved to a git repository on savannah.gnu.org, but the Sourceware host had been renamed to sources.redhat.com.)

The oldest file in the administrative directory of the CVS repository that was created on Sourceware is dated 1998-09-19, while the announcement that automake and autoconf had joined sourceware was made on 1998-10-26. They were among the first projects to be hosted there.

The heedful reader will have noticed Automake was exactly 4 years old on 1998-09-19.

1999-01-05 Ben Elliston releases Autoconf 2.13.
1999-01-14 Automake 1.4

This release adds support for Fortran 77 and for the include statement. Also, `+=' assignments are introduced, but it is still quite easy to fool Automake when mixing this with conditionals.

These two releases, Automake 1.4 and Autoconf 2.13 make a duo that will be used together for years.

automake is 7228 lines, plus 1591 lines of Makefile fragment, 20 macros (some 1.3 macros were finally contributed back to Autoconf), 197 test cases, and 51 pages of documentation.

1999-03-27 The user-dep-branch is created on the CVS repository.

This implements a new dependency tracking schemed that should be able to handle automatic dependency tracking using any compiler (not just gcc) and any make (not just GNU make). In addition, the new scheme should be more reliable than the old one, as dependencies are generated on the end user's machine. Alexandre Oliva creates depcomp for this purpose.

See section Dependency Tracking in Automake, for more details about the evolution of automatic dependency tracking in Automake.

1999-11-21 The user-dep-branch is merged into the main trunk.

This was a huge problem since we also had patches going in on the trunk. The merge took a long time and was very painful.

2000-05-10

Since September 1999 and until 2003, Akim Demaille will be zealously revamping Autoconf.

I think the next release should be called "3.0".
Let's face it: you've basically rewritten autoconf.
Every weekend there are 30 new patches.
I don't see how we could call this "2.15" with a straight face.
- Tom Tromey on autoconf@gnu.org

Actually Akim works like a submarine: he will pile up patches while he works off-line during the weekend, and flush them in batch when he resurfaces on Monday.

2001-01-24

On this Wednesday, Autoconf 2.49c, the last beta before Autoconf 2.50 is out, and Akim has to find something to do during his week-end :)

2001-01-28

Akim sends a batch of 14 patches to automake@gnu.org.

Aiieeee! I was dreading the day that the Demaillator turned his sights on automake… and now it has arrived! - Tom Tromey

It's only the beginning: in two months he will send 192 patches. Then he would slow down so Tom can catch up and review all this. Initially Tom actually read all these patches, then he probably trustingly answered OK to most of them, and finally gave up and let Akim apply whatever he wanted. There was no way to keep up with that patch rate.

Anyway the patch below won't apply since it predates Akim's sourcequake; I have yet to figure where the relevant passage has been moved :) - Alexandre Duret-Lutz

All these patches were sent to and discussed on automake@gnu.org, so subscribed users were literally drowning in technical mails. Eventually, the automake-patches@gnu.org mailing list was created in May.

Year after year, Automake had drifted away from its initial design: construct `Makefile.in' by assembling various `Makefile' fragments. In 1.4, lots of `Makefile' rules are being emitted at various places in the automake script itself; this does not help ensuring a consistent treatment of these rules (for instance making sure that user-defined rules override Automake's own rules). One of Akim's goal was moving all these hard-coded rules to separate `Makefile' fragments, so the logic could be centralized in a `Makefile' fragment processor.

Another significant contribution of Akim is the interface with the "trace" feature of Autoconf. The way to scan `configure.in' at this time was to read the file and grep the various macro of interest to Automake. Doing so could break in many unexpected ways; automake could miss some definition (for instance `AC_SUBST([$1], [$2])' where the arguments are known only when M4 is run), or conversely it could detect some macro that was not expanded (because it is called conditionally). In the CVS version of Autoconf, Akim had implemented the `--trace' option, which provides accurate information about where macros are actually called and with what arguments. Akim will equip Automake with a second `configure.in' scanner that uses this `--trace' interface. Since it was not sensible to drop the Autoconf 2.13 compatibility yet, this experimental scanner was only used when an environment variable was set, the traditional grep-scanner being still the default.

2001-04-25 Gary V.@tie{}Vaughan releases Libtool 1.4

It has been more than two years since Automake 1.4, CVS Automake has suffered lot's of heavy changes and still is not ready for release. Libtool 1.4 had to be distributed with a patch against Automake 1.4.

2001-05-08 Automake 1.4-p1
2001-05-24 Automake 1.4-p2

Gary V.@tie{}Vaughan, the principal Libtool maintainer, makes a "patch release" of Automake:

The main purpose of this release is to have a stable automake which is compatible with the latest stable libtool.

The release also contains obvious fixes for bugs in Automake 1.4, some of which were reported almost monthly.

2001-05-21 Akim Demaille releases Autoconf 2.50
2001-06-07 Automake 1.4-p3
2001-06-10 Automake 1.4-p4
2001-07-15 Automake 1.4-p5

Gary continues his patch-release series. These also add support for some new Autoconf 2.50 idioms. Essentially, Autoconf now advocates `configure.ac' over `configure.in', and it introduces a new syntax for AC_OUTPUTing files.

2001-08-23 Automake 1.5

A major and long-awaited release, that comes more than two years after 1.4. It brings many changes, among which:

1.5 did break several packages that worked with 1.4. Enough so that Linux distributions could not easily install the new Automake version without breaking many of the packages for which they had to run automake.

Some of these breakages were effectively bugs that would eventually be fixed in the next release. However, a lot of damage was caused by some changes made deliberately to render Automake stricter on some setup we did consider bogus. For instance, `make distcheck' was improved to check that `make uninstall' did remove all the files `make install' installed, that `make distclean' did not omit some file, and that a VPATH build would work even if the source directory was read-only. Similarly, Automake now rejects multiple definitions of the same variable (because that would mix very badly with conditionals), and `+=' assignments with no previous definition. Because these changes all occurred suddenly after 1.4 had been established for more than two years, it hurt users.

To make matter worse, meanwhile Autoconf (now at version 2.52) was facing similar troubles, for similar reasons.

2002-03-05 Automake 1.6

This release introduced versioned installation (see section Automake API Versioning). This was mainly pushed by Havoc Pennington, taking the GNOME source tree as motive: due to incompatibilities between the autotools it's impossible for the GNOME packages to switch to Autoconf 2.53 and Automake 1.5 all at once, so they are currently stuck with Autoconf 2.13 and Automake 1.4.

The idea was to call this version `automake-1.6', call all its bug-fix versions identically, and switch to `automake-1.7' for the next release that adds new features or changes some rules. This scheme implies maintaining a bug-fix branch in addition to the development trunk, which means more work from the maintainer, but providing regular bug-fix releases proved to be really worthwhile.

Like 1.5, 1.6 also introduced a bunch of incompatibilities, intentional or not. Perhaps the more annoying was the dependence on the newly released Autoconf 2.53. Autoconf seemed to have stabilized enough since its explosive 2.50 release and included changes required to fix some bugs in Automake. In order to upgrade to Automake 1.6, people now had to upgrade Autoconf too; for some packages it was no picnic.

While versioned installation helped people to upgrade, it also unfortunately allowed people not to upgrade. At the time of writing, some Linux distributions are shipping packages for Automake 1.4, 1.5, 1.6, 1.7, 1.8, and 1.9. Most of these still install 1.4 by default. Some distribution also call 1.4 the "stable" version, and present "1.9" as the development version; this does not really makes sense since 1.9 is way more solid than 1.4. All this does not help the newcomer.

2002-04-11 Automake 1.6.1

1.6, and the upcoming 1.4-p6 release were the last release by Tom. This one and those following will be handled by Alexandre Duret-Lutz. Tom is still around, and will be there until about 1.7, but his interest into Automake is drifting away towards projects like gcj.

Alexandre has been using Automake since 2000, and started to contribute mostly on Akim's incitement (Akim and Alexandre have been working in the same room from 1999 to 2002). In 2001 and 2002 he had a lot of free time to enjoy hacking Automake.

2002-06-14 Automake 1.6.2
2002-07-28 Automake 1.6.3
2002-07-28 Automake 1.4-p6

Two releases on the same day. 1.6.3 is a bug-fix release.

Tom Tromey backported the versioned installation mechanism on the 1.4 branch, so that Automake 1.6.x and Automake 1.4-p6 could be installed side by side. Another request from the GNOME folks.

2002-09-25 Automake 1.7

This release switches to the new `configure.ac' scanner Akim was experimenting in 1.5.

2002-10-16 Automake 1.7.1
2002-12-06 Automake 1.7.2
2003-02-20 Automake 1.7.3
2003-04-23 Automake 1.7.4
2003-05-18 Automake 1.7.5
2003-07-10 Automake 1.7.6
2003-09-07 Automake 1.7.7
2003-10-07 Automake 1.7.8

Many bug-fix releases. 1.7 lasted because the development version (upcoming 1.8) was suffering some major internal revamping.

2003-10-26 Automake on screen

Episode 49, `Repercussions', in the third season of the `Alias' TV show is first aired.

Marshall, one of the characters, is working on a computer virus that he has to modify before it gets into the wrong hands or something like that. The screenshots you see do not show any program code, they show a `Makefile.in' generated by automake...

2003-11-09 Automake 1.7.9
2003-12-10 Automake 1.8

The most striking update is probably that of aclocal.

aclocal now uses m4_include in the produced `aclocal.m4' when the included macros are already distributed with the package (an idiom used in many packages), which reduces code duplication. Many people liked that, but in fact this change was really introduced to fix a bug in rebuild rules: `Makefile.in' must be rebuilt whenever a dependency of `configure' changes, but all the `m4' files included in `aclocal.m4' where unknown from automake. Now automake can just trace the m4_includes to discover the dependencies.

aclocal also starts using the `--trace' Autoconf option in order to discover used macros more accurately. This will turn out to be very tricky (later releases will improve this) as people had devised many ways to cope with the limitation of previous aclocal versions, notably using handwritten m4_includes: aclocal must make sure not to redefine a rule that is already included by such statement.

Automake also has seen its guts rewritten. Although this rewriting took a lot of efforts, it is only apparent to the users in that some constructions previously disallowed by the implementation now work nicely. Conditionals, Locations, Variable and Rule definitions, Options: these items on which Automake works have been rewritten as separate Perl modules, and documented.

2004-01-11 Automake 1.8.1
2004-01-12 Automake 1.8.2
2004-03-07 Automake 1.8.3
2004-04-25 Automake 1.8.4
2004-05-16 Automake 1.8.5
2004-07-28 Automake 1.9

This release tries to simplify the compilation rules it outputs to reduce the size of the Makefile. The complaint initially come from the libgcj developers. Their `Makefile.in' generated with Automake 1.4 and custom build rules (1.4 did not support compiled Java) is 250KB. The one generated by 1.8 was over 9MB! 1.9 gets it down to 1.2MB.

Aside from this it contains mainly minor changes and bug-fixes.

2004-08-11 Automake 1.9.1
2004-09-19 Automake 1.9.2

Automake has ten years. This chapter of the manual was initially written for this occasion.

2007-10-29 Automake repository moves to savannah.gnu.org and uses

git as primary repository.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28.2 Dependency Tracking in Automake

Over the years Automake has deployed three different dependency tracking methods. Each method, including the current one, has had flaws of various sorts. Here we lay out the different dependency tracking methods, their flaws, and their fixes. We conclude with recommendations for tool writers, and by indicating future directions for dependency tracking work in Automake.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28.2.1 First Take on Dependency Tracking


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Description

Our first attempt at automatic dependency tracking was based on the method recommended by GNU make. (see (make)Automatic Prerequisites section `Generating Prerequisites Automatically' in The GNU make Manual)

This version worked by precomputing dependencies ahead of time. For each source file, it had a special `.P' file that held the dependencies. There was a rule to generate a `.P' file by invoking the compiler appropriately. All such `.P' files were included by the `Makefile', thus implicitly becoming dependencies of `Makefile'.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Bugs

This approach had several critical bugs.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Historical Note

The code generated by Automake is often inspired by the `Makefile' style of a particular author. In the case of the first implementation of dependency tracking, I believe the impetus and inspiration was Jim Meyering. (I could be mistaken. If you know otherwise feel free to correct me.)


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28.2.2 Dependencies As Side Effects


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Description

The next refinement of Automake's automatic dependency tracking scheme was to implement dependencies as side effects of the compilation. This was aimed at solving the most commonly reported problems with the first approach. In particular we were most concerned with eliminating the weird rebuilding effect associated with make clean.

In this approach, the `.P' files were included using the -include command, which let us create these files lazily. This avoided the `make clean' problem.

We only computed dependencies when a file was actually compiled. This avoided the performance penalty associated with scanning each file twice. It also let us avoid the other problems associated with the first, eager, implementation. For instance, dependencies would never be generated for a source file that was not compilable on a given architecture (because it in fact would never be compiled).


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Bugs


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28.2.3 Dependencies for the User


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Description

The bugs associated with `make dist', over time, became a real problem. Packages using Automake were being built on a large number of platforms, and were becoming increasingly complex. Broken dependencies were distributed in "portable" `Makefile.in's, leading to user complaints. Also, the requirement for gcc and GNU make was a constant source of bug reports. The next implementation of dependency tracking aimed to remove these problems.

We realized that the only truly reliable way to automatically track dependencies was to do it when the package itself was built. This meant discovering a method portable to any version of make and any compiler. Also, we wanted to preserve what we saw as the best point of the second implementation: dependency computation as a side effect of compilation.

In the end we found that most modern make implementations support some form of include directive. Also, we wrote a wrapper script that let us abstract away differences between dependency tracking methods for compilers. For instance, some compilers cannot generate dependencies as a side effect of compilation. In this case we simply have the script run the compiler twice. Currently our wrapper script (depcomp) knows about twelve different compilers (including a "compiler" that simply invokes makedepend and then the real compiler, which is assumed to be a standard Unix-like C compiler with no way to do dependency tracking).


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Bugs

This code is used since Automake 1.5.

In GCC 3.0, we managed to convince the maintainers to add special command-line options to help Automake more efficiently do its job. We hoped this would let us avoid the use of a wrapper script when Automake's automatic dependency tracking was used with gcc.

Unfortunately, this code doesn't quite do what we want. In particular, it removes the dependency file if the compilation fails; we'd prefer that it instead only touch the file in any way if the compilation succeeds.

Nevertheless, since Automake 1.7, when a recent gcc is detected at configure time, we inline the dependency-generation code and do not use the depcomp wrapper script. This makes compilations faster for those using this compiler (probably our primary user base). The counterpart is that because we have to encode two compilation rules in `Makefile' (with or without depcomp), the produced `Makefile's are larger.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28.2.4 Techniques for Computing Dependencies

There are actually several ways for a build tool like Automake to cause tools to generate dependencies.

makedepend

This was a commonly-used method in the past. The idea is to run a special program over the source and have it generate dependency information. Traditional implementations of makedepend are not completely precise; ordinarily they were conservative and discovered too many dependencies.

The tool

An obvious way to generate dependencies is to simply write the tool so that it can generate the information needed by the build tool. This is also the most portable method. Many compilers have an option to generate dependencies. Unfortunately, not all tools provide such an option.

The file system

It is possible to write a special file system that tracks opens, reads, writes, etc, and then feed this information back to the build tool. clearmake does this. This is a very powerful technique, as it doesn't require cooperation from the tool. Unfortunately it is also very difficult to implement and also not practical in the general case.

LD_PRELOAD

Rather than use the file system, one could write a special library to intercept open and other syscalls. This technique is also quite powerful, but unfortunately it is not portable enough for use in automake.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28.2.5 Recommendations for Tool Writers

We think that every compilation tool ought to be able to generate dependencies as a side effect of compilation. Furthermore, at least while make-based tools are nearly universally in use (at least in the free software community), the tool itself should generate dummy dependencies for header files, to avoid the deleted header file bug. Finally, the tool should generate a dependency for each probe, instead of each successful file open, in order to avoid the duplicated new header bug.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28.2.6 Future Directions for Dependencies

Currently, only languages and compilers understood by Automake can have dependency tracking enabled. We would like to see if it is practical (and worthwhile) to let this support be extended by the user to languages unknown to Automake.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

28.3 Release Statistics

The following table (inspired by `perlhist(1)') quantifies the evolution of Automake using these metrics:

Date, Rel

The date and version of the release.

am

The number of lines of the automake script.

acl

The number of lines of the aclocal script.

pm

The number of lines of the Perl supporting modules.

`*.am'

The number of lines of the `Makefile' fragments. The number in parentheses is the number of files.

m4

The number of lines (and files) of Autoconf macros.

doc

The number of pages of the documentation (the Postscript version).

t

The number of test cases in the test suite. Of those, the number in parentheses is the number of generated test cases.

Date

Rel

am

acl

pm

`*.am'

m4

doc

t

1994-09-19

CVS

141

299 (24)

1994-11-05

CVS

208

332 (28)

1995-11-23

0.20

533

458 (35)

9

1995-11-26

0.21

613

480 (36)

11

1995-11-28

0.22

1116

539 (38)

12

1995-11-29

0.23

1240

541 (38)

12

1995-12-08

0.24

1462

504 (33)

14

1995-12-10

0.25

1513

511 (37)

15

1996-01-03

0.26

1706

438 (36)

16

1996-01-03

0.27

1706

438 (36)

16

1996-01-13

0.28

1964

934 (33)

16

1996-02-07

0.29

2299

936 (33)

17

1996-02-24

0.30

2544

919 (32)

85 (1)

20

9

1996-03-11

0.31

2877

919 (32)

85 (1)

29

17

1996-04-27

0.32

3058

921 (31)

85 (1)

30

26

1996-05-18

0.33

3110

926 (31)

105 (1)

30

35

1996-05-28

1.0

3134

973 (32)

105 (1)

30

38

1997-06-22

1.2

6089

385

1294 (36)

592 (20)

37

126

1998-04-05

1.3

6415

422

1470 (39)

741 (23)

39

156

1999-01-14

1.4

7240

426

1591 (40)

734 (20)

51

197

2001-05-08

1.4-p1

7251

426

1591 (40)

734 (20)

51

197

2001-05-24

1.4-p2

7268

439

1591 (40)

734 (20)

49

197

2001-06-07

1.4-p3

7312

439

1591 (40)

734 (20)

49

197

2001-06-10

1.4-p4

7321

439

1591 (40)

734 (20)

49

198

2001-07-15

1.4-p5

7228

426

1596 (40)

734 (20)

51

198

2001-08-23

1.5

8016

475

600

2654 (39)

1166 (29)

63

327

2002-03-05

1.6

8465

475

1136

2732 (39)

1603 (27)

66

365

2002-04-11

1.6.1

8544

475

1136

2741 (39)

1603 (27)

66

372

2002-06-14

1.6.2

8575

475

1136

2800 (39)

1609 (27)

67

386

2002-07-28

1.6.3

8600

475

1153

2809 (39)

1609 (27)

67

391

2002-07-28

1.4-p6

7332

455

1596 (40)

735 (20)

49

197

2002-09-25

1.7

9189

471

1790

2965 (39)

1606 (28)

73

430

2002-10-16

1.7.1

9229

475

1790

2977 (39)

1606 (28)

73

437

2002-12-06

1.7.2

9334

475

1790

2988 (39)

1606 (28)

77

445

2003-02-20

1.7.3

9389

475

1790

3023 (39)

1651 (29)

84

448

2003-04-23

1.7.4

9429

475

1790

3031 (39)

1644 (29)

85

458

2003-05-18

1.7.5

9429

475

1790

3033 (39)

1645 (29)

85

459

2003-07-10

1.7.6

9442

475

1790

3033 (39)

1660 (29)

85

461

2003-09-07

1.7.7

9443

475

1790

3041 (39)

1660 (29)

90

467

2003-10-07

1.7.8

9444

475

1790

3041 (39)

1660 (29)

90

468

2003-11-09

1.7.9

9444

475

1790

3048 (39)

1660 (29)

90

468

2003-12-10

1.8

7171

585

7730

3236 (39)

1666 (31)

104

521

2004-01-11

1.8.1

7217

663

7726

3287 (39)

1686 (31)

104

525

2004-01-12

1.8.2

7217

663

7726

3288 (39)

1686 (31)

104

526

2004-03-07

1.8.3

7214

686

7735

3303 (39)

1695 (31)

111

530

2004-04-25

1.8.4

7214

686

7736

3310 (39)

1701 (31)

112

531

2004-05-16

1.8.5

7240

686

7736

3299 (39)

1701 (31)

112

533

2004-07-28

1.9

7508

715

7794

3352 (40)

1812 (32)

115

551

2004-08-11

1.9.1

7512

715

7794

3354 (40)

1812 (32)

115

552

2004-09-19

1.9.2

7512

715

7794

3354 (40)

1812 (32)

132

554

2004-11-01

1.9.3

7507

718

7804

3354 (40)

1812 (32)

134

556

2004-12-18

1.9.4

7508

718

7856

3361 (40)

1811 (32)

140

560

2005-02-13

1.9.5

7523

719

7859

3373 (40)

1453 (32)

142

562

2005-07-10

1.9.6

7539

699

7867

3400 (40)

1453 (32)

144

570

2006-10-15

1.10

7859

1072

8024

3512 (40)

1496 (34)

172

604

2008-01-19

1.10.1

7870

1089

8025

3520 (40)

1499 (34)

173

617

2008-11-23

1.10.2

7882

1089

8027

3540 (40)

1509 (34)

176

628

2009-05-17

1.11

8721

1092

8289

4164 (42)

1714 (37)

181

732 (20)


[ << ] [ >> ]           [Top] [Contents] [Index] [ ? ]

This document was generated on July, 20 2009 using texi2html 1.76.