AMBuild

AMBuild is a tool for building software projects and creating packages for release. It is targeted at C++ projects, though it can be used for anything. It has been tailored to solve three major problems that most build tools do not address:

• Accuracy. AMBuild guarantees that you never need to "clean" a build. Clean rebuilds are unnecessary and a waste of developer time. Builds are always computed accurately, and any failure to do so is considered a bug in AMBuild.
• Speed. Most build systems discover changes by recursively searching downward through all build steps and their dependencies. This is very slow. AMBuild instead computes the set of files that have changed, and propagates those changes upward, to each build step that cares. An empty-build takes no time, whereas a large project with other build systems can have 30-second empty builds.
• Flexibility. Distributing C++ software is hard. AMBuild provides full tuning of the C++ compiler and offers abstraction and introspection of system properties, to avoid boilerplate often seen with Make or Autoconf. Build scripts are written in Python, offering a great deal of programmatic control.

Keep in mind, AMBuild is neither widely used nor has it been used on a wide variety of systems. AlliedModders has used it successfully for small to medium-sized C++ projects on Linux, Mac, and Windows. Our largest project, SourceMod, has 700 C++ files and over 200,000 lines of code. We're happy to receive feedback (see the bottom of this page) if you use it in your own projects.

Technical Overview

AMBuild is separated into a frontend and a backend. The frontend is responsible for parsing build scripts (this is known as the configure step). The backend is responsible for actually performing builds. The frontend and backend are separate, and it is possible to use a different backend other than AMBuild. For example, the configure step can produce Visual Studio project files.

Configuring

Build scripts are written in Python, and they are parsed whenever you configure the build. If a build script changes, it will automatically re-trigger the configure process on the next build. When a configure takes place, any files left by an old build are removed if they are modified or removed in the new dependency graph.

The details of the generated dependency graph are stored in an SQLite database, which is located in a hidden .ambuild2 folder inside the build path.

Building

The configure process also generates a build.py which should be run to perform a build. The build process involves a few important steps:

1. Damage Computation. Builds a list of all files that have changed since the last build. This is based on filesystem timestamps, and either a forward or backward time change is enough to be considered "damaged" (or dirty). For example:

   $ python build.py --show-changed
   /home/dvander/alliedmodders/mmsource-central/loader/loader.cpp
   

2. Partial DAG Construction. A partial dependency graph is built based on the list of damaged files. This is the entire set of nodes in the graph which need to be recomputed. The output of this step can be seen with --show-damage. For example:

   $ python build.py --show-damage
    - package/addons/metamod/bin/server_i486.so
     - cp "../loader/server_i486/server_i486.so" "package/addons/metamod/bin/server_i486.so"
      - loader/server_i486/server_i486.so
       - c++ loader.o gamedll.o serverplugin.o utility.o -m32 -static-libgcc -shared -o server_i486.so
        - loader/server_i486/loader.o
         - [gcc] -> c++ -Wall -Werror -H -c /home/dvander/alliedmodders/mmsource-central/loader/loader.cpp -o loader.o
          - /home/dvander/alliedmodders/mmsource-central/loader/loader.cpp
   

3. Task Construction. The partial DAG is simplified into a tree of commands to run. The output of this step can be seen with --show-commands. For example:

   $ python build.py --show-commands
    - cp "../loader/server_i486/server_i486.so" "package/addons/metamod/bin/server_i486.so"
     - c++ loader.o gamedll.o serverplugin.o utility.o -shared -o server_i486.so
      - [gcc] -> c++ -Wall -Werror -H -c /home/dvander/alliedmodders/mmsource-central/loader/loader.cpp -o loader.o
   

4. Updating. Each task in the task tree is executed and the results are processed to either reject the build, or update the state of the dependency graph. At this phase, tasks are executed in parallel based on the number of CPU cores available. The task graph is also shared to maximize throughput. You can see the sequential build steps with --show-steps:

   task 0: [gcc] -> c++ -Wall -Werror -H -c /home/dvander/alliedmodders/mmsource-central/loader/loader.cpp -o loader.o
     -> loader/server/loader.o
   task 1: c++ loader.o gamedll.o serverplugin.o utility.o -m32 -static-libgcc -shared -o server.so
     -> loader/server/server.so
   task 2: cp "../loader/server/server.so" "package/addons/metamod/bin/server.so"
     -> package/addons/metamod/bin/server.so