AMBuild API

This documentation is for the AMBuild 2.2 API. See older versions: AMBuild API (2.1) or AMBuild API (2.0).

AMBuild scripts have access to the following object types:

• BuildParser: This tells AMBuild how to begin parsing your AMBuildScript. It is usually created in configure.py.
• Context: Exposed as the builder, this is the entry point for the API and is known as a build context.
• Entry: Represents a node in the dependency graph.
• Compiler: An abstraction representing a C++ compiler environment.
• Project and BinaryBuilder: Abstractions for building C++ compilation jobs.

As a general rule, AMBuild uses the following conventions:

• Methods starting with an upper-case letter are considered public API.
• Methods starting with a lower-case letter are considered private and should not be used. There are a few exceptions for brevity or accessor-like functions.
• Properties and attributes ending with an underscore are considered private and should not be used.
• Spaces are used instead of tabs, and four-space indents are preferred.

BuildParsers

BuildParsers are created in configure.py, and the final step of configure.py is to call parser.Configure(), which will begin parsing the root AMBuildScript of the project. BuildParsers have extra properties worth exploring for more complicated projects:

• options - An instance of argparse.ArgumentParser.
• host - The System object for the host system. (see #Platforms)
• default_build_folder - By default, the build folder for a project is "objdir". This property can be set to change the default. It can be a string, or a function taking a BuildParser object and returning a string.

Contexts

Contexts are implemented in ambuild2/frontend/base_gen.py. They are the entry point for using AMBuild.

When using path strings, it is important to note how AMBuild recognizes paths.

• source path strings may be any absolute path in the file system, as long as that path is not within the build folder. Source paths may also be expressed as relative to builder.currentSourcePath, which is the source folder of the currently executing build script.
• output path strings are always relative to builder.buildFolder.

Attributes

• parent - The context that loaded the current context.
• script - The path, relative to sourcePath, of the current build script.
• sourcePath - The absolute path to the source tree.
• options - The result of evaluating command-line options from configure.py, via the optparse module.
• buildPath - The absolute path to the build folder.
• buildFolder - The working directory of jobs in this context, relative to buildPath.
• localFolder - The Entry for buildFolder; None for the root of the build.
• currentSourcePath - The source folder that the current build script is in.
• host - The System object corresponding to the host system (see #Platforms).
• originalCwd - The working directory that was set when the user first configured the build. This is useful when constructing absolute paths from user inputs that contain relative paths.

Custom attributes can be set on Build Contexts. When evaluating nested build scripts, these attributes will be automatically copied. If the object stored at the attribute inherits from ambuild2.frontend.cloneable.Cloneable, then it will be deep copied. Compiler objects (described later on) are always cloned this way, so nested build scripts do not interfere with parent ones.

Methods

• SetBuildFolder(folder) - Sets the working directory of jobs in this context, relative to buildPath.
  • folder - A string representing the folder path. '.' and './' are allowed.
  • Returns None.
• DetectCxx(**kwargs) - Detects C and C++ compilers and raises an exception if neither are available or they do not match. The full rules for this, and the options available, are under #C++ Detection.
  • Returns a Compiler object.
• Add(taskbuilder) - Some jobs are too complex to use a single API call, and instead provide objects to assist in creation. These objects are finalized into the dependency graph with this function. Currently, the only complex jobs built-in are C/C++ compilation jobs.
  • taskbuilder - The job builder instance.
  • Returns a value based on the taskbuilder. For example, BinaryBuilders return a 'CppNode' described under the Compiler section, and Projects return a list of CppNodes.
• AddFolder(folder) - Ensures that a folder is created when performing a build. The folder is created relative to the context's local build folder.
  • folder - A relative path specifying the folder. Folder chains can be created all at once; i.e. 'a/b/c' is a valid target even if 'a' or 'a/b' do not exist.
  • Returns an Entry instance describing the folder creation node.
• AddSymlink(source, output_path) - Adds a job to the build that will perform a symlink. On systems where symlinks are not available, it is implemented as a copy.
• AddCopy(source, output_path) - Adds a job to the build that will perform a file copy.
  • source - Either a string containing a source file path, or a source node, or an output node, representing the file that will be the source of the operation.
  • output_path - One of the following:
    • A string path ending in a path separator, or '.', specifying the folder to copy or symlink the file to. It is relative to the context's local build folder.
    • A string path ending in a filename, representing the destination file of the operation. It is relative to the context's local build folder.
    • A folder node created via AddFolder(), representing the folder to copy or symlink the file to. In this case, the folder node's path is taken as-is and is not relative to the local build folder.
  • See AddCommand for return values.
• AddCommand(inputs, argv, outputs, folder?, dep_type?, weak_inputs?, shared_outputs?) - Adds a custom command that will be executed manually. The working directory of the command is the context's local build folder. As created, the command has no dependencies. If it has source dependencies they can be specified via AddDependency.
  • inputs - An iterable containing source file paths and/or Entry output-file nodes that are incoming dependencies.
  • argv - The argument vector that will be passed to subprocess.Popen. argv[0] should be the executable.
  • outputs - An iterable containing files that are outputs of this command. Each file must be a relative path from the context's local build folder.
  • folder - The working directory for the command. By default, this is buildFolder. It can be None to specify the root of the build. Otherwise, it must be an Entry from calling AddFolder or reading localFolder.
  • dep_type - Specifies whether the output of the command contains a dependency list. The following three values are supported:
    • None - (default) This command does not support dependency discovery or does not have dynamic dependencies.
    • 'msvc' - Dependencies are spewed in the exact manner of the Visual Studio C++ compiler, in stdout.
    • 'gcc' - Dependencies are spewed in the exact manner of the GNU C Compiler or Clang, in stderr.
    • 'sun' - Dependencies are spewed in the exact manner of the Sun Pro compiler, in stderr.
    • 'fxc' - Dependencies are spewed in the exact manner of the FXC compiler, in stdout.
  • weak_inputs - Optional list of weak dependencies. Weak dependencies enforce an ordering between two commands, but an update only occurs if the command is discovered to actually use the dependency (see the Compiler.sourcedeps attribute).
  • shared_outputs - Optional list of "shared" outputs. Shared outputs are files that are generated by multiple commands. This is a degenerate case in AMBuild, but some systems do this, and AMBuild needs to understand where the files came from. Shared outputs can not be used as an input; they do not participate in the dependency system, except that AMBuild knows when to remove them.
  • Returns a 2-tuple, containing:
    • An Entry instance representing the node for this command in the dependency graph.
    • A list of Entry instances, each instance corresponding to one of the output file paths specified.
• AddConfigureFile(path) - Adds a source path that will trigger automatic reconfiguring if the file changes. This is useful for files that are conceptually part of a build script, but are not actually loaded as a build script.
  • path - A source path.
• HasFeature(name) - Returns whether the given named feature is available. This is not currently used, but exists so we can extend the API in a backwards-compatible way in the future.
• ProgramProject(name) - Returns a Project for building executable programs. Projects are described later on.
• LibraryProject(name) - Returns a Project for building dynamically linked libraries. Projects are described later on.
• StaticLibraryProject(name) - Returns a Project for building statically linked libraries. Projects are described later on.
• CloneableList(*args, **kwargs) - Wrapper around list that implements Cloneable. This can be used to make lists that are deep-copied when assigned to an attribute in a build context.
• CloneableDict(*args, **kwargs) - Wrapper around OrderedDict that implements Cloneable. This can be used to make dictionaries that are deep-copied when assigned to an attribute in a build context.

Contexts and Scripts

AMBuild can run additional, user-provided scripts so the build process is not clumped into one giant file. When running sub-scripts, each script has a "context". This context is a sub-builder that inherits various properties, and allows the script to not potentially interfere with variables and state in the global builder. These contexts can be created in one of three ways:

• Child contexts are relative to the context that created them. Their containing folder must be the parent folder or a folder somewhere underneath it, and their output folder is always the parent's output folder or a sub-folder of it.
• Top-Level contexts are child contexts that have no parent; they can change their output folder.
• Empty contexts have no input or output folder; they cannot perform build steps in their own context.

With that in mind, AMBuild provides the following functions on build contexts to assist in running additional scripts. Each of these functions takes in an optional vars parameter; if not provided, it is initialized to an empty dict. Otherwise the newly run script's globals will be merged with vars, along with the vars used to call the invoking script (if any).

Additionally, each of these methods accepts a path (or in some cases, a list of paths). These paths must either be relative to the current source folder, or they may be specified as relative to the root of the source tree via a leading '/'. Paths may not be outside the source tree.

• Import(path, vars?) - Runs path in an empty context, and returns the globals of the completed script as a dict. If path is not a string, and is iterable, each path will be run in iteration order and None will be returned.
• Eval(path, vars?) - Helper function that runs path in an empty context, and returns the value of the global variable rvalue in the completed script (or None if none was present).
• Build(path, vars?) - Runs path in a child context. path may alternately be an iterable of path strings, in which case each is evaluated in iteration order. If path is a single path string, and the executed script ends with a global variable called rvalue, then the value of that variable is returned. Otherwise, None is returned.

Platforms

AMBuild represents the host and target system via System objects. These objects have platform and arch properties. platform may be one of:

• "windows"
• "mac"
• "linux"
• "freebsd"
• "openbsd"
• "netbsd"
• "solaris"
• "cygwin"

arch may be one of:

• "x86_64"
• "x86"
• "arm64"
• "armv*" (where * is the ARM version and configuration, such as "armv5ejl")
• ... Other architectures are untested, but will be added here as tested.

Some patterns are normalized when passed as arguments. For example, "amd64" or "x64" will be normalized to "x86_64" for convenience.

Entry

Entry objects are considered mostly opaque. However, all frontends must define at least these attributes:

• path - A file system path that uniquely represents nodes that are present in the filesystem, such as source files, output files, or folders.

Compiler

Compiler objects encapsulate information about invoking the C or C++ compiler. Most of its attributes are lists of options, so it is best to use += to extend these lists, to avoid replacing previously set options.

Whenever options come in pairs - for example, C/C++ flags versus C++-only flags, C++ flags will automatically include all C flags during compilation time.

Attributes

• includes - List of C and C++ include paths
• cxxincludes - List of C++ include paths.
• cflags - List of C and C++ compiler flags.
• cxxflags - List of C++ compiler flags.
• defines - List of C and C++ #defines, in the form of 'KEY' or 'KEY=VALUE'
• cxxdefines - List of C++ #defines, in the form of 'KEY' or 'KEY=VALUE'
• rcdefines - List of RC (Resource Compiler) #defines, in the form of 'KEY' or 'KEY=VALUE'
• linkflags - Link flags (see below).
• postlink - Array of objects to link, added to the linker flags after linkflags. See below.
• sourcedeps - An array of output nodes which should be weak dependencies on each source compilation node.
• weaklinkdeps - Array of entries that will act as weak inputs to the linker. This is to ensure ordering; they do not affect the actual linker command.
• symbol_files - One of three values:
  • 'bundled' - If possible, debug information will be generated into the binary. On some compilers this is not possible. For example, MSVC always generates .PDB files.
  • 'separate' - Separates debug information into a separate file (a .pdb file, .sym file, or dSYM package depending on the platform).
• version - Returns an object representing the compiler version. It may be stringified with str(). It can also be compared to either integers, other version objects, or strings. For example: compiler.version >= '4.7.4' will return true if the compiler's version is at least 4.7.3. The exact meaning of the version is vendor-dependent; for example, MSVC versions look like large numbers (1800 corresponds to Visual Studio 2013).
• family - Returns the compiler family. This is either MSVC, GCC, Clang, or SunPro.
• behavior - Returns the compiler meta-family, or the most generic compiler this compiler tries to emulate. This is either MSVC, GCC, or SunPro.
• target - Returns the System object representing the platform and architecture tuple. Currently, AMBuild has support for cross-compiling to different architectures, but not to different platforms, so the platform will always match builder.host.platform.

Entries to linkflags and postlink can be:

• A string representing a linker flag.
• An Entry object.
• A Dep object. Dep objects are useful when precise control is needed over what text is given to the linker in order to link in a file. Essentially, they let you customize the link flag while still including a dependency. They also allow lazy computation of dependencies, since sometimes extra steps must be generated before linking to an object. Dep objects have two attributes:
  • text, the text that will be passed to the linker when constructing its argument list.
  • node, an object which tells AMBuild how to build a dependency for the linker flag. It can be:
    • None, meaning that the text is a file path.
    • A file path.
    • An Entry or list of Entry objects representing the output files of a command, or
    • A function which returns the above, and has the signature (builder, binary), receiving a Context object and a BinaryBuilder object.
• Note: AMBuild does not currently support automatic dependency generation for -L style linking.

For example, to generate a linker invocation like "g++ main.o tier1.so -o main", where tier1.so is a generated file, you could do:

binary.postlink += [binary.Dep('tier1.so', tier1_so_entry)]

If this requires symlinking tier1.so to be in the local folder, you can get more complex, such as:

def make_linker_dep(compiler, name, entry):
  def lazy_dep(builder, binary):
    cmd, (output,) = builder.AddSymlink(entry, '.')
    return output
  return compiler.Dep(name, lazy_dep)

Methods

• Program(name) - Creates a new BinaryBuilder instance, with a copy of the compiler settings. The builder is configured to generate an executable (.exe is automatically appended on Windows).
• Library(name) - Creates a new BinaryBuilder instance, with a copy of the compiler settings. The builder is configured to generate a shared library. .so, .dylib, or .dll is automatically appended depending on the platform. Nothing is ever prepended to the name.
• StaticLibrary(name) - Creates a new BinaryBuilder instance, with a copy of the compiler settings. The builder is configured to generate a static library. .a or .lib is automatically appended depending on the platform. Nothing is ever prepended to the name.
• Dep(text, node) - Creates a Dep object instance (see above for more details).
• like(name) - Returns whether the compiler is "like" another compiler. This is intended to represent the compatibility hierarchy of modern compilers. For example:
  • Visual Studio is "like" 'msvc'.
  • GCC is "like" 'gcc'.
  • Clang is "like" 'gcc' and 'clang'.
  • Note that GCC is not "like" Clang.
• pkg_config(pkg, link = 'dynamic') - Runs the pkg-config program for the given package name, and adds relevant includes, cflags, and libraries to the compiler. If link is 'dynamic' (default), shared libraries are used. If link is 'static', then static libraries are used instead.

BinaryBuilder

BinaryBuilder assists in creating C/C++ compilation tasks. Once you've set all the information needed, they are integrated into the dependency graph by using builder.Add().

Attributes

• compiler - A full copy of the compiler settings used when instantiating this BinaryBuilder. Modifying this compiler will not modify the original.
• sources - An (initially empty) list of C/C++ source file paths. They can be absolute paths, or paths relative to currentSourcePath.
• localFolder - The name of the folder this binary will be generated in, relative to the buildFolder of the context that adds the tasks.
• type - One of 'static', 'library', or 'program'.
• custom - A list of custom steps to include in the compilation process. See #Custom C++ Tools.

Methods

• Dep(text, node) - A wrapper for compiler.Dep.
• Module(builder, name) - Creates and returns a new Module object that is attached to the BinaryBuilder it was invoked on. The module object has two properties: compiler, a clone of the current compiler on the BinaryBuilder, and sources, and empty list. Source files added to the module will be compiled as part of the BinaryBuilder that created it, however, they will use the module's compiler settings instead. Modules may be added from any context, and are intended for large monolithic projects that have many components that must be linked together.

Project

Projects assist in creating multiple configurations of a C++ compilation task. It is essentially a factory for BinaryBuilders. It's useful for tying multiple configurations of the same basic component together. The build output is identical to using BinaryBuilder. However, when used with Visual Studio project file generation, you will get one vcxproj file instead of many. Therefore, it's helpful for organization when using AMBuild's IDE tooling.

Project instances can be obtained through the build context, via ProgramProject, LibraryProject, and StaticLibraryProject. You can set the source list via the sources attribute, then use Configure() to add a new configuration. After all configurations have been added, use builder.Add() to add the project to the dependency graph.

When calling builder.Add() on a project, the return value is a list of CppNodes, one for each configuration in the order they were added.

Attributes

• sources - An (initially empty) list of C/C++ source file paths. They can be absolute paths, or paths relative to currentSourcePath.
• localFolder - The name of the folder this binary will be generated in, relative to the buildFolder of the context that adds the tasks.

Methods

• Configure(compiler, name, tag) - Returns a BinaryBuilder with the given name. The compiler must be an object returned by builder.DetectCxx(). The tag is used to describe the configuration for IDE project files.
• Dep(text, node) - A wrapper for compiler.Dep.

C++ Detection

When using the DetectCxx API, AMBuild uses the following logic to find a C++ compiler:

1. If 'CC' and 'CXX' are defined in the environment, those most result in a valid compiler, and no other detection is performed. On Windows, tools like "rc" and "lib" must be in the environment. Defining CC but not CXX (or vice-versa) is an error.
2. If running on Windows, if 'cl.exe' is in PATH, then AMBuild assumes the environment has already been configured, and looks for, in this order: cl, clang, gcc, icc.
3. On Windows, if AMBUILD_VCVARS_<arch> is defined in the environment, AMBuild will use the .bat file specified to create the Visual Studio environment. The <arch> component can either be a normalized architecture name (such as X86_64) or it can be ALL, in which case, the VS architecture name will be passed to the batch file as an argument.
4. On Windows, AMBuild then looks in the registry for Visual Studio 2015. It will also look for the 'vswhere' tool, and if present, will detect installations of Visual Studio 2017 and 2019. The highest working version is used. It will also detect Microsoft Visual C++ Build Tools (2015). If no working version of anything is found, the logic in step 2 is used as a last-ditch attempt.
5. Otherwise, AMBuild tries to run clang, gcc, or icc (in that order).

In all cases, if CFLAGS and/or CXXFLAGS are defined in the environment, they will be propagated into the detected compiler.

Note that on Windows, this logic is heavily dependent on vcvars being functional. For example, Visual Studio 2015 may not configure Windows Kit 10 properly, and AMBuild will not be able to automatically detect such an install. We recommend that for continuous integration, and reproducible, release-quality builds, that the environment be entirely configured beforehand via AMBUILD_VCVARS.

Cross Compilation

AMBuild tries to automatically detect cross-compilation, eg, compiling to non-native triple. A triple is a platform-arch[subarch][-abi] sequence. For example, windows-x86 or linux-armv7-gnueabihf. On Mac and Windows targets, the ABI part of the triple is always dropped since there is only one ABI.

These triples directly correspond to System objects.

When specifying targets via a triple, AMBuild supports some shorthand sequences:

• arch will only change the target architecture. Eg, "x86" on "linux-x86_64" will result in "linux-x86".
• platform-arch works when the ABI can be elided. Eg, "windows-x86".
• arch-abi works when the host and target platform are the same. Eg, "arm-gnueabihf" on "linux-x86" will result in "linux-arm-gnueabihf".

Options

As of AMBuild 2.2, builder.DetectCxx() supports the following options. Options not recognized are ignored.

• target - A string to force the compiler's triple (or shorthand for a triple), as described above. Normally, AMBuild will use the first working compiler it can find, which could theoretically have any target configuration. This will force AMBuild to find a matching compiler, and if none exists, it will fail.
• target_arch - A string to force only a change in architecture. This is useful for projects that do not support cross-platform, but do support cross-architecture, builds.
• force_msvc_version - A string specifying which version of MSVC can be used during automatic detection. This only applies to searches for MSVC installations, not for "cl.exe" in the environment or when CC/CXX has been manually set. The string should be a version number (eg, "14.0", "15.0", or "16.0" etc).

Custom C++ Tools

BinaryBuilders have a "custom tool" list that allows build scripts to hook into the compilation process. Usually, custom entries will be fully provided by an AMBuild helper or a third party helper. However it is possible to write your own custom tools.

FXC is a good example of when a custom tool is needed. FXC is a Microsoft tool for compiling HLSL shaders. It has a mode to generate C++ headers, but it's extremely cumbersome to work with, and AMBuild can hide that complexity while safely integrating it into the dependency graph.

Custom tools are objects added to the custom list on a BinaryBuilder. This object must have a tool attribute, containing an object with the following methods:

• evaluate(cmd) - Called when the BinaryBuilder is submitted to the build tool.

The cmd parameter to evaluate is an object with the following properties:

• context - The build context for the module that the tool will run against.
• localFolderNode - A folder node, representing the output folder for the module's object files.
• data - The custom object given to the BinaryBuilder, that this tool should process.
• compiler - The compiler object for the module this tool is running in.

Additionally, the cmd object has properties which are considered outputs:

• sources - An optional list of additional sources to compile. The tool may append new items.
• sourcedeps - An optional list of additional source dependencies. The module's sourcedeps array will be extended to include anything the tool adds here.

Finally, the cmd tool has helper methods:

• NameForObjectFile(path) - Computes a "safe" object file name for the given path. All characters that do not conform to C++ identifier name rules are replaced with an underscore.
• ComputeSourcePath(path) - Computes a command-line safe source path for an external tool. The output is either an absolute path, or a path relative to cmd.localFolderNode.
• CustomSource(source, weak_deps=[]) - Returns an object that acts as a source file for a source file list. However, it can be annotated with extra instructions to the build process.
  • weak_deps - An additional list of weak dependencies for this source file.

When the binary is submitted to the build, the custom tool will have the chance to include any extra steps it wants. It should execute these steps, if possible, in the same folder as cmd.localFolderNode. If these steps introduce new source files, they should be included in cmd.sources. New weak dependencies should be included in cmd.sourcedeps. If no dependencies at all are introduced, then you probably don't need a custom tool, as the C++ binary is not dependent on anything custom.

Predefined Custom C++ Tools

• See AMBuild FXC API

Changes from 2.1

• Context.cxx has been removed. You can set it manually after calling DetectCxx if you do not support multi-architecture builds.
• The target field on Contexts has been moved to the Compiler object.
• Projects may no longer be created from Compiler objects. They are created from contexts, and compilers are attached when a project binary is created.
• The BuildParser.options field is now an argparse.ArgumentParser, rather than an optparse.OptionParser.

Changes from 2.0

If upgrading from the 2.0 API, the following changes should be noted:

• Context.DetectCompilers has been renamed to Context.DetectCxx.
• Context.compiler has been removed. You can set it manually after calling DetectCxx.
• Context.RunScript and RunBuildScripts have been renamed to Context.Build.
• Paths to RunBuildScript[s] may now be specified as relative to the root of the source tree, by prefixing the path with '/'.
• The vendor property on Compiler is now undefined. It must not be accessed or compared. Use the new family or behavior accessors, or use the like() method.
• The debuginfo property on Compiler has been renamed to symbol_files, and it no longer accepts None.
• The cc, cxx, and argv properties on Compiler have been removed.
• The host_platform field on Contexts has been replaced with the host System object. It is enough to replace host_platform with host.platform.
• The target_platform field on Contexts has been moved to the Compiler object. It's now a System object and has been renamed to target.
• It is no longer possible to run a script from a context that is no longer the active context.
• The BuildParser.options field is now an argparse.ArgumentParser, rather than an optparse.OptionParser.