AMBuild API (2.1)

This documentation is for the AMBuild 2.1 API. Click here for the 2.0 API.

AMBuild scripts have access to the following object types:

• BuildParser: This tells AMBuild how to begin parsing your AMBuildScript. It is usually created in configure.py.
• Context: Exposed as the builder, this is the entry point for the API and is known as a configure context.
• Entry: Represents a node in the dependency graph.
• Compiler: An abstraction representing a C++ compiler environment.
• ProjectBuilder and BinaryBuilder: Abstractions for building C++ compilation jobs.

As a general rule, AMBuild uses the following conventions:

• Methods starting with an upper-case letter are considered public API.
• Methods starting with a lower-case letter are considered private and should not be used. There are a few exceptions for brevity or accessor-like functions.
• Properties and attributes ending with an underscore are considered private and should not be used.
• Spaces are used instead of tabs, and two-space indents are preferred.

BuildParsers

BuildParsers are created in configure.py, and the final step of configure.py is to call parser.Configure(), which will begin parsing the root AMBuildScript of the project. BuildParsers have extra properties worth exploring for more complicated projects:

• options - An instance of optparse.OptionParser. Note that AMBuild can run on either Python 2 or Python 3, so it is best to use the subset which is common to both versions.
• host - The System object for the host system. (see #Platforms)
• default_build_folder - By default, the build folder for a project is a string constructed out of the platform and architecture, such as "obj-linux-i386". This property can be set to change the default. It can be a string, or a function taking a BuildParser object and returning a string.
• target_arch - If this project only supports a single target architecture, it may be specified here. Otherwise, the target architecture will be inherited from the host architecture, and can be further overridden by the --target-arch command-line option.
• default_arch - By default, this is None. A project can set this to a default architecture if it prefers one. --target-arch will still override it if specified.

Note that architectures specified on the command line are currently not normalized - they are passed as given.

Contexts

Contexts are implemented in ambuild2/frontend/base_gen.py. They are the entry point for using AMBuild.

When using path strings, it is important to note how AMBuild recognizes paths.

• source path strings may be any absolute path in the file system, as long as that path is not within the build folder. Source paths may also be expressed as relative to builder.currentSourcePath, which is the source folder of the currently executing build script.
• output path strings are always relative to builder.buildFolder.

Attributes

• cxx - An instance of a Compiler object, or None if DetectCxx was never called. Calling DetectCxx will set the compiler field for this and all future contexts. Each context gets a clone of its parent's compiler, so it is safe to modify compilers in inner scripts.
• parent - The context that loaded the current context.
• script - The path, relative to sourcePath, of the current build script.
• sourcePath - The absolute path to the source tree.
• options - The result of evaluating command-line options from configure.py, via the optparse module.
• buildPath - The absolute path to the build folder.
• buildFolder - The working directory of jobs in this context, relative to buildPath.
• localFolder - The Entry for buildFolder; None for the root of the build.
• currentSourcePath - The source folder that the current build script is in.
• target - The System object corresponding to the desired target system (see #Platforms).
• host - The System object corresponding to the host system (see #Platforms).
• originalCwd - The working directory that was set when the user first configured the build. This is useful when constructing absolute paths from user inputs that contain relative paths.

Methods

• SetBuildFolder(folder) - Sets the working directory of jobs in this context, relative to buildPath.
  • folder - A string representing the folder path. '.' and './' are allowed.
  • Returns None.
• DetectCxx(options = {}) - Detects C and C++ compilers and raises an exception if neither are available or they do not match. The full rules for this, and the options dictionary, are under #C++ Detection.
  • Returns a Compiler object that is also accessible via the compiler attribute.
• Add(taskbuilder) - Some jobs are too complex to use a single API call, and instead provide objects to assist in creation. These objects are finalized into the dependency graph with this function. Currently, the only complex jobs built-in are C/C++ compilation jobs.
  • taskbuilder - The job builder instance.
  • Returns a value based on the taskbuilder. For example, BinaryBuilders return a 'CppNode' described under the Compiler section, and ProjectBuilders return a list of CppNodes.
• AddFolder(folder) - Ensures that a folder is created when performing a build. The folder is created relative to the context's local build folder.
  • folder - A relative path specifying the folder. Folder chains can be created all at once; i.e. 'a/b/c' is a valid target even if 'a' or 'a/b' do not exist.
  • Returns an Entry instance describing the folder creation node.
• AddSymlink(source, output_path) - Adds a job to the build that will perform a symlink. On systems where symlinks are not available, it is implemented as a copy.
• AddCopy(source, output_path) - Adds a job to the build that will perform a file copy.
  • source - Either a string containing a source file path, or a source node, or an output node, representing the file that will be the source of the operation.
  • output_path - One of the following:
    • A string path ending in a path separator, or '.', specifying the folder to copy or symlink the file to. It is relative to the context's local build folder.
    • A string path ending in a filename, representing the destination file of the operation. It is relative to the context's local build folder.
    • A folder node created via AddFolder(), representing the folder to copy or symlink the file to. In this case, the folder node's path is taken as-is and is not relative to the local build folder.
  • See AddCommand for return values.
• AddCommand(inputs, argv, outputs, folder?, dep_type?, weak_inputs?, shared_outputs?) - Adds a custom command that will be executed manually. The working directory of the command is the context's local build folder. As created, the command has no dependencies. If it has source dependencies they can be specified via AddDependency.
  • inputs - An iterable containing source file paths and/or Entry output-file nodes that are incoming dependencies.
  • argv - The argument vector that will be passed to subprocess.Popen. argv[0] should be the executable.
  • outputs - An iterable containing files that are outputs of this command. Each file must be a relative path from the context's local build folder.
  • folder - The working directory for the command. By default, this is buildFolder. It can be None to specify the root of the build. Otherwise, it must be an Entry from calling AddFolder or reading localFolder.
  • dep_type - Specifies whether the output of the command contains a dependency list. The following three values are supported:
    • None - (default) This command does not support dependency discovery or does not have dynamic dependencies.
    • 'msvc' - Dependencies are spewed in the exact manner of the Visual Studio C++ compiler, in stdout.
    • 'gcc' - Dependencies are spewed in the exact manner of the GNU C Compiler or Clang, in stderr.
    • 'sun' - Dependencies are spewed in the exact manner of the Sun Pro compiler, in stderr.
    • 'fxc' - Dependencies are spewed in the exact manner of the FXC compiler, in stdout.
  • weak_inputs - Optional list of weak dependencies. Weak dependencies enforce an ordering between two commands, but an update only occurs if the command is discovered to actually use the dependency (see the Compiler.sourcedeps attribute).
  • shared_outputs - Optional list of "shared" outputs. Shared outputs are files that are generated by multiple commands. This is a degenerate case in AMBuild, but some systems do this, and AMBuild needs to understand where the files came from. Shared outputs can not be used as an input; they do not participate in the dependency system, except that AMBuild knows when to remove them.
  • Returns a 2-tuple, containing:
    • An Entry instance representing the node for this command in the dependency graph.
    • A list of Entry instances, each instance corresponding to one of the output file paths specified.
• AddConfigureFile(path) - Adds a source path that will trigger automatic reconfiguring if the file changes. This is useful for files that are conceptually part of a build script, but are not actually loaded as a build script.
  • path - A source path.

Contexts and Scripts

AMBuild can run additional, user-provided scripts so the build process is not clumped into one giant file. When running sub-scripts, each script has a "context". This context is a sub-builder that inherits various properties, and allows the script to not potentially interfere with variables and state in the global builder. These contexts can be created in one of three ways:

• Child contexts are relative to the context that created them. Their containing folder must be the parent folder or a folder somewhere underneath it, and their output folder is always the parent's output folder or a sub-folder of it.
• Top-Level contexts are child contexts that have no parent; they can change their output folder.
• Empty contexts have no input or output folder; they cannot perform build steps in their own context.

With that in mind, AMBuild provides the following functions on build contexts to assist in running additional scripts. Each of these functions takes in an optional vars parameter; if not provided, it is initialized to an empty dict. Otherwise the newly run script's globals will be merged with vars, along with the vars used to call the invoking script (if any).

Additionally, each of these methods accepts a path (or in some cases, a list of paths). These paths must either be relative to the current source folder, or they may be specified as relative to the root of the source tree via a leading '/'. Paths may not be outside the source tree.

• Import(path, vars?) - Runs path in an empty context, and returns the globals of the completed script as a dict. If path is not a string, and is iterable, each path will be run in iteration order and None will be returned.
• Eval(path, vars?) - Helper function that runs path in an empty context, and returns the value of the global variable rvalue in the completed script (or None if none was present).
• Build(path, vars?) - Runs path in a child context. path may alternately be an iterable of path strings, in which case each is evaluated in iteration order. If path is a single path string, and the executed script ends with a global variable called rvalue, then the value of that variable is returned. Otherwise, None is returned.

Platforms

AMBuild represents the host and target system via System objects. These objects have platform and arch properties. platform may be one of:

• "windows"
• "mac"
• "linux"
• "freebsd"
• "openbsd"
• "netbsd"
• "solaris"
• "cygwin"

arch may be one of:

• "x86_64"
• "x86"
• "arm64"
• "armv*" (where * is the ARM version and configuration, such as "armv5ejl")
• ... Other architectures are untested, but will be added here as tested.

The target architecture represents what the build should attempt to configure itself for. AMBuild does not attempt to configure the compiler or environment for cross-compiling.

Entry

Entry objects are considered mostly opaque. However, all frontends must define at least these attributes:

• path - A file system path that uniquely represents nodes that are present in the filesystem, such as source files, output files, or folders.

Compiler

Compiler objects encapsulate information about invoking the C or C++ compiler. Most of its attributes are lists of options, so it is best to use += to extend these lists, to avoid replacing previously set options.

Whenever options come in pairs - for example, C/C++ flags versus C++-only flags, C++ flags will automatically include all C flags during compilation time.

Attributes

• includes - List of C and C++ include paths
• cxxincludes - List of C++ include paths.
• cflags - List of C and C++ compiler flags.
• cxxflags - List of C++ compiler flags.
• defines - List of C and C++ #defines, in the form of 'KEY' or 'KEY=VALUE'
• cxxdefines - List of C++ #defines, in the form of 'KEY' or 'KEY=VALUE'
• rcdefines - List of RC (Resource Compiler) #defines, in the form of 'KEY' or 'KEY=VALUE'
• linkflags - Link flags (see below).
• postlink - Array of objects to link, added to the linker flags after linkflags. See below.
• sourcedeps - An array of output nodes which should be weak dependencies on each source compilation node.
• weaklinkdeps - Array of entries that will act as weak inputs to the linker. This is to ensure ordering; they do not affect the actual linker command.
• symbol_files - One of three values:
  • 'bundled' - If possible, debug information will be generated into the binary. On some compilers this is not possible. For example, MSVC always generates .PDB files.
  • 'separate' - Separates debug information into a separate file (a .pdb file, .sym file, or dSYM package depending on the platform).
• version - Returns an object representing the compiler version. It may be stringified with str(). It can also be compared to either integers, other version objects, or strings. For example: compiler.version >= '4.7.4' will return true if the compiler's version is at least 4.7.3. The exact meaning of the version is vendor-dependent; for example, MSVC versions look like large numbers (1800 corresponds to Visual Studio 2013).
• family - Returns the compiler family. This is either MSVC, GCC, Clang, or SunPro.
• behavior - Returns the compiler meta-family, or the most generic compiler this compiler tries to emulate. This is either MSVC, GCC, or SunPro.

Entries to linkflags and postlink can be:

• A string representing a linker flag.
• An Entry object.
• A Dep object. Dep objects are useful when precise control is needed over what text is given to the linker in order to link in a file. Essentially, they let you customize the link flag while still including a dependency. They also allow lazy computation of dependencies, since sometimes extra steps must be generated before linking to an object. Dep objects have two attributes:
  • text, the text that will be passed to the linker when constructing its argument list.
  • node, an object which tells AMBuild how to build a dependency for the linker flag. It can be:
    • None, meaning that the text is a file path.
    • A file path.
    • An Entry or list of Entry objects representing the output files of a command, or
    • A function which returns the above, and has the signature (builder, binary), receiving a Context object and a BinaryBuilder object.
• Note: AMBuild does not currently support automatic dependency generation for -L style linking.

For example, to generate a linker invocation like "g++ main.o tier1.so -o main", where tier1.so is a generated file, you could do:

binary.postlink += [binary.Dep('tier1.so', tier1_so_entry)]

If this requires symlinking tier1.so to be in the local folder, you can get more complex, such as:

def make_linker_dep(compiler, name, entry):
  def lazy_dep(builder, binary):
    cmd, (output,) = builder.AddSymlink(entry, '.')
    return output
  return compiler.Dep(name, lazy_dep)

Methods

There are two APIs for creating C++ binaries - ProjectBuilder and BinaryBuilder. ProjectBuilder makes it possible to group related tasks together. For example, if one binary has multiple build configurations, a Builder will allow you to group them together with a common set of source files. For building, the results are exactly the same as creating individual BinaryBuilders.

For generating IDE project files, the distinction can be important. When the same binary is built across multiple configurations, AMBuild generates a new project file for each configuration. Using ProjectBuilder over BinaryBuilder allows AMBuild to store each of the configurations in a single project file.

• Program(name) - Creates a new BinaryBuilder instance, with a copy of the compiler settings. The builder is configured to generate an executable (.exe is automatically appended on Windows).
• Library(name) - Creates a new BinaryBuilder instance, with a copy of the compiler settings. The builder is configured to generate a shared library. .so, .dylib, or .dll is automatically appended depending on the platform. Nothing is ever prepended to the name.
• StaticLibrary(name) - Creates a new BinaryBuilder instance, with a copy of the compiler settings. The builder is configured to generate a static library. .a or .lib is automatically appended depending on the platform. Nothing is ever prepended to the name.
• ProgramProject(name) - Creates a new ProjectBuilder instance, with a copy of the compiler settings. The builder is configured to generate an executable (.exe is automatically appended on Windows).
• LibraryProject(name) - Creates a new ProjectBuilder instance, with a copy of the compiler settings. The builder is configured to generate a shared library. .so, .dylib, or .dll is automatically appended depending on the platform. Nothing is ever prepended to the name.
• StaticLibraryProject(name) - Creates a new ProjectBuilder instance, with a copy of the compiler settings. The builder is configured to generate a static library. .a or .lib is automatically appended depending on the platform. Nothing is ever prepended to the name.
• Dep(text, node) - Creates a Dep object instance (see above for more details).
• like(name) - Returns whether the compiler is "like" another compiler. This is intended to represent the compatibility hierarchy of modern compilers. For example:
  • Visual Studio is "like" 'msvc'.
  • GCC is "like" 'gcc'.
  • Clang is "like" 'gcc' and 'clang'.
  • Note that GCC is not "like" Clang.
• pkg_config(pkg, link = 'dynamic') - Runs the pkg-config program for the given package name, and adds relevant includes, cflags, and libraries to the compiler. If link is 'dynamic' (default), shared libraries are used. If link is 'static', then static libraries are used instead.

BinaryBuilder

BinaryBuilder assists in creating C/C++ compilation tasks. Once you've set all the information needed, they are integrated into the dependency graph by using builder.Add().

Attributes

• compiler - A full copy of the compiler settings used when instantiating this BinaryBuilder. Modifying this compiler will not modify the original builder.cxx.
• sources - An (initially empty) list of C/C++ source file paths. They can be absolute paths, or paths relative to currentSourcePath.
• localFolder - The name of the folder this binary will be generated in, relative to the buildFolder of the context that adds the tasks.
• type - One of 'static', 'library', or 'program'.
• custom - A list of custom steps to include in the compilation process. See #Custom C++ Tools.

Methods

• Dep(text, node) - A wrapper for compiler.Dep.
• Module(builder, name) - Creates and returns a new Module object that is attached to the BinaryBuilder it was invoked on. The module object has two properties: compiler, a clone of the current compiler on the BinaryBuilder, and sources, and empty list. Source files added to the module will be compiled as part of the BinaryBuilder that created it, however, they will use the module's compiler settings instead. Modules may be added from any context, and are intended for large monolithic projects that have many components that must be linked together.

ProjectBuilder

ProjectBuilder assists in creating multiple configurations of a C++ compilation task. Set the source list via sources, then use Configure() to add a new configuration. After all configurations have been added, use builder.Add() to add the project to the dependency graph.

When calling builder.Add() on a project, the return value is a list of CppNodes, one for each configuration in the order they were added.

Attributes

• compiler - A full copy of the compiler settings used when instantiating this ProjectBuilder. Modifying this compiler will not modify the original builder.compiler.
• sources - An (initially empty) list of C/C++ source file paths. They can be absolute paths, or paths relative to currentSourcePath.
• localFolder - The name of the folder this binary will be generated in, relative to the buildFolder of the context that adds the tasks.

Methods

• Configure(name, tag) - Returns a BinaryBuilder with the given name. The tag is used to describe the configuration for IDE project files.
• Dep(text, node) - A wrapper for compiler.Dep.

C++ Detection

When using the DetectCxx API, AMBuild uses the following logic to find a C++ compiler:

1. If 'CC' and 'CXX' are defined in the environment, those most result in a valid compiler, and no other detection is performed. On Windows, tools like "rc" and "lib" must be in the environment. Defining CC but not CXX (or vice-versa) is an error.
2. If running on Windows, if 'cl.exe' is in PATH, then AMBuild assumes the environment has already been configured, and looks for, in this order: cl, clang, gcc, icc.
3. If running on Windows, and 'cl.exe' is not in PATH, then AMBuild looks in the registry for Visual Studio 2015. It will also look for the 'vswhere' tool, and if present, will detect installations of Visual Studio 2017 and 2019. The highest working version is used. If no working version is found, the logic in step 2 is used as a last-ditch attempt.
4. Otherwise, AMBuild tries to run clang, gcc, or icc (in that order).

Note that on Windows, this logic is heavily dependent on vcvars being functional. For example, Visual Studio 2015 may not configure newer Windows Kits properly, and AMBuild will not be able to automatically detect such an install. We recommend that for continuous integration, and reproducible, release-quality builds, that the environment be entirely configured beforehand (eg, the environment set up, and CC/CXX set properly).

Options

As of AMBuild 2.1.1, options may be specified in adictionary given to DetectCxx. Any unused options will be left in the dictionary after the call returns, and any consumed options will be removed.

Available options:

• force_msvc_version - A string specifying which version of MSVC can be used during automatic detection. This only applies to searches for MSVC installations, not for "cl.exe" in the environment or when CC/CXX has been manually set. The string should be a version number (eg, "14.0", "15.0", or "16.0" etc).

Not that since the options argument is specific to AMBuild 2.1.1 and higher, projects wanting to maintain backwards compatibility can use this snippet to detect its availability:

    apiVersion = getattr(builder, 'apiVerison', None)
    if apiVersion and apiVersion >= '2.1.1':
    	cxx = builder.DetectCxx(cxx_options)
    else:
    	cxx = builder.DetectCxx()

Custom C++ Tools

BinaryBuilders have a "custom tool" list that allows build scripts to hook into the compilation process. Usually, custom entries will be fully provided by an AMBuild helper or a third party helper. However it is possible to write your own custom tools.

FXC is a good example of when a custom tool is needed. FXC is a Microsoft tool for compiling HLSL shaders. It has a mode to generate C++ headers, but it's extremely cumbersome to work with, and AMBuild can hide that complexity while safely integrating it into the dependency graph.

A custom entry may be anything, but it must have a property called tool. This property must contain an object with these methods:

• evaluate(cmd) - Called when the BinaryBuilder is submitted to the build tool.

The cmd parameter to evaluate is an object with the following properties:

• context - The build context for the module that the tool will run against.
• localFolderNode - A folder node, representing the output folder for the module's object files.
• data - The custom object given to the BinaryBuilder, that this tool should process.
• compiler - The compiler object for the module this tool is running in.

Additionally, the cmd object has properties which are considered outputs:

• sources - An optional list of additional sources to compile. The tool may append new items.
• sourcedeps - An optional list of additional source dependencies. The module's sourcedeps array will be extended to include anything the tool adds here.

Finally, the cmd tool has helper methods:

• NameForObjectFile(path) - Computes a "safe" object file name for the given path. All characters that do not conform to C++ identifier name rules are replaced with an underscore.
• ComputeSourcePath(path) - Computes a command-line safe source path for an external tool. The output is either an absolute path, or a path relative to cmd.localFolderNode.
• CustomSource(source, weak_deps=[]) - Returns an object that acts as a source file for a source file list. However, it can be annotated with extra instructions to the build process.
  • weak_deps - An additional list of weak dependencies for this source file.

When the binary is submitted to the build, the custom tool will have the chance to include any extra steps it wants. It should execute these steps, if possible, in the same folder as cmd.localFolderNode. If these steps introduce new source files, they should be included in cmd.sources. New weak dependencies should be included in cmd.sourcedeps. If no dependencies at all are introduced, then you probably don't need a custom tool, as the C++ binary is not dependent on anything custom.

Predefined Custom C++ Tools

• See AMBuild FXC API

Changes from 2.0

If upgrading from the 2.0 API, the following changes should be noted:

• Context.DetectCompilers has been renamed to Context.DetectCxx.
• Context.compiler has been renamed to Context.cxx.
• Context.RunScript and RunBuildScripts have been renamed to Context.Build.
• Paths to RunBuildScript[s] may now be specified as relative to the root of the source tree, by prefixing the path with '/'.
• The vendor property on Compiler is now undefined. It must not be accessed or compared. Use the new family or behavior accessors, or use the like() method.
• The debuginfo property on Compiler has been renamed to symbol_files, and it no longer accepts None.
• The cc, cxx, and argv properties on Compiler have been removed.
• The target_platform and host_platform fields on Contexts have been replaced with the target and host System objects. It is enough to replace target_platform with target.platform and host_platform with host.platform.
• It is no longer possible to run a script from a context that is no longer the active context.