Aspects: the fan-fic of build rules

Aspects are a feature of Bazel that are basically like fan-fic, if build rules were stories: aspects let you add features that require intimate knowledge of the build graph, but that that the rule maintainer would never want to add.

For example, let’s say we want to be able to generate Makefiles from a Bazel project’s C++ targets. Bazel isn’t going to add support for this to the built-in C++ rules. However, lots of projects might want to support a couple of build systems, so it would be nice to be able to automatically generate build files for Make. So let’s say we have a simple Bazel C++ project with a couple of rules in the BUILD file:

cc_library(
    name = "lib",
    srcs = ["lib.cc"],
    hdrs = ["lib.h"],
)
 
cc_binary(
    name = "bin",
    srcs = ["bin.cc"],
    deps = [":lib"],
)

We can use aspects to piggyback on Bazel’s C++ rules and generate new outputs (Makefiles) from them. It’ll take each Bazel C++ rule and generate a .o-file make target for it. For the cc_binary, it’ll link all of the .o files together. Basically, we’ll end up with a Makefile containing:

bin : bin.o lib.o
	g++ -o bin bin.o lib.o
 
bin.o : bin.cc
	g++ -c bin.cc
 
lib.o : lib.cc
	g++ -c lib.cc

(If you have any suggestions about how to make this better, please let me know in the comments, I’m definitely not an expert on Makefiles and just wanted something super-simple.) I’m assuming a basic knowledge of Bazel and Skylark (e.g., you’ve written a Skylark macro before).

Create a .bzl file to hold your aspect. I’ll call mine make.bzl. Add the aspect definition:

makefile = aspect(
    implementation = _impl,
    attr_aspects = ["deps"],
)

This means that the aspect will follow the “deps” attribute to traverse the build graph. We’ll invoke it on //:bin, and it’ll follow //:bin‘s dep to //:lib. The aspect’s implementation will be run on both of these targets.

Add the _impl function. We’ll start by just generating a hard-coded Makefile:

def _impl(target, ctx):
  # If this is a cc_binary, generate the actual Makefile.
  outputs = []
  if ctx.rule.kind == "cc_binary":
    output = ctx.new_file("Makefile")
    content = "bin : bin.cc lib.cc lib.h\n\tg++ -o bin bin.cc lib.cc\n"
    ctx.file_action(content = content, output = output)
    outputs = [output]
 
  return struct(output_groups = {"makefiles" : set(outputs)})

Now we can run this:

$ bazel build //:bin --aspects make.bzl%makefile --output_groups=makefiles
INFO: Found 1 target...
INFO: Elapsed time: 0.901s, Critical Path: 0.00s
$

Bazel doesn’t print anything, but it has generated bazel-bin/Makefile. Let’s create a symlink to it in our main directory, since we’ll keep regenerating it and trying it out:

$ ln -s bazel-bin/Makefile Makefile 
$ make
g++ -o bin bin.cc lib.cc
$

The Makefile works, but is totally hard-coded. To make it more dynamic, first we’ll make the aspect generate a .o target for each Bazel rule. For this, we need to look at the sources and propagate that info up.

The base case is:

  source_list= [f.path for src in ctx.rule.attr.srcs for f in src.files]
  cmd = target.label.name + ".o : {sources}\n\tg++ -c {sources}".format(
      sources = " ".join(source_list)
  )

Basically: run g++ on all of the srcs for a target. You can add a print(cmd) to see what cmd ends up looking like. (Note: We should probably do something with headers and include paths here, too, but I’m trying to keep things simple and it isn’t necessary for this example.)

Now we want to collect this command, plus all of the commands we’ve gotten from any dependencies (since this aspect will have already run on them):

  transitive_cmds = [cmd]
  for dep in ctx.rule.attr.deps:
    transitive_cmds += dep.cmds

Finally, at the end of the function, we’ll return this whole list of commands, so that rules “higher up” in the tree have deps with a “cmds” attribute:

  return struct(
      output_groups = {"makefiles" : set(outputs)},
      cmds = transitive_cmds,
  )

Now we can change our output file to use this list:

    ctx.file_action(
        content = "\n\n".join(transitive_cmds) + "\n",
        output = output
    )

Altogether, our aspect implementation now looks like:

def _impl(target, ctx):
  source_list= [f.path for src in ctx.rule.attr.srcs for f in src.files]
  cmd = target.label.name + ".o : {sources}\n\tg++ -c {sources}".format(
      sources = " ".join(source_list)
  )
 
  # Collect all of the previously generated Makefile targets.                                                                                                                                                                                                                                                                  
  transitive_cmds = [cmd]
  for dep in ctx.rule.attr.deps:
    transitive_cmds += dep.cmds
 
  # If this is a cc_binary, generate the actual Makefile.                                                                                                                                                                                                                                                                      
  outputs = []
  if ctx.rule.kind == "cc_binary":
    output = ctx.new_file("Makefile")
    ctx.file_action(
        content = "\n\n".join(transitive_cmds) + "\n",
        output = output
    )
    outputs = [output]
 
  return struct(
      output_groups = {"makefiles" : set(outputs)},
      cmds = transitive_cmds,
  )

If we run this, we get the following Makefile:

bin.o : bin.cc
	g++ -c bin.cc
 
lib.o : lib.cc
	g++ -c lib.cc

Getting closer!

Now we need the last “bin” target to be automatically generated, so we need to keep track of all the intermediate .o files we’re going to link together. To do this, we’ll add a “dotos” list that this aspect propagates up the deps.

This is similar to the transitive_cmds list, so add a couple lines to our deps traversal function:

  # Collect all of the previously generated Makefile targets.                                                                                                                                                                                                                                                                  
  dotos = [ctx.label.name + ".o"]
  transitive_cmds = [cmd]
  for dep in ctx.rule.attr.deps:
    dotos += dep.dotos
    transitive_cmds += dep.cmds

Now propagate them up the tree:

  return struct(
      output_groups = {"makefiles" : set(outputs)},
      cmds = transitive_cmds,
      dotos = dotos,
  )

And finally, add binary target to the Makefile:

  # If this is a cc_binary, generate the actual Makefile.                                                                                                                                                                                                                                                                      
  outputs = []
  if ctx.rule.kind == "cc_binary":
    output = ctx.new_file("Makefile")
    content = "{binary} : {dotos}\n\tg++ -o {binary} {dotos}\n\n{deps}\n".format(
        binary = target.label.name,
        dotos = " ".join(dotos),
        deps = "\n\n".join(transitive_cmds)
    )
    ctx.file_action(content = content, output = output)
    outputs = [output]

If we run this, we get:

bin : bin.o lib.o
	g++ -o bin bin.o lib.o
 
bin.o : bin.cc
	g++ -c bin.cc
 
lib.o : lib.cc
	g++ -c lib.cc

Documentation about aspects can be found on bazel.io. Like Skylark rules, I find aspects a little difficult to read because they are inherently recursive functions, but it helps to break it down (and use lots of prints).

  • Watson Ladd

    With GNU make and built-in rules you would only need bin: bin.o lib.o. Make knows how to turn x.cc into x.o automatically.

  • kristina1

    Thanks! I’d assume that’s not true if bin.cc/lib.cc needed custom include paths?

  • Watson Ladd

    The rules include variables like LDLIBS and LFLAGS and INCLUDE (not entirely sure of the names) so quite a bit can be done through changing those variables.

kristina chodorow's blog