API changes with extra cheese, hold the fear

Rihanna's dress

When you make a change, how do you know what tests to run? If you’re lucky, no one else depends on your code so you can just run your own tests and you’re done. However, if you’re writing useful code, other people are probably going to start depending on it. Once that happens, it becomes difficult to make changes without breaking them. Bazel can make this easier, by letting you figure out all of the targets that are depending on your code.

Suppose we are working on the pizza library and we need some cheese, so we create a cheese library and depend on it from pizza. If we look at our build graph, it will look something like this:

graph

//italian:pizza is depending on //ingredients:cheese, as expected.

A few weeks later, the macaroni team discovers that it could also use cheese, so it starts depending on our library. Now our build graph looks like this:

graph

Both our team’s pizza target and the macaroni team’s mac_lib target are depending on //ingredients:cheese. However, Team Macaroni never told us that they’re depending on cheese, so as far as we know, we’re still its only users. Suppose we decide to make a backwards-breaking change (e.g., make Cheese::setMilkfat() private). We make our change, run all of the pizza– and cheese-related tests, submit it… and break //american:mac_and_cheese as well as a dozen other projects who were calling setMilkfat() (that we didn’t know about).

If we had known that other people were depending on our code, we could have let them know that they needed to update their API usage. But how could we find out? With Bazel, we can query for everyone depending on our library:

$ bazel query 'rdeps(//..., //ingredients:cheese)'

This means: “query for every target in our workspace that depends on //ingredients:cheese.”

Now we can check that everything in our code base still builds with our cheese changes by running:

$ bazel build $(bazel query 'rdeps(//..., //ingredients:cheese)')

Just because they built doesn’t mean they work correctly! We can then find all of the tests that depend on cheese and run them:

$ bazel test $(bazel query 'kind(test, rdeps(//..., //ingredients:cheese))')

Unpacking that from the innermost parentheses, that means: “find the targets depending on //ingredients:cheese (rdeps(...)), search those for targets that are tests (kind(test, ...)), and run all of those targets (bazel test ...).”

Running that set of builds and tests is a pretty good check that everything that depends on cheese still works. I mean, if they didn’t write a test for it, it can’t matter too much, right?

macandcheese1

Right.

  • ittai zeidman

    Running “bazel test //…” would be as correct, right?
    If so then what’s the motivation for running the above?
    I can think of a few but I’m not sure how expensive they are (calculating what are all targets, checking the cache to know what not to run)
    Intuitively I always want to run what you wrote since I always want to know I’m not breaking any clients, no?

  • kristina1

    You’re correct, `bazel test //…` is equivalent. However, it’s also more expensive if you’re just looking to see “does anyone depend on this?” Also, once you have tens of thousands of packages, it can take forever for your machine to chew through the upward transitive closure of everything. This lets you break it up, say, `bazel query ‘rdeps(//third_party/…, //ingredients:cheese)’`, `bazel query ‘rdeps(//src/…, //ingredients:cheese)’`, etc.

  • ittai zeidman

    I guessed it’s more expensive then your suggestion but I don’t understand exactly why.
    If I run “bazel test //…” twice in a row, what doesn’t get cached?
    I assumed that discovering of which targets exist is cached and also the transitive closure is cached. What am I missing?

    Thanks!

  • kristina1

    I just meant querying is less expensive than running tests, if you don’t need to run the tests. If the test results are cached from a previous run, then they should be equivalent.

kristina chodorow's blog