Combining projects without converting to a monorepo

Bazel allows you to combine multiple directories from across your filesystem and pretend all of the sources are part of your project. This is a little hard to picture, so let’s use a concrete example.

Let’s say you have two projects you’re working on, checked out at ~/gitroot/spaghetti-stable and ~/gitroot/meatballs-master. You don’t want to combine them into one repository, but you have integration tests that run your meatballs service on top of your spaghetti platform and end-to-end tests that make requests (it forks them).

That is, the filesystem might look something like this:

gitroot/
  spaghetti-stable/
    WORKSPACE
    spaghetti/
      BUILD
      plate_of_spaghetti.cc
  meatballs-master/
    WORKSPACE
    meatballs/
      BUILD
      pile_of_meatballs.cc
      end_to_end_test.cc

I’ve made up these directories on GitHub (spaghetti and meatballs), if you want to take a look.

The spaghetti/BUILD file can be pretty simple:

cc_library(
    name = "spaghetti",
    srcs = ["plate_of_spaghetti.cc"],
    visibility = ["//visibility:public"],
)

The meatballs/BUILD file is similar, but you also have an end-to-end test that depends on both //spaghetti and //meatballs:

cc_library(
    name = "meatballs",
    srcs = ["pile_of_meatballs.cc"],
)

cc_test(
    name = "end_to_end_test",
    srcs = ["end_to_end_test.cc"],
    deps = [
        ":meatballs",
        "//spaghetti",
    ],
)

Note that we’re depending on //spaghetti, even though it’s not under meatballs-master/. We can combine the two directories during the build by running bazel with the --package_path argument:

$ bazel test --package_path %workspace%:/home/k/gitroot/spaghetti-stable:/usr/local/lib/bazel/base_workspace 
    //meatballs:end_to_end_test

This means: when you’re looking for package, first check ~/gitroot/meatballs-master (%workspace% is the current directory). Then check ~/gitroot/spaghetti-stable. Finally, check Bazel’s binary installer location (for internal tools Bazel needs during the build).

When your test finishes, take a look at ~/gitroot/meatballs-master/bazel-meatballs-master. This is called the execution root and it’s where Bazel actually runs build commands:

$ ls -l bazel-meatballs-master/
total 36
drwxr-x--- 2 k k 20480 Dec  8 14:08 _bin
drwxr-x--- 3 k k  4096 Dec  8 14:08 bazel-out
drwxr-x--- 2 k k  4096 Dec  8 14:08 external
lrwxrwxrwx 1 k k    64 Dec  8 14:08 meatballs -> home/k/test/meatballs-master/meatballs
lrwxrwxrwx 1 k k    64 Dec  8 14:08 spaghetti -> /home/k/test/spaghetti-stable/spaghetti
lrwxrwxrwx 1 k k    41 Dec  8 14:08 tools -> /usr/local/lib/bazel/base_workspace/tools

You can see that Bazel has combined the directories on the package path to create a single directory that contains both meatballs/ and spaghetti/ subdirectories. The source directory (~/gitroot/meatballs-master) is left unchanged.

If we were going to do this regularly, we could add the package path option to our .bazelrc file and then we don’t have to specify it every build.

To try this out, you can download the sources with:

$ git clone https://github.com/kchodorow/spaghetti-stable.git
$ git clone https://github.com/kchodorow/meatballs-master.git

Then cd into meatballs-master/ and run!