Skip to content

Migrating to Bazel Modules (a.k.a. Bzlmod) - Toolchainization

Part of the promise of Bazel modules is that they are largely self-initializing in an order-independent way. Rule sets, in particular, no longer need to burden users with importing and invoking macros to instantiate repositories and toolchains in a specific order. This burden now shifts to rule set maintainers, but the existing implementation may not provide this ease of use without modification.

This post describes the introduction of a new "toolchainized" API for rules_scala v7.0.0 that better encapsulates toolchain configurations and dependencies. We'll see how this new design enables optimal Bzlmod compatibility, while simultaneously shrinking the legacy WORKSPACE API surface without losing functionality. We'll also see how the Bzlmod and legacy WORKSPACE APIs provide similar interfaces while sharing the same underlying implementation, facilitating Bzlmod migrations.

This article is part of the series "Migrating to Bazel Modules (a.k.a. Bzlmod)":

I occasionally update these blogs based on feedback, noting the changes in the Updates section at the bottom whenever I do. So don't forget to check the earlier blog posts every so often for new and improved information!

Prerequisites

As always, please acquaint yourself with the following concepts pertaining to external repositories if you've yet to do so:

You may also choose to read Bazel's Toolchains documentation first, or read through this post and come back to that document later.

What are toolchains?

Bazel's Toolchains mechanism decouples and hides sets of build tools and their dependencies from the rules and targets that rely upon them. Each concrete set of build tools and their dependencies is called a toolchain.

The toolchain_type concept provides the interface that links concrete build tool implementations to rules at build time. This is reminiscent of dependency injection, whereby the toolchain_type creates a seam that enables switching between different toolchains as necessary.

Toolchain selection is based on toolchain_type and other constraints.

Whereas the toolchain_type defines the interface, Bazel relies on constraints specified by rule and toolchain targets to select a specific implementation. Two of the most common constraints are operating system and CPU architecture, which are often bundled into platform configurations. As we'll see, rules_scala applies the Scala language version as a toolchain constraint. For more details, read the Bazel documentation on Platforms, Configurable Build Constraints, and Platforms and Toolchains Rules (after reading this blog post, of course).

Rule sets already must ensure their rules apply the correct toolchains for a particular target by declaring dependencies on toolchain_types. Properly designed rule sets also encapsulate toolchain dependencies to prevent users needing to reference specific toolchain targets or their dependencies directly.

To achive this design, I find it useful to think of toolchains as comprising a separate dependency graph from the rest of the build. Again, this is analogous to using dependency injection to hide concrete dependencies from client code. This post describes how to achieve this encapsulation of the toolchain dependency graph.

Of course, good design can provide both convenience and flexibility. A properly designed rule set will give the user option to exert control over toolchain definitions and dependencies as well.

User defined toolchains are a topic for a future post.

This post describes how to provide a convenient default toolchain interface that enables some degree of user configuration. rules_scala has provided macros for defining completely custom toolchains for a while, but no module extension currently supplies the same functionality. I've begun to experiment with defining such an extension (or extensions), which I may cover in a future post.

Comparing differences between the Bzlmod and legacy WORKSPACE models

An earlier post in this series contains a table contrasting the legacy WORKSPACE and Bzlmod repository instantiation models. Please keep that table handy for a more detailed, side-by-side comparison of legacy WORKSPACE vs. Bzlmod differences while reading this post. Many of those differences pertain to the toolchain setup and registration issues described below.

The inconvenience of importing toolchain dependencies

The Module Extensions post described how to create your own module extension for a dependency that hasn't yet migrated to Bzlmod. Though feasible, it's annoying as hell, since you have to import all the repositories your dependency needs into your own MODULE.bazel file. (Not to mention having to patch any Bzlmod related breakages.)

For example, this was the EngFlow/example configuration for rules_scala v6.6.0:

EngFlow/example MODULE.bazel configuration for rules_scala v6.6.0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
http_archive(
    name = "io_bazel_rules_scala",
    sha256 = "e734eef95cf26c0171566bdc24d83bd82bdaf8ca7873bec6ce9b0d524bdaf05d",
    strip_prefix = "rules_scala-6.6.0",
    url = "https://github.com/bazelbuild/rules_scala/releases/download/v6.6.0/rules_scala-v6.6.0.tar.gz",
    patches = ["//scala:rules_scala-6.6.0.patch"],
    patch_args = ["-p1"],
)

# Bumped to the latest Scala 2.13 version supported by rules_scala v6.6.0.
SCALA_VERSION = "2.13.16"
SCALA_VERSIONS = [SCALA_VERSION]

scala_config = use_extension("//scala/extensions:config.bzl", "scala_config")
scala_config.settings(
    scala_version = SCALA_VERSION,
    scala_versions = SCALA_VERSIONS,
)
use_repo(
    scala_config,
    "io_bazel_rules_scala_config",
)

repos = [
    "io_bazel_rules_scala_scala_compiler",
    "io_bazel_rules_scala_scala_library",
    "io_bazel_rules_scala_scala_parser_combinators",
    "io_bazel_rules_scala_scala_reflect",
    "io_bazel_rules_scala_scala_xml",
    "io_bazel_rules_scala_scalactic",
    "io_bazel_rules_scala_scalatest",
    "io_bazel_rules_scala_scalatest_core",
    "io_bazel_rules_scala_scalatest_compatible",
    "io_bazel_rules_scala_scalatest_featurespec",
    "io_bazel_rules_scala_scalatest_flatspec",
    "io_bazel_rules_scala_scalatest_freespec",
    "io_bazel_rules_scala_scalatest_funspec",
    "io_bazel_rules_scala_scalatest_funsuite",
    "io_bazel_rules_scala_scalatest_matchers_core",
    "io_bazel_rules_scala_scalatest_mustmatchers",
    "io_bazel_rules_scala_scalatest_shouldmatchers",
]

toolchains = [
    "@io_bazel_rules_scala//scala:toolchain",
    "@io_bazel_rules_scala//testing:scalatest_toolchain",
]

scala_deps = use_extension("//scala/extensions:deps.bzl", "scala_deps")
[
    (
        [use_repo(scala_deps, repo + suffix) for repo in repos],
        [register_toolchains(toolchain + suffix) for toolchain in toolchains],
    )
    # The v.replace() expression mimics the logic to generate version specific
    # repo suffixes from rules_scala.
    for suffix in ["_" + v.replace(".", "_") for v in SCALA_VERSIONS]
]

This configuration was a consequence of Bzlmod and legacy WORKSPACE characteristics for which the user supplied module extension had to compensate:

  • Legacy WORKSPACE files can call macros that instantiate repositories, including those for toolchain dependencies, into the single global scope shared by all repositories. These macros can also call native.register_toolchains. This is why legacy WORKSPACE files don't explicitly reference most repositories directly.

  • MODULE.bazel can't call macros. Module extensions can call macros, which in turn invoke repository rules, but each extension also has its own scope, separate from the invoking module's scope. Hence, modules often must define extensions to instantiate their toolchain dependency repositories, which are visible only within the extension's scope by default.

  • MODULE.bazel must call register_toolchains; module extensions can't. This means toolchain targets passed into register_toolchains calls must be visible within the module's scope.

  • Also worth noting: Under the legacy WORKSPACE model, only the main repository can invoke register_toolchains. Under Bzlmod, any module can invoke register_toolchains, not just the root module. We'll see how this comes into play shortly.

The consequences of these constraints are:

  • Toolchain targets defined in BUILD files of repositories imported by the module are visible within the module's scope. For example, @io_bazel_rules_scala//scala:toolchain is visible within the EngFlow/example module, because MODULE.bazel imports @io_bazel_rules_scala.

  • Toolchain dependencies instantiated by a module extension are not visible within the invoking module's scope by default. For example, the scala_deps extension from //scala/extensions:deps.bzl calls scala_repositories from @io_bazel_rules_scala//scala:scala.bzl. However, the instantiated repos are not visible to @io_bazel_rules_scala//scala:toolchain. They're only visible to other repos instantiated within the same module extension. (Hint: This fact plays a role in our later design...)

  • Therefore, the EngFlow/example module must bring each toolchain dependency repository from the scala_deps module extension into its scope via use_repo. Only then are they visible to @io_bazel_rules_scala//scala:toolchain.

Bazel tries to make this easier to manage by:

  • Enabling extensions to return module_ctx.extension_metadata. This metadata specifies which repos generated by the extension that the root module must import via use_repo. This metadata supports the "all" keyword to insist that the root module import every generated repository.

  • Providing the bazel mod tidy command. This command updates use_repo calls in MODULE.bazel automatically based on module_ctx.extension_metadata values.

If your project actually references all these repositories directly in its own BUILD and .bzl files, this is great. But if you want to use a rule set's builtin toolchains, it's burdensome to have to import all of its dependencies yourself. Even when using bazel mod tidy, the resulting MODULE.bazel file becomes more verbose and difficult to comprehend. This is complicated further if the names of the required repositories incorporate a version number (as with the rules_scala example above).

For example, try updating the EngFlow/example module extension at commit 79b5193 like so:

Patch adding module_ctx.extension_metadata
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
diff --git i/scala/extensions/deps.bzl w/scala/extensions/deps.bzl
index 40f03d1..8d554b9 100644
--- i/scala/extensions/deps.bzl
+++ w/scala/extensions/deps.bzl
@@ -3,9 +3,13 @@
 load("@io_bazel_rules_scala//scala:scala.bzl", "scala_repositories")
 load("@io_bazel_rules_scala//testing:scalatest.bzl", "scalatest_repositories")

-def _scala_dependencies_impl(_ctx):
+def _scala_dependencies_impl(module_ctx):
     scala_repositories(load_dep_rules=False)
     scalatest_repositories()
+    return module_ctx.extension_metadata(
+        root_module_direct_deps = "all",
+        root_module_direct_dev_deps = [],
+    )

 scala_deps = module_extension(
     implementation = _scala_dependencies_impl

then run bazel build //scala/... and bazel mod tidy to see what happens.

This is what happens...

Basically, it will result in MODULE.bazel importing many more repositories than the build actually requires. The list comprehension containing use_repo runs over the repos list to import the Scala version specific repositories. Returning the extension_metadata object with root_module_direct_deps = "all" forces the module to import all the non version specific repos as well. All of these non version specific repos contain aliases to the version specific repos, so they aren't required to build at all. The repos list was the result of trial and error, but it produced a far less noisy MODULE.bazel file.

Multiply this by however many projects that depend on rules_scala, and by similarly unmigrated repositories, and that's a lot of potential friction and frustration.

The magic of "Toolchainization"

The Toolchainization technique described here removes all of the excess Bzlmod configuration complexity seen above, with only the minimum necessary configuration remaining. It involves having a module that defines toolchains:

  • provide a module extension to generate a repository containing its toolchain targets

  • instantiate all of the toolchain's dependency repositories from the same module extension

  • bring the toolchain repo into its own scope via use_repo()

  • pass all the toolchain repo's targets to register_toolchains()

This way, all toolchains and their dependencies are encapsulated within the same scope, and the module registers the toolchains for you.

Due Credit

I developed this insight after studying the module extensions from rules_python and rules_go that generate toolchain repositories. Shortly after that, I noticed that Son Luong Ngoc suggested this exact approach on 2023-07-11.

For example, this is what the EngFlow/example configuration looks like now that rules_scala v7.0.0 is in the Bazel Central Registry:

EngFlow/example MODULE.bazel configuration for rules_scala v7.0.0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
bazel_dep(name = "rules_scala", version = "7.0.0")

scala_config = use_extension(
    "@rules_scala//scala/extensions:config.bzl",
    "scala_config",
)
scala_config.settings(scala_version = "2.13.16")

scala_deps = use_extension(
    "@rules_scala//scala/extensions:deps.bzl",
    "scala_deps",
)
scala_deps.scala()
scala_deps.scalatest()

This is a stripped down version of how the MODULE.bazel file from rules_scala accomplishes this:

rules_scala registering its own toolchain repo
1
2
3
4
5
scala_deps = use_extension("//scala/extensions:deps.bzl", "scala_deps")

use_repo(scala_deps, "rules_scala_toolchains")

register_toolchains("@rules_scala_toolchains//...:all")

The scala_deps module extension from //scala/extensions:deps.bzl performs the magic here. The rest of this post will break down what's happening.

Toolchain repositories

The key to designing a well encapsulated toolchain is to generate a separate toolchain repo using a repository_rule. In the case of rules_scala, the scala_deps module extension from @rules_scala//scala/extensions:deps.bzl generates @rules_scala_toolchains.

As mentioned above, rules_scala's builtin toolchains used to be defined in BUILD files in the rules_scala repo itself. Under the legacy WORKSPACE model, this worked because these targets and their dependencies in other repositories shared the same global scope. This was the easiest, most straightforward way to implement these toolchains at the time.

These same targets now reside in a repository generated by a repository_rule invoked by a module extension. This unlocked the ability to generate their dependency repos in the same module extension, and therefore the same scope.

The trick to accomplishing this for rules_scala's toolchains was to define three new components:

  • The scala_deps module extension, which defines a tag class to enable and/or configure each builtin toolchain.

  • The scala_toolchains macro, which scala_deps invokes to instantiate toolchain dependency repos based on values compiled from its tag classes.

  • The scala_toolchains_repo repository rule, which scala_toolchains invokes to instantiate the toolchain repo after instantiating all the required dependency repos.

Together, these mechanisms produce the @rules_scala_toolchains repo. The Appendix: Implementation details section at the very end contains a brief description of each component. But for now, let's understand how rules_scala automatically registers the toolchains from @rules_scala_toolchains, as this is crucial to the design.

register_toolchains() and dynamic toolchain generation

One feature of the original legacy WORKSPACE configuration schema was that many of the builtin rules_scala toolchains were optional. If you needed a toolchain, you'd import one or more macros from a specific file into your legacy WORKSPACE file. These macros would instantiate dependency repos, then call native.register_toolchains to register that specific toolchain. For example:

Scalafmt and Scalatest legacy WORKSPACE configuration
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
load(
    "//testing:scalatest.bzl",
    "scalatest_repositories",
    "scalatest_toolchain",
)

scalatest_repositories()

scalatest_toolchain()

load(
    "//scala/scalafmt:scalafmt_repositories.bzl",
    "scalafmt_default_config",
    "scalafmt_repositories",
)

scalafmt_default_config()

scalafmt_repositories()

Under the new schema, we use a single module extension to generate all of the builtin toolchains in @rules_scala_toolchains. So the above configuration looks like this under Bzlmod:

Scalafmt and Scalatest Bzlmod configuration
1
2
3
4
5
6
scala_deps = use_extension(
    "@rules_scala//scala/extensions:deps.bzl",
    "scala_deps",
)
scala_deps.scalafmt()
scala_deps.scalatest()

We could've defined separate extensions to provide individual toolchains, generating separate toolchain repos. But that proved unnecessary, and would've made the register_toolchains call in rules_scala more complex.

Instead, scala_toolchains_repo generates a variable number of packages based on the configuration. scala_deps compiles the toolchain options, passes them to scala_toolchains, and scala_toolchains_repo generates packages for each configured toolchain.

For example, here are the contents of the @rules_scala_toolchains repo generated from building rules_scala itself:

@rules_scala_toolchains from rules_scala
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
$ ls -1 "$(bazel info output_base)/external/+scala_deps+rules_scala_toolchains/"

BUILD
jmh/
REPO.bazel
scala/
scala_proto/
scalafmt/
testing/
twitter_scrooge/

$ bazel query 'kind("^toolchain rule$", @rules_scala_toolchains//...:all)'

@rules_scala_toolchains//jmh:jmh_toolchain
@rules_scala_toolchains//scala:toolchain_2_11_12
@rules_scala_toolchains//scala:toolchain_2_12_20
@rules_scala_toolchains//scala:toolchain_2_13_16
@rules_scala_toolchains//scala:toolchain_3_1_3
@rules_scala_toolchains//scala:toolchain_3_3_6
@rules_scala_toolchains//scala:toolchain_3_5_2
@rules_scala_toolchains//scala:toolchain_3_6_4
@rules_scala_toolchains//scala:toolchain_3_7_0
@rules_scala_toolchains//scala_proto:scala_proto_default_deps_toolchain
@rules_scala_toolchains//scala_proto:scala_proto_default_toolchain
@rules_scala_toolchains//scalafmt:scalafmt_toolchain_2_11_12
@rules_scala_toolchains//scalafmt:scalafmt_toolchain_2_12_20
@rules_scala_toolchains//scalafmt:scalafmt_toolchain_2_13_16
@rules_scala_toolchains//scalafmt:scalafmt_toolchain_3_1_3
@rules_scala_toolchains//scalafmt:scalafmt_toolchain_3_3_6
@rules_scala_toolchains//scalafmt:scalafmt_toolchain_3_5_2
@rules_scala_toolchains//scalafmt:scalafmt_toolchain_3_6_4
@rules_scala_toolchains//scalafmt:scalafmt_toolchain_3_7_0
@rules_scala_toolchains//testing:testing_toolchain_2_11_12
@rules_scala_toolchains//testing:testing_toolchain_2_12_20
@rules_scala_toolchains//testing:testing_toolchain_2_13_16
@rules_scala_toolchains//testing:testing_toolchain_3_1_3
@rules_scala_toolchains//testing:testing_toolchain_3_3_6
@rules_scala_toolchains//testing:testing_toolchain_3_5_2
@rules_scala_toolchains//testing:testing_toolchain_3_6_4
@rules_scala_toolchains//testing:testing_toolchain_3_7_0
@rules_scala_toolchains//twitter_scrooge:scrooge_toolchain

Here are the contents of @rules_scala_toolchains generated from building EngFlow/example (notice the longer canonical repo name):

@rules_scala_toolchains contents from EngFlow/example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
$ ls -1 "$(bazel info output_base)/external/rules_scala++scala_deps+rules_scala_toolchains/"

BUILD
REPO.bazel
scala/
testing/

$ bazel query \
  'kind("^toolchain rule$", @@rules_scala++scala_deps+rules_scala_toolchains//...:all)'

@@rules_scala++scala_deps+rules_scala_toolchains//scala:toolchain_2_12_20
@@rules_scala++scala_deps+rules_scala_toolchains//testing:testing_toolchain_2_12_20

This works because:

  • register_toolchains() will register all toolchain targets in the specified target set, ignoring all other targets. The call still succeeds even if the set does not contain any toolchain targets, or is completely empty.

  • The @rules_scala_toolchains//...:all specifier recursively discovers all packages present in the repository, and all targets within each package. This avoids the need to specify specific packages and targets, which can change based on the current toolchain configuration.

  • rules_scala always generates @rules_scala_toolchains with an empty top level BUILD file, even if it contains no subpackages. This ensures that the register_toolchains call always succeeds, since @rules_scala_toolchains//...:all will at least discover the empty root package.

Toolchain registration order

One of the improvements of Bzlmod over the legacy WORKSPACE model is that module registration is order independent. However, toolchain registration remains somewhat order dependent. Bazel selects the first registered toolchain matching the required toolchain_type and other constraints defined by the toolchain and rule targets.

Since modules can register toolchains themselves, the registration order becomes more hierarchical, based on the structure of the module graph. Since the module graph roughly implies a priority ordering of register_toolchain calls, toolchain registration generally works the right way.

Also, from Toolchain resolution:

Pseudo-targets like :all, :*, and /... are ordered by Bazel's package loading mechanism, which uses a lexicographic ordering.

This currently doesn't matter much for @rules_scala_toolchains, since each package registers a different toolchain_type. The packages that register multiple toolchains of the same type define a specific Scala version as a constraint on each toolchain as well. (The introduction of user defined toolchains into the rules_scala module extension API may have to take this behavior into account, however.)

Root module toolchain registration and the dev_dependency attribute

The root module retains these privileges when it comes to toolchain generation and registration:

These features provide Bazel modules the flexibility to specify which of its module extension instances and toolchain registrations apply to a given build.

These elements always apply to all builds that include the rules_scala module:

  • rules_scala invokes one instance of the scala_deps module extension without dev_dependency = True. Though this instance doesn't instantiate any tag classes, it ensures that rules_scala always generates the @rules_scala_toolchains repository, even if it's empty.

  • rules_scala registers the toolchain repo without the dev_dependency = True qualifier. This automatically registers all the toolchains in every package generated in that repository, as determined by the overall build configuration. This is a no-op if @rules_scala_toolchains contains only the empty root package.

These elements only apply when building rules_scala as the root module:

  • rules_scala invokes use_extension(..., dev_dependency = True) on a different instance of the scala_deps module extension. This instance instantiates all the tag classes, generating all the toolchains in @rules_scala_toolchains required by its tests.

  • rules_scala also invokes register_toolchains(..., dev_dependency = True) on toolchain targets required specifically for its own testing. This register_toolchains call appears before the other register_toolchains call to ensure these dev_dependency toolchains take precedence when rules_scala is the root module.

This explains why the rules_scala instance of @rules_scala_toolchains contains many more toolchains than the EngFlow/example instance:

  • When rules_scala is the root module, it generates and registers all the toolchains described above.

  • When EngFlow/example is the root module, Bazel does not generate and register dev_dependency toolchains from the rules_scala module. As a result, rules_scala generates and registers only those toolchains specified by the scala_deps extension instance in the EngFlow/example module.

Compatibility between Bzlmod and legacy WORKSPACE builds

One of the beautiful things about this toolchain repository schema is that it's also compatible with legacy WORKSPACE builds! In fact, I added scala_toolchains and scala_toolchains_repo to rules_scala before adding the scala_deps module extension. This guaranteed legacy WORKSPACE compatibility at every step, to help users migrate to Bzlmod whenever they're ready, rather than forcing an immediate migration.

Legacy WORKSPACE files call scala_toolchains directly; the scala_deps module extension is essentially a thin layer on top of scala_toolchains. scala_deps merely compiles tag class configuration values and passes them through as arguments to scala_toolchains. Not only was this good for code reuse, but it yielded a few other important benefits:

  • The legacy WORKSPACE API surface shrank dramatically, replacing eighteen previous macros with scala_toolchains and scala_register_toolchains. This also replaced up to eight or so load statements with a single load to import these macros from @rules_scala//scala:toolchains.bzl.

  • Sharing scala_toolchains ensures that there are no differences between Bzlmod and legacy WORKSPACE builds using equivalent configuration values. One can build with either the Bzlmod API or the legacy WORKSPACE API and get the same rules_scala behavior and the same results.

  • The scala_deps and scala_toolchains APIs are very similar, such that migrating legacy WORKSPACE configurations to MODULE.bazel should prove very easy.

So rules_scala v7.0.0 makes significant, breaking changes to the legacy WORKSPACE API, but still provides first class support to legacy WORKSPACE builds. Users that aren't yet ready to migrate to Bzlmod gain the following benefits from adapting to the updated legacy WORKSPACE API:

Precompiled protocol buffer compiler (protoc) toolchainization

Toolchainization isn't just for Bzlmod compatibility fixes! We encountered a showstopping breakage of rules_scala Windows MSVC builds when updating past protobuf v21.7. Toolchainizing prebuilt protoc binaries by introducing the @rules_scala_protoc_toolchains repo fixed this Windows breakage, and improved performance across all platforms.

The error happened when compiling the protocol buffer compiler, protoc, due to a transitive dependency on @protobuf//:protoc. The underlying causes were that:

Compounding the problem is the fact that protobuf is in the process of dropping MSVC support. That led me to investigate using a precompiled protoc binary toolchain, such as toolchains_protoc. Bazel and protobuf have supported this via the --incompatible_enable_proto_toolchain_resolution flag for some time (mostly, as we'll see). Though rather than use toolchains_protoc, I decided to implement the @rules_scala_protoc_toolchains repo in the //protoc package.

However, the build was still broken because rules_scala still had targets transitively depending on @protobuf//:protoc (seen as @com_google_protobuf//:protoc below):

Finding a transitive dependency on @protobuf//:protoc
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Explicitly disabling `--incompatible_enable_proto_toolchain_resolution`,
# since it's now set in `.bazelrc`. Otherwise produces empty results.
$ bazel query --noincompatible_enable_proto_toolchain_resolution \
    'somepath(//scala/..., @com_google_protobuf//:protoc)'

//scala/support:test_reporter
//src/java/io/bazel/rulesscala/scalac:scalac
//src/protobuf/io/bazel/rules_scala:diagnostics_java_proto
//src/protobuf/io/bazel/rules_scala:diagnostics_proto
@bazel_tools//tools/proto:protoc
@com_google_protobuf//:protoc

Fortunately, I found that protocolbuffers/protobuf#19679 already existed to solve exactly this problem. It breaks the @protobuf//:protoc dependency when --incompatible_enable_proto_toolchain_resolution is in effect. (I also used details from this PR to create protoc/private/toolchain_impl.bzl and update scala_proto_toolchain.) Unfortunately, the protobuf maintainers refuse to accept this change. They claim that --incompatible_enable_proto_toolchain_resolution is deprecated, and the better way isn't ready yet.

Immensely frustrating as this is, Bazel gives us a way to route around the damage. I created a patch from protocolbuffers/protobuf#19679 and added it to rules_scala as //protoc:0001-protobuf-19679-rm-protoc-dep.patch. Once I wired up the patch and @rules_scala_protoc_toolchains as described in the rules_scala README, we were back in business! And as a bonus, the entire test suite ran much faster, shaving about 4 or 5 minutes off of CI builds.

Conclusion

After all of this talk of "toolchainization," I have to confess something. I could be inventing a word here, or ignorantly abusing an existing term (though if I am, no one's told me yet). Perhaps some helpful readers will gently correct my terminology if I'm indeed misusing it.

What I do know, however, is that the "toolchainized" design of rules_scala v7.0.0 does The Right Thing. It enables Bzlmod compatibility, reduces the legacy WORKSPACE API without losing functionality, and makes it easy to migrate to Bzlmod in the near future. It also unlocked a path around a serious protoc build breakage, with significant performance benefits to boot.

I also know it would've been impossible to make such consequential changes without tests! I'm very grateful that rules_scala already had an extensive test suite that gave me the utmost confidence that my changes were correct. I found it easy to add new test suites for the new Bzlmod helper functions, for MODULE.bazel linting, and for validating version compatibility.

I'm equally as grateful for the wonderful support of the rules_scala maintainers, Simonas Pinevičius and Vaidas Pilkauskas, and the rules_scala community at large. Gergely Fábián, in particular, gave me several bits of helpful feedback after trying my changes along the way (including documentation updates!). Thanks to Fabian Meumertzheim for protocolbuffers/protobuf#19679, in addition to all the help with repo names. Thanks to Yun Peng and Alex Eagle for helping get the rules_scala module published to the BCR. Thanks to Jay Conrod for nerd sniping me into starting the rules_scala Bzlmodification project. And, of course, thanks to EngFlow for allowing me to contribute substantially to an important Open Source project.

So what's next? As far as rules_scala is concerned, I'll continue experimenting with adding user defined toolchains to its toolchain registration interface. I'll likely continue to submit occasional dependency updates as well.

As far as this blog series is concerned, I haven't forgotten about the topic I mentioned last time:

However, I think I'll first write about testing to ensure compatibility with minimum dependency versions, while still testing against the latest dependency versions. A conversation Simon Stewart initiated in the #bzlmod channel of the Bazel Slack workspace on 2025-04-02 inspired me to do this for rules_scala.

As always, I'm open to questions, suggestions, corrections, and updates relating to this series of Bzlmodification posts. It's easiest to find me lurking in the #bzlmod channel of the Bazel Slack workspace. I'd love to hear how your own Bzlmod migration is going—especially if these blog posts have helped!

Appendix: Implementation details

Here's a very brief overview of the new components supporting the toolchainization of rules_scala v7.0.0. The description of each component contains a link to its implementation; open these links to have the implementations handy while reading.

scala_toolchains_repo

scala_toolchains_repo is a repository rule that takes a number of attributes pertaining to the different toolchains built into rules_scala. It compiles these parameters and prepares them for injection into BUILD files that define the packages for each enabled toolchain.

It also injects the canonical repo name of @rules_scala itself into these BUILD files, so they can load toolchain macros from @rules_scala. These are the same toolchain macros currently available to users for defining their own custom toolchains.

scala_toolchains

scala_toolchains wraps the call to scala_toolchains_repo, invoking it after instantiating all of the repositories required by every configured toolchain. It also exposes a more flexible, user friendly interface than the attributes exposed by scala_toolchains_repo.

Notice that each of the toolchains has a corresponding *_artifact_ids() macro. This is part of the dependency schema defined by the scala_*.bzl files in third_party/repositories. These files define repositories for Maven artifacts that are compatible with each supported Scala version, where the repo name is the "artifact id". rules_scala doesn't use rules_jvm_external (yet), so the *_artifact_ids() macros specify the dependencies required by each builtin toolchain.

The internal design of scala_toolchains solves a Bzlmod compatibility problem. Instantiating a repo multiple times works under legacy WORKSPACE builds, but breaks under Bzlmod. Since some toolchains share some of the same dependencies, scala_toolchains dedupes the results from these macros to instantiate each repo only once. It does so by building dictionaries that use the repository labels returned by the *_artifact_ids() macros as keys.

These dictionaries map toolchain dependency repo names to a boolean indicating whether or not to download the repo's sources. The only purpose of the fetch_sources boolean is to preserve the default behavior for each toolchain from v6.6.0. The more important detail is that the dictionaries build lists of toolchain dependencies for each configured Scala version to pass to repositories.

scala_deps

The scala_deps module extension, which ultimately calls scala_toolchains, defines a tag class to enable and/or configure each builtin toolchain. It's a thin layer that translates tag class values to scala_toolchains parameters. However, it provides type checked arguments and well defined evaluation of configurations across modules. This yields convenience and consistency that the legacy WORKSPACE model can't provide.

The _toolchains_settings function enables and configures the builtin toolchains specified throughout all modules in the module graph. It implements the policy that any module can enable a builtin toolchain, but only the root module can customize its configuration. Otherwise a builtin toolchain receives its default configuration values.

Also, only the overridden_artifacts, compiler_srcjars, and settings tags from the root module take effect. Otherwise the first two sets of tag class values remain empty, and the defaults for settings apply.

This is similar to how configuration macro calls in legacy WORKSPACE builds work, but is explicit and easy to reason about. Unlike legacy WORKSPACE builds, the extension automatically instantiates toolchains required by the build that aren't directly referenced by the main repository.

To reduce duplication and conditional logic, I defined a set of common implementation functions in scala/private/macros/bzlmod.bzl. As explained at the top of that file, they're based on the pattern of defining default values for tag class attrs in a separate dictionary. This enables module extension implementations to implement logic based on these default values, which are not directly accessible from attr objects. This, in turn, removes the need for logic to check whether values are present or that they match hardcoded, likely duplicated values. The test/shell/test_bzlmod_macros.sh test script tests these functions in isolation from the scala_deps module extension. (I'm thinking about moving _toolchain_settings into that file as well.)

Thanks to these common functions, it's clear that scala_deps really is just a thin wrapper on top of scala_toolchains:

scala_deps module extension wrapping scala_toolchains
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
def _scala_deps_impl(module_ctx):
    tags = root_module_tags(module_ctx, _tag_classes.keys())
    tc_names = [tc for tc in _toolchain_tag_classes]

    scala_toolchains(
        overridden_artifacts = repeated_tag_values(
            tags.overridden_artifact,
            _overridden_artifact_attrs,
        ),
        scala_compiler_srcjars = repeated_tag_values(
            tags.compiler_srcjar,
            _compiler_srcjar_attrs,
        ),
        **(
            single_tag_values(module_ctx, tags.settings, _settings_defaults) |
            _toolchain_settings(module_ctx, tags, tc_names, TOOLCHAIN_DEFAULTS)
        )
    )