Migrating to Bazel Modules (a.k.a. Bzlmod) - The Easy Parts¶
You may be aware that Bazel will remove support for WORKSPACE
in Bazel 9
in favor of Bazel Modules (a.k.a. Bzlmod). The current mainstream release
is Bazel 7.2.0, so there's plenty of time to migrate. However, there's no
time like the present to get started, to avoid further WORKSPACE
dependencies
and a pile of migration work in the future.
I recently completed the Bzlmod migration for EngFlow/example and our internal repos. This experience taught me a lot about Bzlmod and about migrating complex projects with challenging dependency issues that I'll share over a few blog posts. I'll also borrow from Sara Adams's earlier post, in which she described an example bzlmod migration based on EngFlow's Bazel Invocation Analyzer repo.
This article is part of the series "Migrating to Bazel Modules (a.k.a. Bzlmod)":
- Migrating to Bazel Modules (a.k.a. Bzlmod) - The Easy Parts
- Migrating to Bazel Modules (a.k.a. Bzlmod) - Repo Names and Runfiles
- Migrating to Bazel Modules (a.k.a. Bzlmod) - Repo Names and rules_pkg
Background¶
If you're only starting to learn about Bzlmod, please review the latest version of Bazel's External dependencies documentation first, especially the Bzlmod Migration Guide. This post provides ample links throughout for additional context and details, but it will read more smoothly if you read the official Bazel docs first.
The Bazel documents cover Bzlmod concepts in more detail than this blog post series will, as well as advanced use cases we fortunately didn't require. So if this blog series doesn't cover all your needs, you may find relevant information there.
That said, I'll share useful insights and techniques that I didn't find apparent in the Bazel documentation or that of its rule sets.
The Good¶
Migrating your project is probably easier than you expect. Most dependencies and language rule sets already support Bzlmod. In this section, we'll see how to deal with the common cases. Later, we'll see how to address some more difficult issues you might see along the way.
Standalone applications can migrate gradually without WORKSPACE.bzlmod
¶
The good news for standalone application projects is that you don't have to
migrate everything and swap out WORKSPACE
for MODULE.bazel
all at once.
When Bzlmod is enabled (via --enable_bzlmod
or by default in Bazel 7), it will
parse both MODULE.bazel
and WORKSPACE
for external dependencies. There's
also no need for an intermediate WORKSPACE.bzlmod
file during the migration
process.
This means you can gradually move individual dependencies directly from
WORKSPACE
to MODULE.bazel
, keeping the build intact at every step without
duplicating dependency information. This makes the migration process far more
palatable and manageable for large, complex codebases.
You can begin making headway on the Bzlmod migration at a relaxed pace, saving more challenging dependencies for later. You can revisit them when you have more bandwidth, more experience, the problems have solved themselves upstream, or when you decide to drop the dependency.
Difference from the Bzlmod Migration Guide's recommended process
The advice in the Bzlmod Migration Guide suggests creating a
WORKSPACE.bzlmod file to switch between WORKSPACE
and Bzlmod modes
during the migration. It also recommends a migration process to build up
MODULE.bazel and WORKSPACE.bzlmod independently from WORKSPACE
.
This advice appears geared towards projects with relatively straightforward
dependencies, or projects (such as Bazel rule sets) that themselves
are dependencies of other Bazel projects. It also leads to duplication of
dependency information between WORKSPACE
and MODULE.bazel
+
WORKSPACE.bzlmod
.
Such duplication may be necessary for some projects that are themselves Bazel dependencies of other projects, in order to support non-Bzlmod builds. However, it is not necessary for projects which have no requirement to support non-Bzlmod builds.
Lifting and shifting most dependencies¶
You can "lift and shift" the vast majority of dependencies straight from
WORKSPACE
to MODULE.bazel
rather easily. This section describes some of the
most common mechanisms for doing that.
With bazel_dep()
¶
In many cases, you can use the Bazel Central Registry (BCR) to find the equivalent bazel_dep() to replace existing http_archive() repository rules, if available.
You'll still need to refer to a module repository's README.md
, release notes,
or other documentation if additional configuration is necessary. However, this
configuration is often greatly simplified compared to the equivalent WORKSPACE
configuration. (See the rules_jvm_external example
below.)
The BCR contains all the information necessary to locate repository archives and verify their integrity. The "Fetch external dependencies as Bazel modules" section of the Bzlmod Migration Guide illustrates how to do exactly this.
For a concrete example, see Sara's pull request
EngFlow/bazel_invocation_analyzer: Start using Bzlmod #144. She was able
to replace thirty-eight lines of WORKSPACE
configuration with the
following three lines in MODULE.bazel
:
Text Only | |
---|---|
Example
See Sara's Move buildifier setup from WORKSPACE to Bzlmod #153 for an
even more dramatic simplification afforded by bazel_dep()
. This change
involved replacing several http_file()
downloads of Buildifier binaries
and a complicated copy_file()
rule with a bazel_dep()
and a brief
buildifier_binary()
.
Defining your own custom Bazel module registry
If you'd prefer not to depend on the BCR or other external systems beyond
your control, you can define your own custom Bazel registry. This
allows you to depend on your own custom archive mirrors for your
bazel_dep()
targets. (See Jay Conrod's post "How GitHub's upgrade
broke Bazel builds" for reasons why you might want to do this.) Or
you can vendor your dependencies in your codebase—but see the
local_repository()
and local_path_override()
caveats
below.
With http_archive()
or any other repository rule¶
Even if no Bazel module is available in the registry, http_archive() and its
kin are still available as repository rules in Bzlmod. In fact, you can
still use any repository rule from WORKSPACE
directly in
MODULE.bazel
by replacing load() with use_repo_rule(). Then if the
repository doesn't require any further setup, you're done!
The "Fetch external dependencies with module extensions" section
of the Bzlmod Migration Guide has a straightforward example of using
use_repo_rule()
to import http_file()
:
Text Only | |
---|---|
With rules_jvm_external
¶
Java projects using Bazel often use rules_jvm_external to manage
Maven-style dependencies on external JARs. Under WORKSPACE
, this involves a
series of load()
statements and dependency setup calls.
Alternatively, Java projects may rely on a series of http_jar() calls in
the WORKSPACE
file. This was the case before Sara's
EngFlow/bazel_invocation_analyzer: Use Bzlmod instead of http_jar in WORKSPACE
file #149. She replaced fifteen http_jar()
calls by using the maven
extension from rules_jvm_external
.
She then followed this with Pin Maven artifacts with maven_install.json #160 and Remove transitive Maven deps #161. After the updates from Update to rules_jvm_external 6.2, repin maven deps #183, this looks like:
After the migration, it's also now easier to update versions, and there's no need to explicitly download transitive dependencies. Bazel can also avoid unnecessary downloads while still applying integrity checks.
Tip
See the rules_jvm_external Bzlmod documentation for further details on Bzlmod-specific configuration. See the "Pinning artifacts and integration with Bazel's downloader" section of the rules_jvm_external README for the benefits of artifact pinning in particular.
With other language-specific rule sets¶
Rule sets exist for several other languages that integrate external dependency managers into the Bazel external repository scheme, including (but not limited to):
Module version resolution¶
WORKSPACE
only allows one version of a repository to exist in the build.
You're allowed to declare multiple versions, but Bazel has very subtle rules on
how it picks versions. This can be dangerous if you end up linking in multiple
incompatible versions of a library with significant runtime presence, like gRPC.
Tip
See Jay Conrod's post on "Organizing Bazel WORKSPACE Files"
for a deep dive into WORKSPACE
evaluation and repository selection.
Bzlmod, in contrast, resolves multiple versions of the same module throughout the dependency graph. Its algorithm replaces all compatible minor versions with a single version, while allowing multiple incompatible major versions to exist in the build.
Module version resolution override mechanisms exist to provide control over specific dependency versions. For example, multiple_version_override() enables multiple versions to exist in the build that the module resolution algorithm would otherwise elide.
Updated repository path encoding¶
There are two key differences in how Bzlmod encodes repository paths that generally improve how Bazel works. However, it may affect your project if it depends on such paths directly (as discussed in later sections).
Stable _main
module name¶
Under bzlmod, the runfiles root for the main repo is always
_main/
. From the ctx.workspace_name() definition:
The name of the workspace, which is effectively the execution root name and runfiles prefix for the main repo. If
--enable_bzlmod
is on, this is the fixed string_main
. Otherwise, this is the workspace name as defined in the WORKSPACE file.
This means that any runfile paths to other files in your codebase will always
begin with _main
. More on this below.
Canonical repository names¶
To support module resolution, Bazel has the concept of canonical repository
names (starting with @@
) as distinct from apparent repository
names (starting with @
):
-
@
repository names are used within each repository to refer to other repositories it directly depends on. They may not be globally unique. -
@@
names are used globally by Bazel and the command line user to refer to a specific instance of a downloaded repository. They are "mangled" to encode the module name and the module extension that imported it, as well as version information if necessary.
Info
By default, canonical repo names do not encode the module version since Bazel 7.1.0. They include version numbers only if the build requires multiple versions of the same module.
From the Module extensions: Repository names and visibility section of the Bazel documentation:
Repos generated by extensions have canonical names in the form of
module_repo_canonical_name~extension_name~repo_name
. For extensions hosted in the root module, themodule_repo_canonical_name
part is replaced with the string_main
.
This means that the repository directory names in $(bazel info output_base)/external encode useful information showing how a repository is related to the build. This allows multiple repository versions to exist if necessary, avoiding one silently overwriting the other. It can also be useful when trying to investigate problems with the build.
Examples of canonical repository names
bazel_skylib~
: The@bazel_skylib
repository defined by abazel_dep()
rule in one or moreMODULE.bazel
files in the dependency graph. Since module resolution resulted in a single version, there is no version information in the repo name._main~_repo_rules~com_github_grpc_grpc
: This is the@com_github_grpc_grpc
repository defined by anhttp_archive()
rule (i.e., a_repo_rule
) in the main repository'sMODULE.bazel
(specified by_main
). There is no version information encoded, sincehttp_archive()
doesn't contain aversion
attribute.rules_jvm_external~~maven~junit_junit_4_13_2
: The JAR repository backing the@maven//:junit_junit
target defined by themaven
module extension from therules_jvm_external
module.rules_jvm_external~5.3~maven~junit_junit_4_13_2
: Same as the previous name, except multiple modules in the dependency graph depend on different incompatible versions ofrules_jvm_external
. (That, or the project was built using a version of Bazel prior to 7.1.0.)
Info
See also Benjamin's post The Many Caches of Bazel for information on the repository cache.
The Bad¶
Of course, not everything is going to work out of the box. Here are some more
key differences of the Bzlmod model from WORKSPACE
, along with a few related
breakages.
In later posts, outlined in The Hard Parts, I'll dive into more details of why these differences exist and how to remedy complicated migration challenges.
Enabling Bzlmod may require many fixes at once¶
The act of setting --enable_bzlmod=true
may break many things before moving a
single repo from WORKSPACE
to MODULE.bazel
. In particular:
- New naming conventions break file path dependencies
- Some repositories behave differently with Bzlmod enabled
- Multiple repo versions may manifest, causing breakage
Each of these issues can and should be fixed one at a time, in separate commits,
while migrating as few repositories to MODULE.bazel
as possible. Still, the
first overall change to enable Bzlmod may require fixing many of these problems
at once to keep the overall build intact.
Not all repositories are (totally) Bzlmod-ready¶
While seemingly most major Bazel rules packages have migrated to Bzlmod, there are notable exceptions, such as rules_scala. Or perhaps you depend on a deprecated rule set like rules_nodejs. Upgrading to a newer, Bzlmod-ready rule set is absolutely recommended, but may require substantially more effort, given fundamental differences between the two rule sets.
In these cases, you can still move forward with migrating to Bzlmod, but it
requires a bit of insight and custom work. Many of the issues are related to
having to load()
configuration functions that implicitly define toolchains and
other repositories as described below.
Some repositories behave differently with Bzlmod enabled¶
Some rule sets contain logic to behave differently under Bzlmod, and/or expose
an API lacking features from the earlier WORKSPACE
API. As such, you may not
have a choice but to migrate such repositories to MODULE.bazel
immediately.
Usually this should prove easy, as such rule sets should have a well documented migration path to all cover common cases. However, if you're applying patches or any other sort of custom configuration, and the documentation and examples are lacking, resolving issues could require careful study.
rules_python¶
For example, enabling Bzlmod causes rules_python to explicitly disable
toolchain registration inside
python_register_toolchains()—preventing its WORKSPACE
API from
working at all. This is because module extensions can't call
native.register_toolchains()
,
but this disables Python toolchain registration even when calling
python_register_toolchains()
directly from WORKSPACE
.
Info
Tracing through the code, you can see that the rules_python Bzlmod
extension also calls python_register_toolchains(). This function takes
a register_toolchains
parameter, so the module extension could pass
register_toolchains=False
instead of having python_register_toolchains()
detect Bzlmod enablement. It's possible to change this upstream, and
there is an open issue for it: Enabling bzlmod causes workspace toolchains
to no longer be registered #1675. But it's probably better to migrate
your rules_python
usage to Bzlmod regardless, rather than contributing or
waiting for an upstream fix.
In other words, when you enable Bzlmod, you can no longer use rules_python
in
WORKSPACE
. You must move it to MODULE.bazel
. This, in turn, forces you to
contend with API changes such as:
- Under
WORKSPACE
: Thepython_register_toolchains
macro accepts atool_versions
parameter to provide fine-grained control over archive sources for python versions. Itspython_version
parameter can be specified down to the patch level, e.g., 3.12.4. - Under Bzlmod: The
python
extension exposes thepython.toolchain
tag, which currently does not provide a tool_versions parameter. In versions earlier than 0.32.0, which resolves rules_python #1371, itspython_version
parameter only accepts values specified down to the minor version, e.g., 3.12. The extension uses thispython_version
to select an archive from its own internal table.
Similarly, when it comes to applying package modifications:
- Under
WORKSPACE
:pip_parse
accepts anannotations
parameter to apply updates to imported packages, as a dictionary mapping package names topackage_annotation
objects. - Under Bzlmod:
pip.parse
accepts awhl_modifications
parameter with a different interface: a dictionary of repo target labels, defined using thepip.whl_mods
tag, to package names.
Tip
The best example of pip.whl_mods
usage that I've found is in
examples/bzlmod/MODULE.bazel from bazelbuild/rules_python.
New naming conventions break file path dependencies¶
While a stable _main
module name and a new canonical repository name schema
provide long term benefits, both may introduce immediate pain. The Bazel
documentation has this to say about the canonical repository name format in
Bazel modules: Repository names and strict deps (emphasis theirs):
Note that the canonical name format is not an API you should depend on and is subject to change at any time. Instead of hard-coding the canonical name, use a supported way to get it directly from Bazel...
Danger
The change to no longer encode module versions in canonical repo names
in Bazel 7.1.0 is a recent example of Bazel maintainers altering the
format. They're also about to change the canonical repo name format again
to fix build performance issues on Windows caused by the ~
characters.
Targets that don't handle Bazel repository and/or runfiles paths in a portable way can cause problems:
- Rules or programs that depend directly on paths in the execroot, rather than getting them via runfiles libraries or predefined source/output path variables, may break. Paths to specific files or directories within a filegroup target are especially susceptible to this kind of breakage.
- Rules that repackage external archives using rules_pkg and that strip
the repository path prefix may build successfully, but encode broken file
paths. This may be caught by verify_archive_test() from
rules_pkg
, but will otherwise pass undetected.
Hardcoding these paths to contain the current canonical repo names may get things working, but are vulnerable to breaking again at any time. It is possible to update such paths to properly include the new canonical repository names in a portable way. See the The Not Quite Ugly, Yet Not That Great, But It'll Do section below.
Runfiles libraries
Runfiles libraries are available for different languages to enable tests or other programs to locate their runfiles programatically during execution. I'll discuss runfiles libraries in greater detail in the next post in this series.
Some tools don't yet grok MODULE.bazel¶
Some tools may not yet handle MODULE.bazel
(ibazel), or you may need to
upgrade (bazelisk). Even Bazel's own local_repository() and
local_path_override() rules don't yet grok repos in the same source tree.
A lot of this has to do with a tool seeing WORKSPACE
as a repo boundary, but
not yet seeing MODULE.bazel
(or REPO.bazel) as such.
load()
is not allowed in MODULE.bazel
¶
WORKSPACE
files are evaluated top-to-bottom, and allow load()
statements
anywhere, in order to load macros from .bzl
files within an http_archive()
repository. These macros register the repository's own dependencies, as
well as the toolchains and other repositories that it provides.
Using load()
in a MODULE.bazel
file, however, will produce an error:
Because MODULE.bazel
files are evaluated differently than WORKSPACE
files ,
load()
statements aren't allowed. So if a Bazel module isn't yet available for
a dependency...
Module extensions are often necessary¶
...you may need to write your own module extension (or extensions) to
configure it. Module extensions aren't too different from other functions
defined in.bzl
files, but they are required to perform configuration tasks
previously performed directly in WORKSPACE
. Any http_archive()
followed by
load()
and a macro call in WORKSPACE
will require adding a module extension.
You can call http_archive()
in MODULE.bazel
to define a repository, then
define an extension in a .bzl
file to load()
and invoke its configuration
macros. Then you'll call the use_extension() function in MODULE.bazel
(instead of load()
) to bring the extension into scope. After resolving the
module dependency graph, Bzlmod will then evaluate the extension, invoking its
implementation function to instantiate its repositories.
Warning
You can't call http_archive()
and load()
a macro file from it in the
same module extension. All load()
statements must
appear at the top of the extension's .bzl
file, before any function
definitions that may call http_archive()
. In this way, Bzlmod forbids
dependencies between repositories instantiated within the same extension,
which macros have the potential to introduce.
Implicit repository registration is not supported¶
One of the biggest problems stemming from the inability to load()
configuration macros from a repository in MODULE.bazel
is registering
additional repositories.
Under WORKSPACE
, there is no built-in mechanism for resolving transitive
repository dependencies. The idiom of load()
ing a file from a repo to invoke a
macro to configure its dependencies arose in response to this. There was also no
need to reference these transitive dependencies directly in the WORKSPACE
file.
Under Bzlmod, each Bazel module will define the repos it requires without
requiring you to reference it explicitly. At the same time, all repos
referenced by your project's BUILD
files must be brought into scope within
MODULE.bazel
itself. You must do so using bazel_dep()
, repository rules
imported from module extensions by use_repo_rule()
(such as http_archive()
),
or by calling use_repo() on a module extension. The benefit is that you
can see how every direct dependency repository is configured all within
MODULE.bazel
.
Importing repositories that aren't Bzlmod-ready into
MODULE.bazel
requires dealing with the following consequences of these design
differences from WORKSPACE
:
- You have to write your own module extension separate from
MODULE.bazel
toload()
and invoke these configuration macros. - You have to import each transitive dependency repo generated by these macros
directly into
MODULE.bazel
. This is because there is no Bazel module other than_main
that can claim ownership of these transitive dependencies.
Fortunately, while sometimes tedious, writing these module extensions and
bringing them into scope with use_repo()
isn't that hard. It can get tricky
when a repo defines additional layers of repos that depend on one another.
Multiple repo versions may manifest, causing breakage¶
During the migration, you may introduce multiple versions of a repository without realizing it at first. If you depend directly on a non-Bzlmod version of a repository, a Bazel module dependency may introduce a second, Bzlmod-ready version as a transitive dependency.
This may not break the build at all—but it might. If it does, you may have
to add a MODULE.bazel
to the non-Bzlmod version and write a module extension
to resolve the incompatibility.
native.register_toolchains()
and native.bind()
are not supported¶
The other big problem stemmming from the inability to load()
configuration
macros deals with registering toolchains.
Module extensions do not support native.register_toolchains() or
native.bind() calls. For fully Bzlmod-compatible dependencies, that
module's extensions should provide a suitable API for configuring toolchains.
For non-module dependencies, in some cases, configuration macros tailored for
WORKSPACE
that lead to native.register_toolchains()
or native.bind()
calls
won't work. Whatever toolchains such macros used to register must be specified
explicitly in MODULE.bazel
or wrapped in a custom module extension.
MODULE.bazel.lock
can be noisy¶
The MODULE.bazel.lock file enables Bazel to resolve dependencies more
efficiently and deterministically than WORKSPACE
. By default, it will also
show when dependency information changes based on updates to MODULE.bazel
and
any module extensions.
Unfortunately, at the moment, it can also be a bit noisy. Adding and updating dependencies, particularly in large batches during a migration, generates huge diffs. Having different people use different versions of Bazel and related tooling can also result in diffs due to thrashing between lock file versions. The Module Extensions section is a particular source of volatility as well. The documentation even suggests using an automated git merge driver to resolve lockfile conflicts.
If you and your team are comfortable with the noise, or with configuring the merge driver, then go ahead check the lockfile into source control. Otherwise, you may wish to .gitignore the lockfile until after the migration, and after the tools and your team have gotten comfortable with Bzlmod.
Tip
Doing so won't break the build, as Bazel will update the file locally as
necessary; the changes just won't appear in git diff
. This effectively
maintains parity with the inability of WORKSPACE
to document resolved
dependencies. However, you may miss some of the benefits of detecting and
validating dependency updates, so do so judiciously.
The Not Quite Ugly, Yet Not That Great, But It'll Do¶
Despite some of these rough edges, there are some easy fixes and workarounds for some of the most common problems.
Fix broken runfiles paths using runfiles libraries or Bazel variables¶
Use runfiles libraries in tests and other programs to map runfile paths to their
actual locations under Bzlmod. Alternatively, apply predefined source/output
path variables in your rules to inject runfile paths into target programs
via command line arguments or env
properties. There are several variations of
doing so that we'll cover later, if the way forward isn't immediately obvious.
Fix broken paths with _main
(temporarily)¶
As a special case, for broken runfiles paths that contain the previous
workspace() name, replace that name with _main
.
Longer term, translating such paths using runfiles libraries or injecting them
via Bazel variables is the proper solution. But if updating all broken targets
will take a long time for any reason, this temporary approach won't make
existing uses any worse. It also provides an easily grep
pable string to help
discover files when you are ready to try applying runfiles libraries or Bazel
variables instead.
@@
is the new @
¶
In Bazel 7.0.0 and earlier, the repository label @
(as in @//foo
)
referred to the main repository. In Bazel 7.1.0 and later, the repository
label @@
now refers to the main repository (as in @@//foo
):
Labels starting with @@// are references to the main repository, which will still work even from external repositories.
The fix is to replace all instances of @//
in your BUILD
files with @@//
.
Conversely, if your project contains references to external repositories
starting with @@
(possibly a typo), you need to update them to start with @
instead. For example, @@repo
would become @repo
.
Info
I'm pretty sure this works when using WORKSPACE
, even since Bazel 7.1.0,
for two reasons. First, because WORKSPACE
doesn't mangle canonical
repository names. Second, because Bazel's Label::getShorthandDisplayForm()
method elides @@repo
to @repo
in the context of the main
repository.
Keep WORKSPACE
around for tools don't yet grok MODULE.bazel
¶
Keep an empty WORKSPACE
file around until you can upgrade the tools in
question. Better yet, have WORKSPACE
contain only comments explaining why it's
there and when it can go away. For example:
Text Only | |
---|---|
Ignore local_repository()
and local_path_override()
paths¶
Until the Bazel team resolves bazelbuild/bazel#22208, you can add any local_repository() and local_path_override() to your repository's .bazelignore file. This will enable you to build the project successfully—but it may interfere with your IDE's autocompletion feature.
Potential implications of (the not yet official) Vendor Mode
Bazel now advertises a Vendor Mode feature that may prove useful for
copying external dependencies into your main repository, appearing in
version 7.3.0rc1. It appears to be orthogonal to local_repository()
and local_path_override()
. (i.e., You can use Vendor Mode to copy the
dependencies, but you may still need the local_*
rules to build them.) See
also: bazelbuild/bazel: Bzlmod: vendor mode #19563.
The Hard Parts¶
Here's a list of deeper Bzlmod migration topics I plan to cover in depth. A number of these (but not all of them) involve working around problems in your dependencies.
Arguably one should wait for problems to be fixed upstream, or you should
contribute upstream fixes yourself. You can still migrate as many dependencies
as you can to Bzlmod, and keep your WORKSPACE
in place for the rest. However,
if you want to get on with completing your own Bzlmod migration, you need not
necessarily remain blocked by your dependencies.
You may find some of the upcoming advice useful—and applying it might even help you develop fixes to contribute upstream. Either way, you should be able to safely discard any dependency workarounds once they're no longer needed.
Canonical repo name-related fixes¶
This isn't the hardest class of problem to solve, but it requires careful handling to avoid depending on the canonical name format itself.
- Translating file paths via runfiles libraries
- Injecting file paths via predefined source/output path variables (runfiles, etc.)
- Injecting canonical repo names programmatically via genrule()s
- Injecting canonical repo names as custom Make variables
- Tactics for updating
rules_pkg
targets that need to strip file prefixes from external repository paths
Solving problems with module extensions¶
These posts will cover additional Bzlmod details accounting for specific challenges and provide fitting solutions (outside of the dependencies themselves becoming Bzlmod-ready).
- Defining repositories using
load()
ed constants - Avoiding circular repository definitions between
MODULE.bazel
and extensions when migrating a series of relatedWORKSPACE
statements
Patching external repos/modules to solve tricky problems¶
When in doubt, use brute force—i.e., patch your dependencies.
These are some of the problems you can solve using patches in http_archive()
,
archive_override(), and other repository rules to work around problems in
your dependencies. (Until the problems are properly fixed upstream, of course.)
- Avoiding
native.register_toolchain()
andnative.bind()
calls - Resolving breakages from non-Bzlmod and Bzlmod-ready versions of a repo in the same build
- Hacking around Windows path length issues due to longer canonical repo names
- De-mangling canonical repo names to fix breakages in a low-risk way
Migrating from rules_nodejs to rules_js + rules_ts¶
This will be a series in itself, given the fundamental differences between
rules_nodejs
(deprecated and not Bzlmod-ready) and rules_js
(Bzlmod-ready):
rules_nodejs
exposes an@npm
repo that makes all transitive dependencies accessible.rules_js
contains a newjs_library
implementation with different providers that are required by its other rules (i.e., thejs_library
fromrules_nodejs
won't work with them). It also exposes a more restrictive set of//:node_modules
targets that are less tolerant of improperly declared transitive dependencies.
In addition to dependency graph details, there's also the matter of how
different npm
s adapt to these different repository models and their
implementation details.
Conclusion¶
Bzlmod may not be perfect, but it does provide many benefits over the previous
WORKSPACE
model. However, for large, complex code bases deeply dependent on
the existing WORKSPACE
ecosystem, migrating to Bzlmod may require some
persistence, insight, and ingenuity.
This post, and the future posts in this series, aim to make some key Bzlmod migration insights and techniques more accessible. Even if your codebase presents specific challenges not explicitly covered by these posts, hopefully they at least provide sufficient inspiration to help you overcome them.
Stay tuned—there's lots more fun to come!
Updates¶
2024-08-09¶
- Started adding links to every post in the series in the introduction. Stole this idea from Jay Conrod's "Writing Bazel Rules" series.
2024-07-31¶
- Updated the
rules_python
section to clarify that version 0.32.0 added support for Python versions down to the patch level. (Hat tip to Ignas Anikevicius for the notification.) - Updated to mention runfiles libraries throughout, and to mark the
_main
path component fix as temporary. - Updated Vendor Mode info to note its inclusion in Bazel 7.3.0rc1.
2024-07-12¶
- Updated the
rules_jvm_external
example to use version 6.2, which eliminates the need for theunpinned_maven
repo. (Hat tip to Simon Stewart for the notification.) - Added a reference to the
rules_python
issue corresponding to the describedWORKSPACE
breakage when enabling Bzlmod. - Added a reference to the upcoming change to the canonical repo name format to address Windows performance issues.
- Clarified that adding
MODULE.bazel.lock
to.gitignore
effectively matches the inability ofWORKSPACE
to document resolved dependencies. - Updated link to Bazel 7.3.0 Vendor mode issue to reflect its current "not planned" status.