Bazel scales more than just builds

Bazel creates a structured layer between the details of building and testing for individual languages and the users, both human and machine, that perform build operations. This abstraction simplifies workflows and is the foundation for powerful build-adjacent systems not possible with other build tools. Scalable Bazel builds underpin a scalable organization and development culture.

Build problems in a large software development organization

"Software builds" become a priority for engineering leaders when complaints rise up from their organization about developer happiness and productivity. An organization’s code base has exploded in size, and existing tools are grinding to a halt. Build and test cycles grow from seconds to minutes and hours. Shipping features becomes a months-long slog. Proactive engineering leaders notice these patterns early and initiate the search for alternative, modern tooling. That often drives a deeper evaluation of the fundamental language-specific build system versus the newer generation cross-language build and test platforms. The centrality of builds and build systems to engineering productivity has become apparent.

Some organizations ran into the challenges of big code earlier than others. Sixteen years ago Google created an internal build system to address them. This system was open-sourced as the multi-language build system Bazel in 2015. Around the same time, Facebook and Twitter created their own Bazel-like systems, Buck and Pants. Of these newer generation build platforms, Bazel has seen the most significant adoption and community engagement. Even in its beta version in 2016, it was adopted by world’s most innovative companies contending with significantly growing critical codebases. Among the Bazel adopters are SpaceX, Uber, Stripe, Wix, Dropbox, and Spotify.

Language-native build systems hit scaling limits

Bazel was created to scale large development organizations while integrating the experiences and needs of thousands of individual developers.

In evaluating Bazel, an individual engineer’s experience with Bazel is often compared with their experience using other build systems like make or language-native systems like Rust’s cargo. When working in single-project, single-language repositories, these tools indeed often work well, and Bazel provides no compelling advantage. However, while language-native build systems have the advantage of familiarity, maturing organizations often find themselves searching for relief from the pain language-native build systems cause.

The first common issue is scale. Many language-native build systems were not designed for the size of codebases and engineering teams today at medium to large companies. These build systems consequently are stretched past the breaking point by large-scale software engineering leading to slow builds, mounting infrastructure costs, and untameable complexity.

The second common issue is polyglot engineering. Even the slickest language-native build systems often have no answer for cross-language dependencies. Yet, almost every software company has systems comprising multiple programming languages.

The third common issue is cultural. Many build systems are notorious for their incorrectness and cause engineers to not trust their tools. Unhappy developers learn they must perform bad practices like manually mutating internal build system state or running slow builds from scratch when they encounter inscrutable build problems.

The result of these issues is that the growing organization becomes sharded into separate cells, each speaking their own development and build dialect, unable to coordinate efficiently. Everyone wrestles with local fixes and improvements while global development velocity and quality suffers. The organization grows weaker as hundreds or thousands of individuals feel the aggravation by an inefficient system over which they exert little influence.

How Bazel helps

Bazel addresses these issues by providing an abstraction over language-specific build details. Build system and language experts feed their knowledge of how to efficiently build a language into Bazel using the powerful extension API. For engineers who are the “end user” of the build, Bazel then exposes a high-level interface for uniformly declaring library, executable, and test targets across languages.

Upon this foundation, many technical and cultural benefits emerge.

Portability and agility

While engineers accustomed to language-native build systems will need to invest some time in learning Bazel, the investment can save time later. In particular, a uniform Bazel build interface enhances developer portability. Consider a backend engineer used to running their service’s unit and integration tests with bazel test. How do they run tests when they need to work on the web frontend or mobile app? bazel test! They have no need to install new ecosystem tools or run unreliable setup scripts.

Bazel also elegantly solves the cross-language dependency problem; shipping a service with a foreign binary that it shells out to or giving an integration test access to a dependency service are simple operations reflected in the build graph.

Combined with common language guides and effective testing practices, this enables people to switch between projects very quickly. Even if switching only temporarily, such as when debugging an internal dependency, this consistency enables people to be effective quickly.

Erasing differences between local development and continuous integration

It’s not just humans that benefit from the consistent Bazel build interface. Developer tools and infrastructure generally can lean on Bazel to abstract over the different languages used in a company. The CI system, for example, does not have to know about the package managers and build tools of individual languages. It simply runs bazel build and bazel test like the humans. With a hermetic, reproducible build, both engineers and CI systems see the exact same build artifacts and test results.

Tests are first class part of the build

Bazel’s build abstraction is special in that it includes tests. Unlike most other build systems, Bazel recognizes that tests are simply another kind of build action. Thus, tests in Bazel accrue all the benefits of being “in the build”. For example, tests can start running while other targets are still building avoiding the artificial synchronization caused by separate build and test steps. Tests can even be cached if their dependencies do not change.

Bazel superpowers

While Bazel cannot exactly provide the experience of a language-native build system, it offers its own ecosystem of tools. Bazel exposes the build graph through bazel query, which enables deep understanding of a codebase’s structure. Bazel's Build Event Protocol provides comprehensive data about each Bazel invocation to third-party tools for observability and debugging. Techniques like remote caching, remote execution, profile analysis, and target selection built on Bazel can both improve developer iteration speed and save money.

Addictive correctness

To discourage fast-yet-incorrect build hacks, Bazel prioritizes the hermeticity and reproduciblity of build actions with sandboxing.

It’s admittedly a challenge to bring previously incorrect builds up to Bazel’s standards. Once it’s done, though, the benefits are legion. Correct incremental builds are addictive for engineers. Tools like remote caching and remote execution are primarily possible because Bazel demands the hermeticity and reproduciblity of build actions.

Bazel enables a consistent and scalable development culture

While comparing day-to-day workflows are critical in picking tools, the organization-level advantages and disadvantages of various tools also merit attention. This is critical when coordinating separate projects within a large organization working on a large codebase—and this is where Bazel shines. Bazel provides a consistent build interface across a codebase by cleanly separating build details from the abstraction used to modify, perform, and inspect builds. By investing in modern build tooling, engineering leaders can help their engineering iteration loops scale with the organization’s growth and foster high-quality software engineering culture.