Skip to content

Home

Not all builds are made equal: Using priorities to expedite remote execution of the builds and tests that matter most

TL;DR

Imagine reading this post on your internal Slack:

⚠ Please hold off on pushing your PRs for a bit until the patch release is fully cut. We don't have enough capacity to handle all the load and the patch is blocked because of that.

Ouch.

But let's be real: not all builds are made equal - some builds are more urgent than others. Like here, when creating a patch release, and wanting to expedite the builds required for it over others.

Yet there's got to be a better solution than asking everyone else to not push changes.

Migrating to Bazel Modules (a.k.a. Bzlmod) - Fixing and Patching Breakages

In the previous Bzlmod post, we covered writing your own Bazel module extensions to adapt your own setup code for dependencies that aren't Bzlmod compatible. However, there are other Bzlmod incompatibilities and related breakages that module extensions alone can't fix, such as forbidden API usage or Windows path length errors.

This post shows you how to patch your dependencies and covers several situations where patching is the only solution. We'll describe how to create and apply patches for your dependencies, if you can't wait for upstream fixes (or contribute them yourself).

What I learned in Colombia

EngFlow has multiple entities around the world, one of which is in Colombia. This year we’ve chosen Colombia as the site for our 2025 Leadership Summit, and I added a few days and weekends to explore the country with our team members, see how they live and meet their families. It’s been an incredible experience visiting Medellin, El Retiro, Itagui, Cali and Bogota, and I’ve captured a few learnings from this trip in a blog.

Colombia team photos

Migrating to Bazel Modules (a.k.a. Bzlmod) - Module Extensions

So far, we've covered relatively easy Bzlmod fixes, how to hold runfiles and pkg_tar correctly, and how to access canonical repo names when absolutely necessary. Now we'll discuss replacing WORKSPACE statements with your own module extensions. You can use them to wrap your project's setup macros, and to adapt external repositories that aren't Bzlmod compatible to work with your Bzlmodified project.

EngFlow 2024 Year End Wrap

As we look forward to 2025, we are grateful to our employees, customers, and the broader build community for the continued investment in achieving our mission of making developers productive and happy to keep Engineers in Flow.

Holiday baking

In 2024 EngFlow continued our multi-year track record of exponential growth. This year we onboarded some of the most complex and advanced engineering organizations across autonomous driving, e-commerce, SaaS, finance, and chip manufacturing, resulting in significant developer velocity and cloud cost savings for these teams.

EngFlow has become more than a platform provider - we are elevating the developer experience practice for our customers, continuously pushing the cost and performance innovation, passing the savings directly to our customers. We are connecting engineers working on similar technologies to share best practices and talent across the Bazel, Buck2, Chromium, AOSP, and CMake ecosystems. While doing that, we're enriching our customers with music and are making the planet a greener place (read more about this below)!

How to Evaluate Remote Caching and Execution

Between roughly 2006 to 2008, Google developed remote caching and execution technologies to scale its massive monorepo based software development operation. This platform included Forge, the remote caching and execution platform, and Blaze, a tool for building large multilanguage software projects, eventually open sourced as Bazel. The advantages of this original platform were so obvious that it literally sold itself, and ultimately inspired the EngFlow platform.

Today, EngFlow is one of several competing remote caching and execution products now available in the commercial space. This post describes how we continuously benchmark our own product against different configurations to ensure that we offer the best possible value. We hope that sharing our methodology might help you evaluate whether remote caching or execution is right for your organization.

Build Meetup in Tokyo — Recap

Co-hosted by EngFlow and Google, the build community in Tokyo came together for an afternoon of tech talks and a happy hour filled with beverages and good vibes.

The event went straight into talks, starting with introductory words from our Google host, Philipp and EngFlow Developer Support Engineer Kip. The talks that followed spanned build systems and build issues:

Following the talks, we headed out to a nearby bar for some canapés and drinks, meeting fellow build engineers and enthusiasts from around the region.

Migrating to Bazel Modules (a.k.a. Bzlmod) - Repo Names, Macros, and Variables

The previous two posts in this series showed how to use runfiles mechanisms and rules_pkg mechanisms to avoid dealing with canonical repository names under Bzlmod. However, one special case remains: when you need to depend on the name of a repository directory, either at build time or runtime. This post explains how to access canonical repository names in a portable way to solve such problems. We'll use a macro when we can, and a custom Make Variable when we can't, including when dealing with alias targets.