Democratizing Build Scalability: Eugene Yokota on Bringing Bazel Features to sbt 2.0¶
In this interview, Eugene Yokota—a software build expert who spent years maintaining Scala's sbt tool at Lightbend before working with hyperscaled Bazel monorepos at Twitter and Netflix—details his multi-year project to build a Bazel-compatible remote caching system into the newly released sbt 2.0. He explores the mechanics and benefits of Bazel, such as its robust remote caching and test cycle speed, and highlights how these modern, scalable build tools can eliminate CI bottlenecks for growing teams while protecting toolchain security.
When did you first get interested in Scala, sbt, and Bazel?¶
After being an sbt enthusiast and plugin author for several years, I was called in to essentially take on Mark Harrah's role with Josh Suereth when Mark left Typesafe (now Lightbend/Akka).
I didn't know almost anything about open source, but Josh taught me a lot of things. Among them, he taught me about this hyperscaled build tool that internal Googlers were using, called Blaze at the time.
How did Bazel become a career focal point for you?¶
Even back in the 2014 era, people knew that Google was using this really interesting build tool that was capable of remote caching in order to share the build product. Google was probably one of the first companies to reach that hyperscale where you can no longer build everything on your computer—or you shouldn't, because there are too many artifacts. This was in the early days of continuous integration and cloud workloads; all of it was emerging around the same time.
Then in 2015, Blaze got open-sourced by Google and renamed Bazel. Until then, Twitter was developing Pants, and Facebook was making Buck, so there were all these build tools popping up. Honestly, for the seven years I was at Lightbend maintaining sbt, I knew a lot about Bazel, but I was almost like a fanboy—like a person who's really into skating and reads all the skateboarding magazines, but never actually gets on the skateboard.
Then I joined Twitter because they were looking for people to help migrate their Pants monorepo to a Bazel monorepo. For several years, I worked with one of the biggest build teams I've ever seen—dozens of people, serving about 2,000 engineers. They were all writing really amazing Scala code and using Bazel with their own data centers; it was a pretty amazing experience.
I currently work for Netflix, helping manage a much smaller monorepo environment within the machine learning organization, where we also use Bazel. Machine learning is an emerging area where data processing and ML code intermixes different teams and programming languages, and Bazel is uniquely capable of addressing that. With one build tool, you can elegantly handle Python, Rust, and Scala all together.
When did you get the idea to bring Bazel into sbt 2?¶
Around five years ago, even though I had my day job, I'd occasionally during the summer go to Connecticut because I think of it as kind of like the economy plus of the Hamptons. Across the Long Island Sound is Connecticut, and it's like a tiny calm ocean that's protected by Long Island. I went on summer vacation and thought “maybe I should start working on this crazy project where I take sbt and then kind of take it to almost like a theoretical limit of what I think is the ultimate build tool using Scala 3”.
If I were to explain what sbt 2 is, it's sbt written using Scala 3 that uses a Bazel compatible cache system. The motivation is that if I were to kind of ride off into the sunset and have to give up on sbt at some point, I thought one of the things I could leave behind is transporting sbt into Scala 3, a current version of Scala.
No one understands how the macro works in sbt systems; it's the core code that's very complicated and it's written using Scala 2 macros. You can't just drop and shift it; someone has to read it and understand everything in order to do this port work. So during these writing retreats or summer vacations, I started porting the macro system to an experimental version of sbt 2.
How does Bazel’s remote caching help scale PRs?¶
To think of scaling, let’s imagine you have a 10-person team. On Monday morning you type git pull and you're basically compiling not only your code but nine other people's code.
If you want to scale that 10-person team to a 20-person team, now you're compiling 19 people's stuff. You go to the coffee room and say, "Hey, I'm just gonna let this do it for a while," and there are going to be 19 other guys doing the exact same thing. Essentially, the repository and the CI system grows in an explosive, quadratic curve as you add more and more people.
But when sending pull requests to multiple repositories, the true friction isn't even tooling; it's just a human in the middle. This turns into a whole multi-day journey because you have to talk to people, then they have to release the JAR to Artifactory, and you have to consume it. Whereas with a monorepo strategy this all goes away. You have access to all the parts, so you write your pull request—and it doesn't even have to be good in draft mode—and you can do experimental things on your own to prove out that an idea works.
Nowadays you can use GenAI to do the exploration, and having a monorepo is truly a zero-to-one capability for this. If you put a human in the loop, microservices mean micro pull requests, which is a huge friction.
How does Bazel speed up test cycles?¶
Bazel’s remote caching and overall basic architecture allows build scalability to go linearly with team size because everybody shares binary outputs and you don't have to build everyone else's changes every time. One side is the developer productivity aspect for a laptop experience, but another thing that sometimes people tend to ignore is testing capability. When you send a pull request, you're actually running thousands, or tens of thousands, of tests.
What Bazel allows you to do is act like a cheat code for builds by caching the test result. If some human has ever written a specific kind of test, that test never runs again. If you construct your Bazel monorepo correctly, your CI should end within minutes because it only ran like five tests. But in companies where you couldn't get the hermeticity correct and caching set up correctly, if you're running thousands of tests all the time due to cache misses, your build is going to take 30 minutes or maybe hours. How you construct your monorepo sometimes is just as important to make sure that small changes don't invoke thousands of tests.
How does Bazel treat toolchain security?¶
Bazel uses SHA-256 encryption for digests, file sizes, and task inputs and outputs. It's kind of like the building block that makes sure that whatever you produce from your computer, other people can produce it too. Instead of doing file comparison because it's too much work, Bazel creates this thing called a digest, which consists of the hash algorithm—typically SHA-256—and the length of the file.
That's a really clever trick Google came up with because it's very difficult to fake both at the same time. If no one is injecting some code, Bazel systems will catch it because your SHA doesn't match up. It's certainly a great practice to lock things down that way. The tricky part is that SHA-256 is actually really slow on thousands of files, so there's a tradeoff of how you can make it fast and how you can make it secure. It's really concerned about reproducibility and that definitely leads to having a good secure infrastructure.
Is it true that the German government helps keep sbt funded?¶
Yes, in fact! Some years ago, we transferred the legal rights to the Scala Center. About a year ago, we sent an application to the Sovereign Tech Fund in Germany, which basically says that open source is used as critical infrastructure to run a bunch of big things like Apache Spark datasets or powering nuclear power plants.
We were granted multi-year funding as a critical digital infrastructure to develop and maintain sbt 2 going forward. Helping me is another guy in Japan named Anatoli, and he and I work together to maintain sbt. It’s definitely been a community effort as well because we have about 100 plugins that need to be ported to sbt 2.
When did sbt 2 launch and who helped?¶
We released sbt 2.0.0 in June 2026. Now by default it caches the task, which means that similar to Bazel, on the CI it would only run the test that you changed. This gives you a chance to take your existing CI to a whole different limit. But caching means that you have somewhere to cache. In a CI system you definitely need a disk cache, but that is going to fill up at some point.
What you really want to have is a remote caching system. If you're using it at work and you want your CI time to scale after your team is over 10 people, and your build becomes a bottleneck taking more than 30 minutes, I'd say either try setting it up yourself or contact one of the companies to run the remote cache for you. That's really what I wanted to do with sbt 2.
I also have to shout out Bill Autrey. Billy was one of the few people who believed in sbt 2 to happen, even when a lot of people thought it was vaporware in my crazy head. It was taking like five years, and he joined my Slack channel, and while taking a walk in New York City with him and others at EngFlow, he offered to help. I'm thankful to Billy (now my teammate at Netflix) and by extension EngFlow, because when he started contributing to sbt 2 he was at EngFlow, and there was a client who was interested in remote caching for Play Framework. That’s where things really started moving forward.
–
Thanks to Eugene Yokota for his time in this interview. Follow his blog at https://eed3si9n.com/