Build Forensics: How we uncovered a Rust HTTP/2 bug
Last year, one of our customers began reporting that 5% of their jobs based on Meta's Buck2 build system were failing. Even though there was no simple way to reproduce the phenomenon, all signs pointed to network hangs when a Buck2 client made API calls to their EngFlow cluster. If a simple timeout was added, the request would quickly time out, potentially indicating the issue was a bug in our remote execution platform.
Because of the potential for it to expose a broader issue in our software, especially in terms of compatibility issues with Buck2, we elevated the urgency of this investigation. After examining a few thread dumps that revealed nothing conclusive, we eventually scheduled a call with the customer. After performing multiple tests, it became evident that our schedulers were simply idle. If it were truly a problem in our system, we were unable to find evidence.