Pinterest Engineering Reduces Android CI Build Times by 36% with Runtime-Aware Sharding

"Pinterest published a technical case study detailing how its engineering team cut Android end-to-end (E2E) continuous integration (CI) build times by more than 36 percent by adopting a runtime-aware test-sharding strategy and building an internal testing platform. Before the change, Pinterest's Android CI pipeline suffered from slow, unpredictable build times because its test suite was split by package names on a third-party platform, causing the slowest shard to gate the entire build."

"This not only prolonged total build times but also introduced flakiness and setup overhead that added minutes to each run. After evaluating third-party alternatives, the team concluded that none met its requirements for native emulator support, reliability, and fine-grained control without a custom orchestration layer. This analysis led to the development of PinTestLab, an in-house testing infrastructure running Android emulators on EC2 instances, giving engineers full control over scheduling, environment setup, and runtime orchestration."

"The core innovation involved a runtime-aware sharding algorithm that uses historical test execution data stored in Pinterest's Metro test management system to group tests by expected duration rather than count. Instead of simple round-robin or package-based splits, the algorithm sorts tests by expected runtime and assigns them to shards in a way that minimizes variation in wall-clock execution time across shards. Using this method, the team compressed the gap between the fastest and slowest shards from hundreds of seconds to just a few dozen seconds,"

Pinterest's Android CI pipeline was slow and unpredictable due to package-based test sharding on a third-party platform, where the slowest shard gated builds. Reliance on Firebase Test Lab introduced flakiness and setup overhead that added minutes per run. Engineers built PinTestLab, an in-house testing platform running Android emulators on EC2 to gain full control over scheduling, environment setup, and orchestration. A runtime-aware sharding algorithm uses historical Metro test execution data to group tests by expected duration and assign them to shards to minimize wall-clock variation. Balanced shard execution reduced the gap between fastest and slowest shards from hundreds of seconds to a few dozen, cutting end-to-end CI times by more than 36 percent.

Read at InfoQ