"The growth rate didn't match what we would expect from normal development activity, even at Dropbox's scale. That suggested the problem wasn't just what we were storing, but how it was being stored."
"Git uses delta compression to reduce storage by identifying similarities between files and storing differences efficiently. At scale, Dropbox engineers observed that these heuristics produced suboptimal packfiles, resulting in disproportionately large repository growth compared to actual code changes."
Dropbox engineers addressed inefficiencies in Git's storage and delta compression model, reducing their backend monorepo size from 87GB to 20GB. This optimization improved developer productivity and continuous integration performance. Slow clone operations and degraded CI pipeline performance were challenges faced as the monorepo grew. The issue stemmed from Git's internal compression heuristics, which produced suboptimal packfiles. The team implemented optimized repacking strategies and adjusted Git's object delta structures to enhance performance and manage storage behavior effectively.
Read at InfoQ
Unable to calculate read time
Collection
[
|
...
]