How We Saved 12% in Resources with Smarter Heap Management
Briefly

from Medium3 months ago
To investigate the memory issues we were experiencing in our new Scala service, we utilized a heap dump from Jxray. This revealed underutilized buffers managed by the Netty framework.
Mediumhttps://engineering.depop.com/how-we-saved-12-in-resources-with-smarter-heap-management-233f661e870d?gi=9701eca2b096
The Play WS client, relying on an older version of AsyncHttpClient, was using 16MB buffers by default, leading to excessive memory consumption. We optimized our buffer size to 4MB, reducing memory usage significantly.
Mediumhttps://engineering.depop.com/how-we-saved-12-in-resources-with-smarter-heap-management-233f661e870d?gi=9701eca2b096
Through Datadog's memory leak feature, we noticed persistent growth in the 'Heap After Full GC'. This indicated that some long-lived objects in the Old generation weren't being collected.
Mediumhttps://engineering.depop.com/how-we-saved-12-in-resources-with-smarter-heap-management-233f661e870d?gi=9701eca2b096
The Jsoniter JSON library, using a default 32KB buffer, was causing issues when handling larger data. Buffers expanded rapidly, generating temporary garbage that contributed to out-of-memory errors.
Mediumhttps://engineering.depop.com/how-we-saved-12-in-resources-with-smarter-heap-management-233f661e870d?gi=9701eca2b096
Read at Medium
#scala #memory-management #garbage-collection #netty #json-serialization
[
|
]