Blaise Sanouillet dd95bf65b6 [caffe2/FC DNNLOWP] Shrink Y_int32_ vector capacity when appropriate ... Summary: The FullyConnectedDNNLowPOp::Y_int32_ vectors consume between 1GB and 2GB on one of FB's larger applications. By adding tracing I noticed that the number of elements in each instance oscillates wildy over time. As the buffer backing a vector can only be extended in a resize operation, this means there is wasted memory space. So as a simple optimization, I added code to right-size the buffer backing the vector when the number of elements is less than half the vector capacity at that point; this doesn't affect the existing elements. There is of course a memory/cpu tradeoff here - with the change we are doing more mallocs and frees. I added tracing to measure how many times we grow or shrink per second: it's about 100 per second on average, which is not a great deal. Test Plan: Memory growth impact: over 24 hours and after the startup period, the memory consumed by this code grows from 0.85GB to 1.20GB vs 0.95GB to 1.75GB in the baseline. [ source: https://fburl.com/scuba/heap_profiles/wm47kpfe ] https://pxl.cl/1pHlJ Reviewed By: jspark1105 Differential Revision: D24592098 fbshipit-source-id: 7892b35f24e42403653a74a1a9d06cbc7ee866b9		2020-10-29 11:19:45 -07:00
..
server	[caffe2/FC DNNLOWP] Shrink Y_int32_ vector capacity when appropriate	2020-10-29 11:19:45 -07:00
__init__.py	remediation of S205607	2020-07-17 17:19:47 -07:00
CMakeLists.txt	operators/quantized/server -> quantization/server (#13660)	2018-11-07 22:54:13 -08:00