pytorch/caffe2/core
Rohith Menon 879a90b322 [ModelLoading] Use byte encoding for uint8, fp16 etc. instead of int32 (#34343)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34343

Use byte encoding for uint8, fp16 etc. instead of int32 in TensorProto serialization/deserialization

tl;dr
- fp16 tensor deserialization 12x faster, serialized size 25% lower
- uint8 tensor deserialization 36x faster, serialized size 25% lower

Test Plan:
```
============================================================================
caffe2/caffe2/fb/predictor/ModelLoaderBenchmark.cpprelative  time/iter  iters/s
============================================================================
BlobProtoInt32DeserializationFloat16                        12.37ms    80.82
BlobProtoByteDeserializationFloat16             1125.46%     1.10ms   909.64
----------------------------------------------------------------------------
BlobProtoInt32DeserializationUInt8                          17.57ms    56.92
BlobProtoByteDeserializationUInt8               3629.45%   484.02us    2.07K
============================================================================
```

Reviewed By: yinghai

Differential Revision: D20137451

fbshipit-source-id: 8ed4be2286a6d4c7e134fcb0832f22bc645039a1
2020-03-06 11:58:30 -08:00
..
hip Replace caffe2::DeviceGuard with c10::cuda::CUDAGuard (#17623) 2019-03-06 10:48:15 -08:00
nomnigraph fix warnings reported by PVS (#33868) 2020-03-02 18:51:38 -08:00
__init__.py re-enable copy of python files, but be careful that the copy is only … (#14982) 2018-12-11 16:54:08 -08:00
allocator.cc unify c2 and TH allocator (#16892) 2019-02-12 21:16:34 -08:00
allocator.h unify c2 and TH allocator (#16892) 2019-02-12 21:16:34 -08:00
asan.h
blob_gpu_test.cc Replace caffe2::DeviceGuard with c10::cuda::CUDAGuard (#17623) 2019-03-06 10:48:15 -08:00
blob_serialization_gpu.cc
blob_serialization.cc [ModelLoading] Use byte encoding for uint8, fp16 etc. instead of int32 (#34343) 2020-03-06 11:58:30 -08:00
blob_serialization.h Fix typos, via a Levenshtein-type corrector (#31523) 2020-01-17 16:03:19 -08:00
blob_serializer_base.h
blob_stats.cc
blob_stats.h Fix include paths for typeid.h (#13689) 2018-11-14 18:04:09 -08:00
blob_test.cc [ModelLoading] Use byte encoding for uint8, fp16 etc. instead of int32 (#34343) 2020-03-06 11:58:30 -08:00
blob.h Adding quantized tensor shape/type info support for caffe2=>glow in caffe2 side (#18621) 2019-03-31 17:42:27 -07:00
CMakeLists.txt get rid of protobuf dependencies (#25650) 2019-09-06 08:48:20 -07:00
common_cudnn.cc
common_cudnn.h
common_gpu.cc switch to rocThrust for thrust/cub APIs (#25620) 2019-09-03 22:16:30 -07:00
common_gpu.h Split libtorch.so back into libtorch_{cpu,cuda,hip} (#30315) 2019-12-04 08:04:57 -08:00
common_omp.h
common_test.cc Windows shared build (#13550) 2018-11-16 12:16:28 -08:00
common.cc
common.h fix windows clang attributes (#33959) 2020-03-02 13:20:51 -08:00
context_base.cc Remove Tensor.h, TensorMethods.h from src/core. (#27086) 2019-10-06 09:37:50 -07:00
context_base.h Fix is_fundamental template for MSVC (#30959) 2019-12-19 12:10:22 -08:00
context_gpu_test.cc Replace caffe2::DeviceGuard with c10::cuda::CUDAGuard (#17623) 2019-03-06 10:48:15 -08:00
context_gpu.cu [caffe2] fix no matching function min/max Clang errors (#33563) 2020-02-28 11:33:24 -08:00
context_gpu.h Fix typos (#30606) 2019-12-02 20:17:42 -08:00
context_test.cc unify c2 and TH allocator (#16892) 2019-02-12 21:16:34 -08:00
context.cc Change Tensor::CopyFrom to a simple double dispatch (#14268) 2018-11-28 15:45:37 -08:00
context.h Call RandomNumberSeed() on-demand (#33539) 2020-02-22 01:22:18 -08:00
cudnn_wrappers.h Replace caffe2::DeviceGuard with c10::cuda::CUDAGuard (#17623) 2019-03-06 10:48:15 -08:00
db.cc Apply modernize-use-override (2nd iteration) 2019-02-14 16:52:57 -08:00
db.h Change message on unknown db type to be friendly (#17795) 2019-03-08 10:46:24 -08:00
event_cpu.h
event_gpu_test.cc
event_gpu.cc Add error message if CUDA startup fails (#29670) 2019-11-13 16:48:40 -08:00
event_test.cc caffe2/event: allow multiple errors such as when cancelled (#31335) 2019-12-18 13:10:57 -08:00
event.cc caffe2/event: allow multiple errors such as when cancelled (#31335) 2019-12-18 13:10:57 -08:00
event.h Fix issues under caffe round 1 2019-01-23 19:04:59 -08:00
export_c10_op_to_caffe2.cc Rename caffe2<->c10 operator wrappers (#21322) 2019-06-07 13:48:10 -07:00
export_c10_op_to_caffe2.h Replace c10::guts::stuff with std::stuff (#30915) 2019-12-16 13:57:19 -08:00
export_caffe2_op_to_c10.h [jit] do the code reorg (#33851) 2020-02-27 13:02:51 -08:00
flags.h
graph_test.cc
graph.cc
graph.h
init_denormals.cc add denormal options (ftz and daz) 2018-12-20 17:04:39 -08:00
init_intrinsics_check.cc cleanup warnings 2019-08-12 16:12:30 -07:00
init_omp.cc
init_test.cc Allow for registration after GlobalInit (#15876) 2019-01-10 09:35:33 -08:00
init.cc Update intra_inter_benchmark (#22051) 2019-06-21 23:06:27 -07:00
init.h Update intra_inter_benchmark (#22051) 2019-06-21 23:06:27 -07:00
int8_serialization.cc Fix include paths for typeid.h (#13689) 2018-11-14 18:04:09 -08:00
logging.h
macros.h
macros.h.in Remove duplicated Numa detection code. (#30628) 2020-01-03 08:48:46 -08:00
memonger.cc Fix typos, via a Levenshtein-type corrector (#31523) 2020-01-17 16:03:19 -08:00
memonger.h
module_test.cc
module.cc Allow checking for cached module before asserting (#33954) 2020-03-02 15:43:50 -08:00
module.h cleanup warnings 2019-08-12 16:12:30 -07:00
net_async_base.cc Replace c10::guts::stuff with std::stuff (#30915) 2019-12-16 13:57:19 -08:00
net_async_base.h Remove Tensor.h, TensorMethods.h from src/core. (#27086) 2019-10-06 09:37:50 -07:00
net_async_scheduling.cc AsyncIf op implementation 2019-11-07 08:51:31 -08:00
net_async_scheduling.h AsyncIf op implementation 2019-11-07 08:51:31 -08:00
net_async_task_future.cc Replace c10::guts::stuff with std::stuff (#30915) 2019-12-16 13:57:19 -08:00
net_async_task_future.h Back out "Revert D13043261: [caffe2] Task graph and task future abstractions in executor" 2018-12-10 19:30:58 -08:00
net_async_task_graph.cc Replace c10::guts::stuff with std::stuff (#30915) 2019-12-16 13:57:19 -08:00
net_async_task_graph.h Back out "Revert D13043261: [caffe2] Task graph and task future abstractions in executor" 2018-12-10 19:30:58 -08:00
net_async_task.cc Back out "Revert D13043261: [caffe2] Task graph and task future abstractions in executor" 2018-12-10 19:30:58 -08:00
net_async_task.h Back out "Revert D13043261: [caffe2] Task graph and task future abstractions in executor" 2018-12-10 19:30:58 -08:00
net_async_tracing_test.cc AsyncNet: option for time based tracing and trace path (#13440) 2018-11-08 11:34:34 -08:00
net_async_tracing.cc Fix typos, via a Levenshtein-type corrector (#31523) 2020-01-17 16:03:19 -08:00
net_async_tracing.h Add batch id to tracer event (#21446) 2019-06-13 17:13:42 -07:00
net_dag_utils_test.cc
net_dag_utils.cc Fix typos (#30606) 2019-12-02 20:17:42 -08:00
net_dag_utils.h Back out "[nomnigraph][executor] computeChains with nomnigraph" (#15451) 2018-12-21 11:09:27 -08:00
net_gpu_test.cc Remove unused executors, part 3 (#14199) 2018-11-26 19:10:43 -08:00
net_parallel.cc Replace c10::guts::stuff with std::stuff (#30915) 2019-12-16 13:57:19 -08:00
net_parallel.h TBB task graph (#15041) 2018-12-10 21:35:04 -08:00
net_simple_refcount_test.cc Remove unused executors, part 3 (#14199) 2018-11-26 19:10:43 -08:00
net_simple_refcount.cc Fix typos, via a Levenshtein-type corrector (#31523) 2020-01-17 16:03:19 -08:00
net_simple_refcount.h net_simple_refcount type to help experimentation with dynamic allocation. (#13370) 2018-10-31 15:59:16 -07:00
net_simple.cc Back out "Back out "[Caffe2] Fix device_option propagation"" (#25908) 2019-09-17 04:01:36 -07:00
net_simple.h
net_test.cc fixup unit tests (#34105) 2020-03-03 10:33:21 -08:00
net.cc Lightweight at-most-once logging for API usage (#20745) 2019-05-23 23:17:59 -07:00
net.h Fix typos, via a Levenshtein-type corrector (#31523) 2020-01-17 16:03:19 -08:00
numa.cc Move numa.{h, cc} to c10/util (#15024) 2018-12-12 12:21:10 -08:00
numa.h Move numa.{h, cc} to c10/util (#15024) 2018-12-12 12:21:10 -08:00
observer_test.cc Replace c10::guts::stuff with std::stuff (#30915) 2019-12-16 13:57:19 -08:00
observer.h
operator_gpu_test.cc
operator_gradient.h Fix "missing return statement" warning (#22216) 2019-06-25 16:57:42 -07:00
operator_schema_test.cc
operator_schema.cc Replace c10::guts::stuff with std::stuff (#30915) 2019-12-16 13:57:19 -08:00
operator_schema.h Fix typos, via a Levenshtein-type corrector (#31523) 2020-01-17 16:03:19 -08:00
operator_test.cc Register gradient op with engine (#21205) 2019-05-31 18:48:47 -07:00
operator.cc Renaming IValue List functions (#32093) 2020-01-17 15:17:45 -08:00
operator.h Allow converting IValue to vector<string> (#34269) 2020-03-05 12:31:23 -08:00
parallel_net_test.cc fixup unit tests (#34105) 2020-03-03 10:33:21 -08:00
plan_executor.cc Replace c10::guts::stuff with std::stuff (#30915) 2019-12-16 13:57:19 -08:00
plan_executor.h
prof_dag_counters.cc Added engine information to the profiling result. (#20493) 2019-05-28 16:41:12 -07:00
prof_dag_counters.h Better handling of net errors in prof_dag counters (#17384) 2019-02-22 18:38:31 -08:00
qtensor_serialization.cc
qtensor_serialization.h
qtensor.cc
qtensor.h rest of uses for deprecation of dims() in Tensor (#16118) 2019-01-18 11:52:12 -08:00
scope_guard.h Fix typos, via a Levenshtein-type corrector (#31523) 2020-01-17 16:03:19 -08:00
static_tracepoint_elfx86.h Fix typos, via a Levenshtein-type corrector (#31523) 2020-01-17 16:03:19 -08:00
static_tracepoint.h
stats_test.cc
stats.cc
stats.h
storage.h Move files to/from c10/core and c10/util (#15316) 2019-01-10 16:22:22 -08:00
tensor_impl.h Remove Tensor.h, TensorMethods.h from src/core. (#27086) 2019-10-06 09:37:50 -07:00
tensor_int8.cc Fix include paths for typeid.h (#13689) 2018-11-14 18:04:09 -08:00
tensor_int8.h
tensor.cc Remove TensorImpl::is_variable, deprecate Tensor::is_variable (#29653) 2019-11-14 11:41:02 -08:00
tensor.h Rename TensorTypeId to DispatchKey (#32154) 2020-01-15 11:16:08 -08:00
test_utils.cc C++ C2/Glow operator unittest 2020-01-17 12:13:34 -08:00
test_utils.h C++ C2/Glow operator unittest 2020-01-17 12:13:34 -08:00
timer_test.cc
timer.h
transform_test.cc fixup unit tests (#34105) 2020-03-03 10:33:21 -08:00
transform.cc
transform.h
types.cc Add support to serialize qtensor in JIT. (#23356) 2019-07-26 15:52:15 -07:00
types.h move GetDimFromOrderString to caffe2/core/types.h (#25671) 2019-09-05 04:32:04 -07:00
workspace_test.cc
workspace.cc
workspace.h Fix typos, via a Levenshtein-type corrector (#31523) 2020-01-17 16:03:19 -08:00