pytorch/torch/csrc/jit/python
Lily Johnson 0dd90cceaf [package] track storages across lifetime of PackageExporter (#59735)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/59735

1. Fixes ABA storage identity problem during serialization for `torch.package` by keeping reference of serialized storages through lifetime of `PackageExporter` to prevent reuse of memory address. Achieved by extending logic used in solution to mobile's same issue.
2. Adds determinism to naming scheme of serialized storages in export code paths which utilize `tensor_cdata_naming_scheme`(introduced 2nd mapping in `StorageContext`, now maps `storage cdata ptr` -> `unique id`, `unique id` -> `c10::Storage`)
3. Additionally uses presence of a storage in the `StorageContext` instance as marker for if a storage has been serialized or not, removing the need to scan the `PythonStreamWriter` for presence of the storage's serialization file

Test Plan: Imported from OSS

Reviewed By: suo

Differential Revision: D29075276

Pulled By: Lilyjjo

fbshipit-source-id: 15a5c30b1de99c5bd7079388f2db9b6ece2eca12
2021-06-29 14:16:54 -07:00
..
init.cpp [package] track storages across lifetime of PackageExporter (#59735) 2021-06-29 14:16:54 -07:00
init.h
module_python.h
pybind_utils.cpp [torch] Remove migration warning for ScriptDict 2021-06-03 20:55:40 -07:00
pybind_utils.h Support default values on NamedTuple fields (#54682) 2021-06-26 15:18:21 -07:00
pybind.h Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_arg_flatten.cpp Replace all direct cdata access with THPVariable_Unpack (#55799) 2021-04-15 08:57:04 -07:00
python_arg_flatten.h Move torch/csrc/utils/hash.h to c10/util/hash.h. (#42503) 2020-08-29 17:47:00 -07:00
python_custom_class.cpp Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_custom_class.h
python_dict.cpp [JIT] Add torch._C.ScriptDict (#52659) 2021-05-27 10:25:30 -07:00
python_dict.h [JIT] Add torch._C.ScriptDict (#52659) 2021-05-27 10:25:30 -07:00
python_interpreter.cpp Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_ir.cpp Add handling of symbolic shapes (#55925) 2021-05-21 08:50:52 -07:00
python_ir.h Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_ivalue.h Make DataPtr extraction in CUDAFuture faster for Python values (#56918) 2021-05-06 01:12:53 -07:00
python_sugared_value.cpp Support default values on NamedTuple fields (#54682) 2021-06-26 15:18:21 -07:00
python_sugared_value.h Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_tracer.cpp [Usability] Capture argument names for traced functions and modules (#51775) 2021-02-10 18:28:08 -08:00
python_tracer.h [Usability] Capture argument names for traced functions and modules (#51775) 2021-02-10 18:28:08 -08:00
python_tree_views.cpp Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_tree_views.h
script_init.cpp [package] track storages across lifetime of PackageExporter (#59735) 2021-06-29 14:16:54 -07:00
script_init.h
update_graph_executor_opt.cpp Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
update_graph_executor_opt.h