pytorch/torch
Igor Sugak 7940f2428f [torch/package_importer] add compatibility name mapping (#134376)
Summary:
This enables patching extern modules to provide compatibility with serialized code depending on different versions of those extern modules.

The main motivation is to enable Numpy upgrade. In the recent release many alias to builtin types were deprecated and removed [1]. This breaks loading pickled modules that reference the removed aliases. While the proper solution is to re-generate pickled modules, it's not always feasible.

This proposes a way to define mapping with a new type, for a module member. It is only set if it's not present in the loaded module, thus removes the need to check for exact versions.

https://numpy.org/doc/stable/release/1.20.0-notes.html#using-the-aliases-of-builtin-types-like-np-int-is-deprecated

Differential Revision: D61556888

Pull Request resolved: https://github.com/pytorch/pytorch/pull/134376
Approved by: https://github.com/SherlockNoMad
2024-08-25 19:34:46 +00:00
..
_awaits [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
_C add relevant function (#133946) 2024-08-21 23:04:59 +00:00
_C_flatbuffer
_custom_op Tighten torch.library.infer_schema input types (#130705) 2024-07-29 16:01:19 +00:00
_decomp Register nll_loss2d decompositions for core aten (#133534) 2024-08-19 18:26:48 +00:00
_dispatch [BE][Easy][15/19] enforce style for empty lines in import segments in torch/_d*/ (#129767) 2024-07-31 21:18:11 +00:00
_dynamo [dynamo][guards] De-dupe DUPLICATE_INPUT guard (#134354) 2024-08-24 15:17:56 +00:00
_export line by line logging (#134298) 2024-08-25 02:57:11 +00:00
_functorch [aotd] Support HOP effects in backward (#132638) 2024-08-23 15:30:58 +00:00
_higher_order_ops [hop] ban creating hop by directly instantiating HigherOrderOperator. (#133645) 2024-08-23 17:28:02 +00:00
_inductor [inductor]Let output or input_as_strided match exact strides (#130956) 2024-08-24 17:04:05 +00:00
_lazy Add None return type to init (#132335) 2024-08-01 15:26:45 +00:00
_library [HOP] support generating schema for hop (#133521) 2024-08-21 17:34:21 +00:00
_logging line by line logging (#134298) 2024-08-25 02:57:11 +00:00
_numpy [test/torch_np] Fix usages of deprecated NumPy 2.0 APIs in numpy_tests (#131909) 2024-08-05 16:21:08 +00:00
_prims [hop] ban creating hop by directly instantiating HigherOrderOperator. (#133645) 2024-08-23 17:28:02 +00:00
_prims_common [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_refs Use integer divison in arange length calculation when start/end/step are integral (#134296) 2024-08-24 21:09:28 +00:00
_strobelight [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_subclasses [traced-graph][sparse] enable to_dense() for compressed (#133371) 2024-08-24 20:33:23 +00:00
_vendor
amp Revert "[MPS] Add support for autocast in MPS (#99272)" 2024-08-05 19:59:04 +00:00
ao [export][training ir migration] Fix getitem not exist (#134259) 2024-08-22 22:00:14 +00:00
autograd [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
backends [sparse] Add cuSPARSELt as a backend (#128534) 2024-08-21 22:06:07 +00:00
compiler [dynamo] ensure polyfill function has the same signature as the original function in substitute_in_graph (#133813) 2024-08-22 16:38:06 +00:00
contrib
cpu [BE][Easy][17/19] enforce style for empty lines in import segments in torch/[a-c]*/ and torch/[e-n]*/ (#129769) 2024-08-04 10:24:09 +00:00
csrc [Distributed/Profiler] Fix input/output dimension overflow (#134360) 2024-08-25 16:25:56 +00:00
cuda Fix partially initialized module error (#134019) 2024-08-20 22:20:02 +00:00
distributed Revert "[dtensor][MTPG] make sharding prop lru cache not shared among threads (#134294)" 2024-08-25 11:16:04 +00:00
distributions Add correct typing annotations to rsample() for all distributions (#133516) 2024-08-18 20:31:54 +00:00
export line by line logging (#134298) 2024-08-25 02:57:11 +00:00
fft
func
futures
fx Replace sympy Min/Max with reimplementations (#133319) 2024-08-25 05:05:59 +00:00
jit [BE]: Update ruff to 0.6.0 (#133609) 2024-08-16 14:11:01 +00:00
legacy
lib
linalg SparseCsrCUDA: cuDSS backend for linalg.solve (#129856) 2024-08-22 07:57:30 +00:00
masked Revert "[BE]: Update Typeguard to TypeIs for better type inference (#133814)" 2024-08-21 16:13:34 +00:00
monitor [pytorch][counters] Pybind for WaitCounter (#132357) 2024-08-02 16:08:10 +00:00
mps [BE][Easy][17/19] enforce style for empty lines in import segments in torch/[a-c]*/ and torch/[e-n]*/ (#129769) 2024-08-04 10:24:09 +00:00
mtia Improve error msg on _lazy_init() error (#134159) 2024-08-22 15:10:50 +00:00
multiprocessing Allow mp.start_processes to create processes in parallel (#133707) 2024-08-23 17:11:20 +00:00
nested [BE] typing for decorators - fx/_compatibility (part 1) (#134202) 2024-08-22 17:07:33 +00:00
nn [Inductor][FlexAttention] Fix IS_DIVISIBLE bug and add unit tests (#134055) 2024-08-23 16:11:09 +00:00
onnx [ONNX] Update exporter logic (#134304) 2024-08-24 00:49:54 +00:00
optim Avoid Host & Device Sync In LR Scheduler (#133663) 2024-08-22 03:52:43 +00:00
package [torch/package_importer] add compatibility name mapping (#134376) 2024-08-25 19:34:46 +00:00
profiler [Profiler/CPU] Add Test for Dynamic Activity Toggling [4/n] (#134149) 2024-08-23 01:13:42 +00:00
quantization [BE][Easy][19/19] enforce style for empty lines in import segments in torch/[o-z]*/ (#129771) 2024-08-01 17:07:14 +00:00
signal Revert "[BE] typing for decorators - signal/windows/windows (#131582)" 2024-07-28 03:29:31 +00:00
sparse [sparse] Update cuSPARSELt to v0.6.2 (#134022) 2024-08-23 19:34:53 +00:00
special
testing [inductor] fix munge_exc not support windows path (#134348) 2024-08-24 05:51:35 +00:00
utils Replace sympy Min/Max with reimplementations (#133319) 2024-08-25 05:05:59 +00:00
xpu [BE][Easy][19/19] enforce style for empty lines in import segments in torch/[o-z]*/ (#129771) 2024-08-01 17:07:14 +00:00
__config__.py
__future__.py
__init__.py Revert "[BE]: Update Typeguard to TypeIs for better type inference (#133814)" 2024-08-21 16:13:34 +00:00
_appdirs.py [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_classes.py Add None return type to init (#132335) 2024-08-01 15:26:45 +00:00
_compile.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_custom_ops.py [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_deploy.py
_guards.py [fbode-testing][dynamo][reland][inline-inbuilt-nn-modules] Mark attri… (#134136) 2024-08-22 17:54:58 +00:00
_jit_internal.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_linalg_utils.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_lobpcg.py [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
_lowrank.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_meta_registrations.py Added meta registration for _fused_adamw_ (#133728) 2024-08-17 00:28:31 +00:00
_namedtensor_internals.py [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_ops.py [hop] ban creating hop by directly instantiating HigherOrderOperator. (#133645) 2024-08-23 17:28:02 +00:00
_python_dispatcher.py Add None return type to init (#132335) 2024-08-01 15:26:45 +00:00
_size_docs.py
_sources.py
_storage_docs.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_streambase.py
_tensor_docs.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_tensor_str.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_tensor.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_torch_docs.py [docs] improve torch.stack example code to be reproducible (#133857) 2024-08-21 14:07:02 +00:00
_utils_internal.py [export][training ir migration] Fix getitem not exist (#134259) 2024-08-22 22:00:14 +00:00
_utils.py Revert "Deprecate torch._utils.is_compiling() and torch._dynamo.external_utils.is_compiling() (#127690)" 2024-08-07 00:05:20 +00:00
_VF.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_vmap_internals.py
_weights_only_unpickler.py Fix failures when default is flipped for weights_only (#127627) 2024-08-16 00:22:43 +00:00
abi-check.cpp
CMakeLists.txt [sparse] Add cuSPARSELt as a backend (#128534) 2024-08-21 22:06:07 +00:00
custom_class_detail.h Enable clang-tidy coverage on torch/*.h (#133422) 2024-08-15 18:52:08 +00:00
custom_class.h Enable clang-tidy coverage on torch/*.h (#133422) 2024-08-15 18:52:08 +00:00
extension.h
functional.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
hub.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
library.h Enable clang-tidy coverage on torch/*.h (#133422) 2024-08-15 18:52:08 +00:00
library.py [custom_ops] torch.library.{custom_op, register_kernel} disable Dynamo (#133125) 2024-08-12 18:29:18 +00:00
overrides.py Add new ops wrapped_linear_prepack and wrapped_quantized_linear_prepacked (#134232) 2024-08-23 04:54:26 +00:00
py.typed
quasirandom.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
random.py
README.txt
return_types.py
script.h
serialization.py Revert "[BE]: Update Typeguard to TypeIs for better type inference (#133814)" 2024-08-21 16:13:34 +00:00
storage.py Fix public API tests (#131386) 2024-07-30 18:42:54 +00:00
torch_version.py Add mypy typing to torch_version.py (#131447) 2024-07-23 17:31:07 +00:00
types.py FakeTensor cache SymInt support (#127596) 2024-07-21 19:26:38 +00:00
version.py.tpl

Note [TH abstraction violation]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TH/THC provide some hpp headers, which are proper C++ headers rather than
C headers.  These headers serve double duty as *internal implementation
detail* headers, whose contents should largely not be used by external
clients.

Ideally, we would not install these headers at all; instead, you should
use public functions (in headers like `THTensor.h`, NOT `THTensor.hpp`)
to manipulate these structs.  However, there are a few places
in torch/csrc where we violate this abstraction.  They are marked with
a pointer to this note.  Each of those sites will have to be refactored
when we refactor the guts of THTensor and related structures.