pytorch/torch
Jiashen Cao 10d2373abd Add a registry for GraphModuleSerializer (#126550)
This PR adds a registration function and a global registry for GraphModuleSerializer. After this PR, custom serialization methods can be done through registration instead of subclassing for ease of maintenance.

## Changes
- Add a test case where it injects custom op to test serialization.
- Add custom op handler
- Change allowed op for verifier
Co-authored-by: Zhengxu Chen <zhxchen17@outlook.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/126550
Approved by: https://github.com/zhxchen17
2024-05-29 03:12:48 +00:00
..
_awaits
_C [1/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort stdlib (#127122) 2024-05-25 08:25:50 +00:00
_C_flatbuffer
_custom_op
_decomp [decomp] Fix default values missing from inplace rrelu decomposition (#126978) 2024-05-26 23:49:40 +00:00
_dispatch
_dynamo [export] handle new roots & root swapping in derived dims suggested fixes (#125543) 2024-05-28 20:41:43 +00:00
_export Add a registry for GraphModuleSerializer (#126550) 2024-05-29 03:12:48 +00:00
_functorch Back out "Prevent partitioner from ever saving views (#126446)" (#127316) 2024-05-29 00:29:44 +00:00
_higher_order_ops [User-Written Triton] Handle the scf.for and scf.while case (#127065) 2024-05-24 21:01:13 +00:00
_inductor [inductor] Use symbolic_hint when bounding fallback size hint (#127262) 2024-05-28 22:51:45 +00:00
_lazy
_library Inductor respects strides for custom ops by default (#126986) 2024-05-24 11:11:18 +00:00
_logging [inductor] Add kernel_code logging artifact (#126631) 2024-05-21 23:12:42 +00:00
_numpy Remove removed ruff rule TRY200 (#126256) 2024-05-17 16:31:05 +00:00
_prims [custom_op] use new python custom ops API on prims ops (#124665) 2024-05-22 17:48:33 +00:00
_prims_common [1/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort stdlib (#127122) 2024-05-25 08:25:50 +00:00
_refs Add guard_size_oblivious to vector_norm (#126772) 2024-05-21 19:53:21 +00:00
_subclasses [1/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort stdlib (#127122) 2024-05-25 08:25:50 +00:00
_vendor
amp generalize custom_fwd&custom_bwd to be device-agnostic (#126531) 2024-05-25 06:48:16 +00:00
ao [5/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort torch (#127126) 2024-05-27 14:49:57 +00:00
autograd Remove MAX_STACK_ENTRY from _build_table (#126583) 2024-05-21 18:52:04 +00:00
backends Fix public binding to actually traverse modules (#126103) 2024-05-15 19:36:03 +00:00
compiler Fix broken docs (#124940) 2024-04-26 19:24:52 +00:00
contrib
cpu Deprecate device-specific GradScaler autocast API (#126527) 2024-05-25 06:41:34 +00:00
csrc [NCCL][CUDA] Optionally avoid rethrowing CUDA Errors in NCCL Watchdog (#126587) 2024-05-28 22:17:15 +00:00
cuda [AMD] Fix deprecated amdsmi api (#126962) 2024-05-26 20:11:23 +00:00
distributed [dtensor][debug] added c10d gather, reduce, scatter tracing to CommDebugMode (#127134) 2024-05-28 22:48:07 +00:00
distributions [BE]: Update ruff to 0.4.1 (#124549) 2024-04-21 14:06:23 +00:00
export [export] handle new roots & root swapping in derived dims suggested fixes (#125543) 2024-05-28 20:41:43 +00:00
fft
func
futures
fx [export] handle new roots & root swapping in derived dims suggested fixes (#125543) 2024-05-28 20:41:43 +00:00
jit [TorchScript] Expand TorchScript __init__ annotation warning (#127045) 2024-05-28 23:49:10 +00:00
legacy
lib [codemod] Fix missing field initializer in caffe2/torch/lib/libshm/core.cpp +2 (#125047) 2024-04-27 15:52:56 +00:00
linalg Fix global flake8 issues (#124771) 2024-04-26 15:35:53 +00:00
masked Fix global flake8 issues (#124771) 2024-04-26 15:35:53 +00:00
monitor
mps Conform torch.mps to device module interface (#124676) 2024-04-23 18:38:48 +00:00
mtia torch.mtia module for MTIA device backend (#123612) 2024-04-26 16:17:54 +00:00
multiprocessing [BE]: Update ruff to v0.4.4 (#125031) 2024-05-12 20:02:37 +00:00
nested [NT] Make NestedTensor register as having symbolic sizes/strides (#124687) 2024-05-13 16:50:25 +00:00
nn [1/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort stdlib (#127122) 2024-05-25 08:25:50 +00:00
onnx [1/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort stdlib (#127122) 2024-05-25 08:25:50 +00:00
optim Radam support the flag for "maximize" (#126765) 2024-05-27 06:34:50 +00:00
package [BE]: Try TCH autofixes on torch/ (#125536) 2024-05-05 23:13:59 +00:00
profiler [1/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort stdlib (#127122) 2024-05-25 08:25:50 +00:00
quantization
signal Fix global flake8 issues (#124771) 2024-04-26 15:35:53 +00:00
sparse Fix a bug in retrieving approximate bsr_dense_addmm kernel meta data (#124371) 2024-04-24 13:59:18 +00:00
special Fix global flake8 issues (#124771) 2024-04-26 15:35:53 +00:00
testing Revert "[Submodule] Remove deprecated USE_TBB option and TBB submodule (#127051)" 2024-05-29 03:02:35 +00:00
utils Revert "[Submodule] Remove deprecated USE_TBB option and TBB submodule (#127051)" 2024-05-29 03:02:35 +00:00
xpu Add some type annotations to python stream and event classes (#126171) 2024-05-15 04:58:07 +00:00
__config__.py
__future__.py
__init__.py Remove torch._constrain_as_value (#127103) 2024-05-24 22:49:46 +00:00
_appdirs.py
_classes.py
_compile.py
_custom_ops.py Replace torch.library.impl_abstract with torch.library.register_fake (#126606) 2024-05-19 13:21:39 +00:00
_deploy.py
_guards.py [BE]: Try TCH autofixes on torch/ (#125536) 2024-05-05 23:13:59 +00:00
_jit_internal.py [BE][Easy] Remove usage of deprecated ast.Str, ast.Ellipsis and ast.NameConstant (#125912) 2024-05-10 17:35:35 +00:00
_linalg_utils.py
_lobpcg.py
_lowrank.py
_meta_registrations.py Support dtype kwarg in _foreach_norm (#125665) 2024-05-22 20:27:50 +00:00
_namedtensor_internals.py
_ops.py Refresh OpOverloadPacket if a new OpOverload gets added (#126863) 2024-05-22 14:13:27 +00:00
_python_dispatcher.py
_size_docs.py
_sources.py
_storage_docs.py
_streambase.py Add some type annotations to python stream and event classes (#126171) 2024-05-15 04:58:07 +00:00
_tensor_docs.py
_tensor_str.py [small] guard_size_oblivious, skip check for meta (#127298) 2024-05-28 17:53:08 +00:00
_tensor.py Improve the clarity of the torch.Tensor.backward doc (#127201) 2024-05-28 19:25:51 +00:00
_torch_docs.py Add usage explanation in torch.dot ducment (#125908) 2024-05-22 20:33:12 +00:00
_utils_internal.py Log export result of torch.jit.trace to scuba (#126900) 2024-05-28 17:49:34 +00:00
_utils.py [1/N][Easy] fix typo for usort config in pyproject.toml (kown -> known): sort stdlib (#127122) 2024-05-25 08:25:50 +00:00
_VF.py
_vmap_internals.py
_weights_only_unpickler.py Allow tensor subclasses and add torch.serialization.add_safe_globals that allows users to allowlist classes for weights_only load (#124331) 2024-05-17 17:56:57 +00:00
abi-check.cpp
CMakeLists.txt [rfc] opentelemetry in pytorch (#122999) 2024-04-21 15:20:21 +00:00
custom_class_detail.h [codemod] c10:optional -> std::optional (#126135) 2024-05-14 19:35:51 +00:00
custom_class.h
extension.h
functional.py
hub.py [BE]: TRY002 - Ban raising vanilla exceptions (#124570) 2024-04-21 22:26:40 +00:00
library.h [Reland] Verify types in custom op schemas (#126861) 2024-05-23 19:53:52 +00:00
library.py [BE]: Update ruff to 0.4.5 (#126979) 2024-05-24 18:38:35 +00:00
overrides.py torch.mtia module for MTIA device backend (#123612) 2024-04-26 16:17:54 +00:00
py.typed
quasirandom.py Fix SobolEngine default dtype handling (#126781) 2024-05-22 01:55:48 +00:00
random.py [Doc] Update docstrings for torch/random.py (#125265) 2024-05-02 20:55:23 +00:00
README.txt
return_types.py
script.h
serialization.py Fix map_location for wrapper subclass and device tensors that go through numpy (#126728) 2024-05-24 16:39:30 +00:00
storage.py xpu: implement xpu serialization (#125530) 2024-05-16 20:22:17 +00:00
torch_version.py
types.py
version.py.tpl

Note [TH abstraction violation]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TH/THC provide some hpp headers, which are proper C++ headers rather than
C headers.  These headers serve double duty as *internal implementation
detail* headers, whose contents should largely not be used by external
clients.

Ideally, we would not install these headers at all; instead, you should
use public functions (in headers like `THTensor.h`, NOT `THTensor.hpp`)
to manipulate these structs.  However, there are a few places
in torch/csrc where we violate this abstraction.  They are marked with
a pointer to this note.  Each of those sites will have to be refactored
when we refactor the guts of THTensor and related structures.