pytorch/torch
Wanchao Liang 2ee6b97464 [dtensor] move DTensor to public namespace (#133113)
Moving DTensor to be in the public namespace, to formally add the
documentation page that includes all the public APIs. This includes:

* many path renames and path import fixes
* a dedicated doc page without too much content yet (adding in the next
  PRs)
* To preserve the BC for users still using the `torch.distributed._tensor`,
  I added a shim script to redirect old path calls to the new module

The BC preserving is evidented by the fact that all DTensor tests are still
working without changing the public imports. So it's safe to land the
changes

Pull Request resolved: https://github.com/pytorch/pytorch/pull/133113
Approved by: https://github.com/XilunWu
ghstack dependencies: #133305, #133306
2024-08-17 05:09:52 +00:00
..
_awaits [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
_C [sigmoid] Support custom obj deserialization. (#133463) 2024-08-15 17:58:44 +00:00
_C_flatbuffer
_custom_op Tighten torch.library.infer_schema input types (#130705) 2024-07-29 16:01:19 +00:00
_decomp Add a private _safe_softmax (#131060) 2024-08-08 23:09:38 +00:00
_dispatch [BE][Easy][15/19] enforce style for empty lines in import segments in torch/_d*/ (#129767) 2024-07-31 21:18:11 +00:00
_dynamo [dtensor] move DTensor to public namespace (#133113) 2024-08-17 05:09:52 +00:00
_export [ts_converter][reland] Add support for LinearOpContext and Conv2dOpContext in quantization pass (#133622) 2024-08-16 01:55:45 +00:00
_functorch [compiled autograd] use same graph node names as AOTDispatcher (#133148) 2024-08-17 00:46:52 +00:00
_higher_order_ops Improvements for associative_scan - Reverse feature (#133011) 2024-08-16 23:06:31 +00:00
_inductor Add a fudge factor to ephemeral NCCL timeout increase (#133722) 2024-08-17 03:08:40 +00:00
_lazy Add None return type to init (#132335) 2024-08-01 15:26:45 +00:00
_library [custom_ops] torch.library.{custom_op, register_kernel} disable Dynamo (#133125) 2024-08-12 18:29:18 +00:00
_logging Add ChromiumEventLogger, log FXGraphCache and AOTAutogradCache (#132864) 2024-08-10 01:15:53 +00:00
_numpy [test/torch_np] Fix usages of deprecated NumPy 2.0 APIs in numpy_tests (#131909) 2024-08-05 16:21:08 +00:00
_prims [BE] Simplify code interacting with get_proxy_mode/enable_tracing (#132675) 2024-08-08 12:03:00 +00:00
_prims_common [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_refs Don't attempt to compute hints for unbacked expressions (#132060) 2024-08-02 16:39:14 +00:00
_strobelight [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_subclasses Reland "Construct NJT without graph breaks" (#133196) 2024-08-14 01:11:13 +00:00
_vendor
amp Revert "[MPS] Add support for autocast in MPS (#99272)" 2024-08-05 19:59:04 +00:00
ao [export] fix test for training ir migration (#133587) 2024-08-15 23:55:09 +00:00
autograd [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
backends [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
compiler
contrib
cpu [BE][Easy][17/19] enforce style for empty lines in import segments in torch/[a-c]*/ and torch/[e-n]*/ (#129769) 2024-08-04 10:24:09 +00:00
csrc [compiled autograd] move non-hot path logs into default logger (#133541) 2024-08-17 00:46:52 +00:00
cuda [Reland] Add wrappers for synchronous GPUDirect Storage APIs (#133489) 2024-08-15 17:11:52 +00:00
distributed [dtensor] move DTensor to public namespace (#133113) 2024-08-17 05:09:52 +00:00
distributions [BE][Easy][18/19] enforce style for empty lines in import segments in torch/d*/ (#129770) 2024-08-01 04:22:50 +00:00
export [easy] Fix type annotation for ExportedProgram.run_decompositions (#133720) 2024-08-16 22:11:42 +00:00
fft
func
futures
fx remove dead code for suggesting legacy dynamic shapes fixes (#133700) 2024-08-17 01:59:34 +00:00
jit [BE]: Update ruff to 0.6.0 (#133609) 2024-08-16 14:11:01 +00:00
legacy
lib
linalg fix a typo in the householder_product docs (#124279) 2024-08-15 09:34:17 +00:00
masked [BE][Easy][17/19] enforce style for empty lines in import segments in torch/[a-c]*/ and torch/[e-n]*/ (#129769) 2024-08-04 10:24:09 +00:00
monitor [pytorch][counters] Pybind for WaitCounter (#132357) 2024-08-02 16:08:10 +00:00
mps [BE][Easy][17/19] enforce style for empty lines in import segments in torch/[a-c]*/ and torch/[e-n]*/ (#129769) 2024-08-04 10:24:09 +00:00
mtia [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
multiprocessing [BE][Easy][17/19] enforce style for empty lines in import segments in torch/[a-c]*/ and torch/[e-n]*/ (#129769) 2024-08-04 10:24:09 +00:00
nested Reland "Construct NJT without graph breaks" (#133196) 2024-08-14 01:11:13 +00:00
nn Fix docs for L1Loss and MSELoss (#133501) 2024-08-15 18:56:55 +00:00
onnx [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
optim move param's device check to _init_group for fused (#131153) 2024-08-17 04:49:47 +00:00
package [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
profiler [Profiler/CPU] Add API for Dynamic Activity Toggling [3/n] (#133353) 2024-08-16 16:36:57 +00:00
quantization [BE][Easy][19/19] enforce style for empty lines in import segments in torch/[o-z]*/ (#129771) 2024-08-01 17:07:14 +00:00
signal Revert "[BE] typing for decorators - signal/windows/windows (#131582)" 2024-07-28 03:29:31 +00:00
sparse Add tests to bsr_dense_addmm_meta. Tune bsr_dense_addmm kernel for ViT shapes. (#132646) 2024-08-05 20:22:33 +00:00
special
testing [dtensor] move DTensor to public namespace (#133113) 2024-08-17 05:09:52 +00:00
utils Remove AMD restrictions on triton hashing (#133616) 2024-08-16 08:02:48 +00:00
xpu [BE][Easy][19/19] enforce style for empty lines in import segments in torch/[o-z]*/ (#129771) 2024-08-01 17:07:14 +00:00
__config__.py
__future__.py
__init__.py [Reland] Add wrappers for synchronous GPUDirect Storage APIs (#133489) 2024-08-15 17:11:52 +00:00
_appdirs.py [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_classes.py Add None return type to init (#132335) 2024-08-01 15:26:45 +00:00
_compile.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_custom_ops.py [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_deploy.py
_guards.py Revert #132806, #132736, #132539, #132487 (#133570) 2024-08-15 20:54:21 +00:00
_jit_internal.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_linalg_utils.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_lobpcg.py [BE][Easy] enable ruff rule PIE790: unnecessary pass statement (#133200) 2024-08-15 15:50:19 +00:00
_lowrank.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_meta_registrations.py Added meta registration for _fused_adamw_ (#133728) 2024-08-17 00:28:31 +00:00
_namedtensor_internals.py [BE][Easy][14/19] enforce style for empty lines in import segments in torch/_[a-c]*/ and torch/_[e-h]*/ and torch/_[j-z]*/ (#129765) 2024-07-31 10:42:50 +00:00
_ops.py Make the __module__ name of HOO to be always "torch.ops.higher_order" (#132775) 2024-08-08 16:55:09 +00:00
_python_dispatcher.py Add None return type to init (#132335) 2024-08-01 15:26:45 +00:00
_size_docs.py
_sources.py
_storage_docs.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_streambase.py
_tensor_docs.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_tensor_str.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_tensor.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_torch_docs.py Prevent an unnecessary device -> host copy for CuPy arrays when not explicitly setting a device in torch.as_tensor. (#132595) 2024-08-08 00:26:58 +00:00
_utils_internal.py Turn off remote caching in unit tests unless explicitly on (#133258) 2024-08-13 02:49:43 +00:00
_utils.py Revert "Deprecate torch._utils.is_compiling() and torch._dynamo.external_utils.is_compiling() (#127690)" 2024-08-07 00:05:20 +00:00
_VF.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
_vmap_internals.py
_weights_only_unpickler.py Fix failures when default is flipped for weights_only (#127627) 2024-08-16 00:22:43 +00:00
abi-check.cpp
CMakeLists.txt [Reland] Add wrappers for synchronous GPUDirect Storage APIs (#133489) 2024-08-15 17:11:52 +00:00
custom_class_detail.h Enable clang-tidy coverage on torch/*.h (#133422) 2024-08-15 18:52:08 +00:00
custom_class.h Enable clang-tidy coverage on torch/*.h (#133422) 2024-08-15 18:52:08 +00:00
extension.h
functional.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
hub.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
library.h Enable clang-tidy coverage on torch/*.h (#133422) 2024-08-15 18:52:08 +00:00
library.py [custom_ops] torch.library.{custom_op, register_kernel} disable Dynamo (#133125) 2024-08-12 18:29:18 +00:00
overrides.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
py.typed
quasirandom.py [BE] Format uncategorized Python files with ruff format (#132576) 2024-08-04 17:13:31 +00:00
random.py
README.txt
return_types.py
script.h
serialization.py Fix all RuntimeErrors during weights_only load from being erroneously reported with the weights_only message (#132349) 2024-08-16 00:22:43 +00:00
storage.py Fix public API tests (#131386) 2024-07-30 18:42:54 +00:00
torch_version.py
types.py
version.py.tpl

Note [TH abstraction violation]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TH/THC provide some hpp headers, which are proper C++ headers rather than
C headers.  These headers serve double duty as *internal implementation
detail* headers, whose contents should largely not be used by external
clients.

Ideally, we would not install these headers at all; instead, you should
use public functions (in headers like `THTensor.h`, NOT `THTensor.hpp`)
to manipulate these structs.  However, there are a few places
in torch/csrc where we violate this abstraction.  They are marked with
a pointer to this note.  Each of those sites will have to be refactored
when we refactor the guts of THTensor and related structures.