OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-08 07:39:33 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
77,596 Commits 8,304 Branches 150 Tags 1.5 GiB
7af38eb98b
Commit Graph

12 Commits

Author SHA1 Message Date
Aaron Orenstein
57536286e2 Flip default value for mypy disallow_untyped_defs [10/11] (#127847) 
See #127836 for details.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/127847
Approved by: https://github.com/oulgen
ghstack dependencies: #127842, #127843, #127844, #127845, #127846
 2024-06-08 18:50:06 +00:00
suo
686a3e0bf0 [pytorch][PR] introduce WeakHashRef (#115216) 
We would like weak dictionaries that have `torch.ScriptObject` keys. Similar to tensors, we need to override the behavior of the ref to dot he right thing under comparison.

This change also makes it so that WeakIdKeyDictionary works with a pluggable ref_type.

Differential Revision: [D51828205](https://our.internmc.facebook.com/intern/diff/D51828205/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/115216
Approved by: https://github.com/albanD
 2023-12-07 17:48:11 +00:00
zabboud
7f9fafed53 Resolve docstring errors in throughput_benchmark.py, weak.py, _traceback.py, file_baton.py, _contextlib.py, _device.py, cpp_backtrace.py, bundled_inputs.py, run_cpu.py, hooks.py, mobile_optimizer.py, _freeze.py, __init__.py, mkldnn.py, dlpack.py (#113311) 
Fixes #112633

Fixed errors relating to pydocstyle in the following files. The remaining errors are not covered in this issue. `torch/utils/dlpack.py` was not modified as the errors are relating to the function signature in the first line in the docstring which must be maintained as is for proper Sphinx interpretation.

```python
def from_dlpack(ext_tensor: Any) -> 'torch.Tensor':
    """from_dlpack(ext_tensor) -> Tensor
         .....
    """
```

pydocstyle torch/utils/_contextlib.py --count
before: 4
after: 0

pydocstyle torch/backends/mps/__init__.py --count
before: 8
after: 1

**remaining errors**
```
torch/backends/mps/__init__.py:1 at module level:
        D104: Missing docstring in public package
```

pydocstyle torch/backends/xeon/run_cpu.py --count
before: 13
after: 1

**remaining errors**
```
torch/backends/xeon/run_cpu.py:864 in public function `main`:
        D103: Missing docstring in public function
```

pydocstyle torch/backends/cpu/__init__.py --count
before: 2
after: 1

**remaining errors**
```
torch/backends/cpu/__init__.py:1 at module level:
        D104: Missing docstring in public package
```

pydocstyle torch/utils/cpp_backtrace.py --count
before: 4
after: 1

**remaining errors**
```
torch/utils/cpp_backtrace.py:1 at module level:
        D100: Missing docstring in public module
```

pydocstyle torch/utils/bundled_inputs.py --count
before: 8
after: 1

**remaining errors**
```
torch/utils/bundled_inputs.py:1 at module level:
        D100: Missing docstring in public module
```

pydocstyle torch/utils/file_baton.py --count
before: 8
after: 1

**remaining errors**
```
torch/utils/file_baton.py:1 at module level:
        D100: Missing docstring in public module
```

pydocstyle torch/utils/mobile_optimizer.py --count
before: 6
after: 1

**remaining errors**
```
torch/utils/mobile_optimizer.py:8 in public class `LintCode`:
        D101: Missing docstring in public class
```

pydocstyle torch/backends/opt_einsum/__init__.py --count
before: 7
after: 5

**remaining errors**
```
torch/backends/opt_einsum/__init__.py:1 at module level:
        D104: Missing docstring in public package
torch/backends/opt_einsum/__init__.py:67 in public function `set_flags`:
        D103: Missing docstring in public function
torch/backends/opt_einsum/__init__.py:77 in public function `flags`:
        D103: Missing docstring in public function
torch/backends/opt_einsum/__init__.py:93 in public class `OptEinsumModule`:
        D101: Missing docstring in public class
torch/backends/opt_einsum/__init__.py:94 in public method `__init__`:
        D107: Missing docstring in __init__
```

pydocstyle torch/utils/_device.py --count
before:  9
after: 6

**remaining errors**
```
torch/utils/_device.py:58 in public class `DeviceContext`:
        D101: Missing docstring in public class
torch/utils/_device.py:59 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/_device.py:62 in public method `__enter__`:
        D105: Missing docstring in magic method
torch/utils/_device.py:68 in public method `__exit__`:
        D105: Missing docstring in magic method
torch/utils/_device.py:73 in public method `__torch_function__`:
        D105: Missing docstring in magic method
torch/utils/_device.py:80 in public function `device_decorator`:
        D103: Missing docstring in public function

```

pydocstyle torch/utils/_freeze.py --count
before: 15
after: 7

**remaining errors**
```
torch/utils/_freeze.py:77 in public function `indent_msg`:
        D103: Missing docstring in public function
torch/utils/_freeze.py:89 in public class `FrozenModule`:
        D101: Missing docstring in public class
torch/utils/_freeze.py:100 in public class `Freezer`:
        D101: Missing docstring in public class
torch/utils/_freeze.py:101 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/_freeze.py:106 in public method `msg`:
        D102: Missing docstring in public method
torch/utils/_freeze.py:185 in public method `get_module_qualname`:
        D102: Missing docstring in public method
torch/utils/_freeze.py:206 in public method `compile_string`:
        D102: Missing docstring in public method

```

pydocstyle torch/utils/throughput_benchmark.py --count
before: 25
after: 8
**remaining errors**
```
torch/utils/throughput_benchmark.py:1 at module level:
        D100: Missing docstring in public module
torch/utils/throughput_benchmark.py:27 in public class `ExecutionStats`:
        D101: Missing docstring in public class
torch/utils/throughput_benchmark.py:28 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/throughput_benchmark.py:33 in public method `latency_avg_ms`:
        D102: Missing docstring in public method
torch/utils/throughput_benchmark.py:37 in public method `num_iters`:
        D102: Missing docstring in public method
torch/utils/throughput_benchmark.py:46 in public method `total_time_seconds`:
        D102: Missing docstring in public method
torch/utils/throughput_benchmark.py:50 in public method `__str__`:
        D105: Missing docstring in magic method
torch/utils/throughput_benchmark.py:94 in public method `__init__`:
        D107: Missing docstring in __init__

```

pydocstyle torch/utils/hooks.py --count

before: 14
after: 11

**remaining errors**
```
torch/utils/hooks.py:1 at module level:
        D100: Missing docstring in public module
torch/utils/hooks.py:23 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/hooks.py:34 in public method `remove`:
        D102: Missing docstring in public method
torch/utils/hooks.py:44 in public method `__getstate__`:
        D105: Missing docstring in magic method
torch/utils/hooks.py:50 in public method `__setstate__`:
        D105: Missing docstring in magic method
torch/utils/hooks.py:64 in public method `__enter__`:
        D105: Missing docstring in magic method
torch/utils/hooks.py:67 in public method `__exit__`:
        D105: Missing docstring in magic method
torch/utils/hooks.py:82 in public function `warn_if_has_hooks`:
        D103: Missing docstring in public function
torch/utils/hooks.py:103 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/hooks.py:188 in public method `setup_input_hook`:
        D102: Missing docstring in public method
torch/utils/hooks.py:197 in public method `setup_output_hook`:
        D102: Missing docstring in public method
```

pydocstyle torch/utils/_traceback.py --count
before: 19
after: 14

**remaining errors**
```
torch/utils/_traceback.py:47 in public function `report_compile_source_on_error`:
        D103: Missing docstring in public function
torch/utils/_traceback.py:160 in public class `CapturedTraceback`:
        D101: Missing docstring in public class
torch/utils/_traceback.py:163 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/_traceback.py:167 in public method `cleanup`:
        D102: Missing docstring in public method
torch/utils/_traceback.py:170 in public method `summary`:
        D102: Missing docstring in public method
torch/utils/_traceback.py:182 in public method `__getstate__`:
        D105: Missing docstring in magic method
torch/utils/_traceback.py:190 in public method `extract`:
        D205: 1 blank line required between summary line and description (found 0)
torch/utils/_traceback.py:190 in public method `extract`:
        D400: First line should end with a period (not 't')
torch/utils/_traceback.py:213 in public method `format`:
        D205: 1 blank line required between summary line and description (found 0)
torch/utils/_traceback.py:213 in public method `format`:
        D400: First line should end with a period (not 'f')
torch/utils/_traceback.py:213 in public method `format`:
        D401: First line should be in imperative mood (perhaps 'Format', not 'Formats')
torch/utils/_traceback.py:224 in public method `format_all`:
        D200: One-line docstring should fit on one line with quotes (found 3)
torch/utils/_traceback.py:247 in private function `_extract_symbolized_tb`:
        D205: 1 blank line required between summary line and description (found 0)
torch/utils/_traceback.py:247 in private function `_extract_symbolized_tb`:
        D400: First line should end with a period (not 'f')
```

pydocstyle torch/utils/mkldnn.py --count
before: 28
after: 26

**remaining errors**
```
torch/utils/mkldnn.py:1 at module level:
        D100: Missing docstring in public module
torch/utils/mkldnn.py:4 in public class `MkldnnLinear`:
        D101: Missing docstring in public class
torch/utils/mkldnn.py:5 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/mkldnn.py:19 in public method `__getstate__`:
        D105: Missing docstring in magic method
torch/utils/mkldnn.py:23 in public method `__setstate__`:
        D105: Missing docstring in magic method
torch/utils/mkldnn.py:29 in public method `forward`:
        D102: Missing docstring in public method
torch/utils/mkldnn.py:75 in public class `MkldnnConv1d`:
        D101: Missing docstring in public class
torch/utils/mkldnn.py:76 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/mkldnn.py:82 in public method `__setstate__`:
        D105: Missing docstring in magic method
torch/utils/mkldnn.py:88 in public class `MkldnnConv2d`:
        D101: Missing docstring in public class
torch/utils/mkldnn.py:89 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/mkldnn.py:100 in public method `__setstate__`:
        D105: Missing docstring in magic method
torch/utils/mkldnn.py:110 in public class `MkldnnConv3d`:
        D101: Missing docstring in public class
torch/utils/mkldnn.py:111 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/mkldnn.py:122 in public method `__setstate__`:
        D105: Missing docstring in magic method
torch/utils/mkldnn.py:133 in public class `MkldnnBatchNorm`:
        D101: Missing docstring in public class
torch/utils/mkldnn.py:136 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/mkldnn.py:155 in public method `__getstate__`:
        D105: Missing docstring in magic method
torch/utils/mkldnn.py:163 in public method `__setstate__`:
        D105: Missing docstring in magic method
torch/utils/mkldnn.py:171 in public method `forward`:
        D102: Missing docstring in public method
torch/utils/mkldnn.py:184 in public class `MkldnnPrelu`:
        D101: Missing docstring in public class
torch/utils/mkldnn.py:185 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/mkldnn.py:190 in public method `__getstate__`:
        D105: Missing docstring in magic method
torch/utils/mkldnn.py:194 in public method `__setstate__`:
        D105: Missing docstring in magic method
torch/utils/mkldnn.py:199 in public method `forward`:
        D102: Missing docstring in public method
torch/utils/mkldnn.py:205 in public function `to_mkldnn`:
        D103: Missing docstring in public function
```

pydocstyle torch/utils/weak.py --count
before: 32
after: 30

**remaining errors**
```
torch/utils/weak.py:1 at module level:
        D100: Missing docstring in public module
torch/utils/weak.py:42 in public class `WeakIdRef`:
        D101: Missing docstring in public class
torch/utils/weak.py:45 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/weak.py:54 in public method `__call__`:
        D102: Missing docstring in public method
torch/utils/weak.py:61 in public method `__hash__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:64 in public method `__eq__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:84 in public class `WeakIdKeyDictionary`:
        D101: Missing docstring in public class
torch/utils/weak.py:87 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/weak.py:131 in public method `__delitem__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:135 in public method `__getitem__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:138 in public method `__len__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:145 in public method `__repr__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:148 in public method `__setitem__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:151 in public method `copy`:
        D102: Missing docstring in public method
torch/utils/weak.py:162 in public method `__deepcopy__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:172 in public method `get`:
        D102: Missing docstring in public method
torch/utils/weak.py:175 in public method `__contains__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:182 in public method `items`:
        D102: Missing docstring in public method
torch/utils/weak.py:189 in public method `keys`:
        D102: Missing docstring in public method
torch/utils/weak.py:198 in public method `values`:
        D102: Missing docstring in public method
torch/utils/weak.py:216 in public method `popitem`:
        D102: Missing docstring in public method
torch/utils/weak.py:224 in public method `pop`:
        D102: Missing docstring in public method
torch/utils/weak.py:228 in public method `setdefault`:
        D102: Missing docstring in public method
torch/utils/weak.py:231 in public method `update`:
        D102: Missing docstring in public method
torch/utils/weak.py:241 in public method `__ior__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:245 in public method `__or__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:252 in public method `__ror__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:262 in public method `__eq__`:
        D105: Missing docstring in magic method
torch/utils/weak.py:276 in public method `__init__`:
        D107: Missing docstring in __init__
torch/utils/weak.py:280 in public method `__call__`:
        D102: Missing docstring in public method

```

@mikaylagawarecki @jbschlosser @svekars
Pull Request resolved: https://github.com/pytorch/pytorch/pull/113311
Approved by: https://github.com/ezyang
 2023-11-15 17:40:04 +00:00
Justin Chu
abc1cadddb [BE] Enable ruff's UP rules and autoformat utils/ (#105424) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105424
Approved by: https://github.com/ezyang, https://github.com/malfet
 2023-07-18 20:17:25 +00:00
Nikita Shulga
5837e95d30 [Reland] Update mypy to 1.4.1 (#105227) 
This PR re-lands
- [Typing] Fix PEP 484 Violation (#105022)
- Update mypy to 1.4.1 (#91983)

That were reverted due to the conflict with internal source repo.

Mostly fixes for PEP-484 violation (i.e. when default arg is set to None, but type is not annotated as optional)
Plus few real fixes:
  - Add missing `_get_upgraders_entry_map` to `torch/_C/__init__.pyi`
  - Add missing return statement to `torch._export. deserialize_graph`
  - Fix error message in `torch.ao.ns.fx.weight_utils.get_lstm_mod_weights`
  - Add assert it `torch/optim/optimizer.py` that Optional list is not None
TODO (in followup PR):
  - Fix erroneous `isinstance` check in `torch/ao/quantization/_pt2e/qat_utils.py`

Unrelated, to bypass CI failures due to the gcc9 dependency update in Ubuntu-18.04:
- Add hack to squash older libstdc++ from conda environment in favor one from OS to `.ci/docker/install_conda.sh`
- Update bazel cuda builds to focal, as with libstdc++-6.0.32 bazel builds loose the ability to catch exceptions (probably because they link with cupti statically, but I could not found where it is done)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105227
Approved by: https://github.com/atalman, https://github.com/albanD, https://github.com/Skylion007
 2023-07-15 20:30:20 +00:00
PyTorch MergeBot
15fd1ea118 Revert "[Reland] Update mypy to 1.4.1 (#105227)" 
This reverts commit c9c4f8efc3.

Reverted https://github.com/pytorch/pytorch/pull/105227 on behalf of https://github.com/atalman due to trying to mitigate ci sev #105248 ([comment](https://github.com/pytorch/pytorch/pull/105227#issuecomment-1636510935))
 2023-07-14 22:28:35 +00:00
Nikita Shulga
c9c4f8efc3 [Reland] Update mypy to 1.4.1 (#105227) 
This PR re-lands
- [Typing] Fix PEP 484 Violation (#105022)
- Update mypy to 1.4.1 (#91983)

That were reverted due to the conflict with internal source repo.

Mostly fixes for PEP-484 violation (i.e. when default arg is set to None, but type is not annotated as optional)
Plus few real fixes:
  - Add missing `_get_upgraders_entry_map` to `torch/_C/__init__.pyi`
  - Add missing return statement to `torch._export. deserialize_graph`
  - Fix error message in `torch.ao.ns.fx.weight_utils.get_lstm_mod_weights`
  - Add assert it `torch/optim/optimizer.py` that Optional list is not None
TODO (in followup PR):
  - Fix erroneous `isinstance` check in `torch/ao/quantization/_pt2e/qat_utils.py`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105227
Approved by: https://github.com/atalman, https://github.com/albanD, https://github.com/Skylion007
 2023-07-14 20:45:12 +00:00
PyTorch MergeBot
3c5a494d7a Revert "Update mypy to 1.4.1 (#91983)" 
This reverts commit 634659e262.

Reverted https://github.com/pytorch/pytorch/pull/91983 on behalf of https://github.com/malfet due to It's dependent change was reverted, so reverting this one as well, to keep CI clean ([comment](https://github.com/pytorch/pytorch/pull/91983#issuecomment-1636059709))
 2023-07-14 15:59:16 +00:00
Nikita Shulga
634659e262 Update mypy to 1.4.1 (#91983) 
Mostly fixes for PEP-484 violation (i.e. when default arg is set to None, but type is not annotated as optional)
Plus few real fixes:
  - Add missing `_get_upgraders_entry_map` to `torch/_C/__init__.pyi`
  - Add missing return statement to `torch._export. deserialize_graph`
  - Fix error message in `torch.ao.ns.fx.weight_utils.get_lstm_mod_weights`
  -
TODO (in followup PR):
  - Fix erroneous `isinstance` check in `torch/ao/quantization/_pt2e/qat_utils.py`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/91983
Approved by: https://github.com/kit1980, https://github.com/ZainRizvi, https://github.com/huydhn, https://github.com/thiagocrepaldi, https://github.com/aaronenyeshi
 2023-07-13 16:30:36 +00:00
Elias Ellison
571f96bf59 cudagraph trees (#89146) 
CUDA Graph Trees

Design doc: https://docs.google.com/document/d/1ZrxLGWz7T45MSX6gPsL6Ln4t0eZCSfWewtJ_qLd_D0E/edit

Not currently implemented :

- Right now, we are using weak tensor refs from outputs to check if a tensor has dies. This doesn't work because a) aliasing, and b) aot_autograd detaches tensors (see note [Detaching saved tensors in AOTAutograd]). Would need either https://github.com/pytorch/pytorch/issues/91395 to land to use storage weak refs or manually add a deleter fn that does what I want. This is doable but theres some interactions with the caching allocator checkpointing so saving for a stacked pr.

- Reclaiming memory from the inputs during model recording. This isn't terribly difficult but deferring to another PR. You would need to write over the input memory during warmup, and therefore copy the inputs to cpu. Saving for a stacked pr.

- Warning on overwriting previous generation outputs. and handling nested torch.compile() calls in generation tracking

Differential Revision: [D43999887](https://our.internmc.facebook.com/intern/diff/D43999887)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/89146
Approved by: https://github.com/ezyang
 2023-03-17 02:47:03 +00:00
Edward Z. Yang
0b22f5ae9f Deeply rework WeakIdKeyDictionary (#90825) 
In the prior patch, I just YOLOed a mutable mapping implementation.
Many edge cases were not handled correctly.  In this PR, I just
copy paste the WeakKeyDictionary from CPython and the hacked it up
to use WeakIdRef instead of weakref.ref.  You can see each line
I changed with the comment CHANGED; there aren't many.

Being exactly API compatible with WeakKeyDictionary means I can also
rob all of the tests from CPython, which I also did for
test/test_weak.py

How to review?  You could either try taking the delta from CPython
(recommended), or review everything from scratch (not recommended).
Can post diff representing delta on request.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90825
Approved by: https://github.com/albanD
 2022-12-15 08:43:08 +00:00
Edward Z. Yang
54563e6288 Don't put tracing state on Tensor (#90628) 
Fixes https://github.com/pytorch/pytorch/issues/89626

Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90628
Approved by: https://github.com/voznesenskym
 2022-12-15 08:43:08 +00:00