OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00
Code Issues Actions 37 Packages Projects Releases Wiki Activity
53,040 Commits 8,304 Branches 150 Tags 1.5 GiB
bc19494814
Commit Graph

172 Commits

Author SHA1 Message Date
Michael Voznesensky
bc19494814 [Dynamo] Symbolic shape guards (#87570) 
**Introduces symbolic shape guards into dynamo.**

In this PR, we take the existing fake tensor infra and plumbing in dynamo and we start passing a shape_env around. This shape_env does not get plumbed down to middle layers / backend yet - it only collects expressions from frontend invocations at the moment. We then translate these expressions into guards at the point where we take other guards installed throughout dynamo - and add them to check_fn.

Part 1 of https://docs.google.com/document/d/1QJ-M4zfMkD-fjHIqW089RptjLl9EgozZGCceUbvmgfY/edit#

cc @jansel @lezcano @fdrocha @mlazos @soumith @yanboliang @penguinwu @anijain2305
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87570
Approved by: https://github.com/ezyang
 2022-10-25 21:15:40 +00:00
Horace He
569eebb43c Add get_guard_expr to symbolic_shapes which returns all guards in a single expression (#87665) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87665
Approved by: https://github.com/ezyang, https://github.com/voznesenskym
 2022-10-25 16:58:18 +00:00
Ryan Spring
9bb4926de0 Add xlogy and xlog1py references (#77712) 
* Add reference implementations for `xlogy` and `xlog1py`
 * Replace `_wrap_scalar` helper function with `scalar_tensor` prim
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77712
Approved by: https://github.com/mruberry
 2022-10-22 17:59:25 +00:00
albanD
9199f9188c Add inplace function testing to test_proxy_tensor (#87324) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87324
Approved by: https://github.com/ezyang
 2022-10-20 14:20:19 +00:00
albanD
254b681dc6 Convert torch.Size() argument to sym size in test_proxy_tensor (#87304) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87304
Approved by: https://github.com/ezyang
 2022-10-20 14:20:19 +00:00
Horace He
bd757b364c Ensure that symbolic variables incorporate fresh constraints before they're used (#87254) 
cc @jansel @lezcano @fdrocha
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87254
Approved by: https://github.com/jansel
 2022-10-20 00:37:40 +00:00
albanD
12b2f70a89 Symintify pad ops (#87046) 
Following comments below, we need to add support for `std::negate`/`std::min`/`std::max`/`operator-` for SymInt
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87046
Approved by: https://github.com/ezyang
 2022-10-19 21:43:08 +00:00
Horace He
5e23074f0d Fixed FakeTensor not calling CompositeImplicitAutograd decomps sometimes (#87252) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87252
Approved by: https://github.com/ezyang, https://github.com/bdhirsh
 2022-10-19 07:13:30 +00:00
albanD
1463013c85 autograd clone_obey_contract() symint support (#87044) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87044
Approved by: https://github.com/ezyang
 2022-10-17 13:14:12 +00:00
albanD
c21dcffc00 Very limited pow support (#87042) 
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87042
Approved by: https://github.com/ezyang
 2022-10-17 13:14:07 +00:00
albanD
3a4c0900c7 Reland 3 of Merge more symbolic meta kernels and symint changes from branch (#86795) 
Take 3
Contains:
- symintification of split*
- floor support on SymFloat
- pad_backward, gather, scatter meta
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86795
Approved by: https://github.com/z-a-f
 2022-10-17 02:09:40 +00:00
Horace He
2c1bc216b8 Fixed partitioner issue with getitem and made metadata a storage more consistent (#87012) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87012
Approved by: https://github.com/ngimel
 2022-10-15 17:58:55 +00:00
Brian Hirsh
34c86adec4 symintify all of derivatives.yaml (#86610) 
Big-bang PR to symintify **all** .sizes() calls in derivatives.yaml, which will be needed for symbolic tracing.

* with the exception of `split()`, which is tougher to land because it requires internal changes.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/86610
Approved by: https://github.com/albanD
 2022-10-14 20:15:48 +00:00
Horace He
b3b9786fdd Unified symbolic shape variables between AOTAutograd and Inductor (#86659) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86659
Approved by: https://github.com/wconstab
 2022-10-14 00:24:43 +00:00
Jason Ansel
c7c09722ad Move TorchDynamo into PyTorch core (#86461) 
Context:
https://github.com/pytorch/torchdynamo/issues/1588

This PR moves [TorchDynamo](https://github.com/pytorch/torchdynamo) and TorchInductor into PyTorch core.
- `torchdynamo` becomes `torch._dynamo`
- `torchinductor` becomes `torch._inductor`

This PR was generated by running `copy_to_core.sh` in https://github.com/pytorch/torchdynamo/pull/1538

Pull Request resolved: https://github.com/pytorch/pytorch/pull/86461
Approved by: https://github.com/voznesenskym
 2022-10-13 23:18:06 +00:00
Khushi Agrawal
77d29bcee2 [primTorch] special: ndtr, ndtri, log_ndtr, erfcx (#86077) 
- Adds prims and _refs for `erfcx` and `ndtri`.
- Adds _refs for `ndtr`, and `log_ndtr`.

cc @kshitij12345 @lezcano @mruberry
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86077
Approved by: https://github.com/mruberry
 2022-10-13 01:18:30 +00:00
albanD
66cab5245f Reland 2 min/max support for SymInt/Floats, finish as_strided/scatter/squeeze() backward symint support (#86797) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86797
Approved by: https://github.com/bdhirsh
 2022-10-13 00:31:19 +00:00
Khushi
2344135179 [primTorch] special: entr, expit (#86592) 
Add _refs for `entr` & `expit`.

cc @mruberry @kshitij12345!
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86592
Approved by: https://github.com/mruberry
 2022-10-12 07:00:40 +00:00
PyTorch MergeBot
2aa981ab74 Revert "Reland 2 of Merge more symbolic meta kernels and symint changes from branch (#86334) (#86488)" 
This reverts commit 978b46d7c9.

Reverted https://github.com/pytorch/pytorch/pull/86488 on behalf of https://github.com/osalpekar due to Broke executorch builds internally with the following message: RuntimeError: Missing out variant for functional op: aten::split.Tensor(Tensor(a -> *) self, SymInt split_size, int dim=0) -> Tensor(a)[] . Make sure you have loaded your custom_ops_generated_lib
 2022-10-11 23:39:50 +00:00
PyTorch MergeBot
811b8e012b Revert "min/max support for SymInt/Floats, finish as_strided/scatter/squeeze() backward symint support (#86643)" 
This reverts commit 86f914e996.

Reverted https://github.com/pytorch/pytorch/pull/86643 on behalf of https://github.com/osalpekar due to Need to revert this to cleanly revert https://github.com/pytorch/pytorch/pull/86488. This should be safe to re-land later
 2022-10-11 23:12:40 +00:00
albanD
86f914e996 min/max support for SymInt/Floats, finish as_strided/scatter/squeeze() backward symint support (#86643) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86643
Approved by: https://github.com/anjali411
 2022-10-11 17:37:30 +00:00
albanD
be8627827e More symintification of get/set item (#86605) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86605
Approved by: https://github.com/anjali411
 2022-10-11 12:00:40 +00:00
albanD
49c9b0a154 symintify einsum (#86603) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86603
Approved by: https://github.com/anjali411
 2022-10-11 12:00:35 +00:00
albanD
978b46d7c9 Reland 2 of Merge more symbolic meta kernels and symint changes from branch (#86334) (#86488) 
symintify split_with_sizes, dropout, fused_fake_obs_quant. meta for padding_2d ops

add meta_bernoulli_

meta kernel for at::gather

get pytorch_struct to pass: meta for scatter_add, fix backward

symintify split ops
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86488
Approved by: https://github.com/ezyang
 2022-10-10 15:54:28 +00:00
albanD
55663b7f81 Reland 3 of Symintify getitem and add the required helper functions (#86207) (#86487) 
Note that this might not cover every use of the function (we know it doesn't)
But this is enough to get few models passing.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86487
Approved by: https://github.com/ezyang
 2022-10-10 15:54:28 +00:00
anjali411
c89d286af6 symintify unbind_backward and tensor_split (#86357) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86357
Approved by: https://github.com/albanD
 2022-10-09 16:25:55 +00:00
Edward Z. Yang
33f0e98a49 Re-land*4 "SymIntify cat and narrow" (#86468) 
This re-lands https://github.com/pytorch/pytorch/pull/86289 but with more wrappers.

Contains implicit inclusion of <ATen/native/NonSymbolicBC.h> in internal usage.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86468
Approved by: https://github.com/albanD
 2022-10-08 07:17:37 +00:00
PyTorch MergeBot
65b408074f Revert "Relandx3 "SymIntify cat and narrow" (#86289)" 
This reverts commit a00f8489df.

Reverted https://github.com/pytorch/pytorch/pull/86289 on behalf of https://github.com/malfet due to @seemether  unlanded the rest of the stack and it will fail intern import anyway
 2022-10-07 16:29:27 +00:00
PyTorch MergeBot
5b69b87d5a Revert "Symintify getitem and add the required helper functions (#86207)" 
This reverts commit fd5085c445.

Reverted https://github.com/pytorch/pytorch/pull/86207 on behalf of https://github.com/seemethere due to  Fails internal tests, see: https://www.internalfb.com/intern/sandcastle/job/22517998926071860/insights
 2022-10-07 16:10:30 +00:00
PyTorch MergeBot
75df4b5e3d Revert "Merge more symbolic meta kernels and symint changes from branch (#86334)" 
This reverts commit 08e3999fa4.

Reverted https://github.com/pytorch/pytorch/pull/86334 on behalf of https://github.com/seemethere due to Trying to revert https://github.com/pytorch/pytorch/pull/86207, this PR causes merge conflicts with the initial revert so will have to revert this as well
 2022-10-07 16:03:30 +00:00
Edward Z. Yang
a00f8489df Relandx3 "SymIntify cat and narrow" (#86289) 
This reverts commit fc94a2115b.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86289
Approved by: https://github.com/wconstab
 2022-10-07 14:04:10 +00:00
PyTorch MergeBot
2110c89443 Revert "Revert "Revert "SymIntify cat and narrow (#86191)"" (#86289)" 
This reverts commit e778fbf519.

Reverted https://github.com/pytorch/pytorch/pull/86289 on behalf of https://github.com/seemethere due to Fails internal tests see: https://www.internalfb.com/intern/sandcastle/job/27021598552487548/
 2022-10-07 05:20:36 +00:00
Brian Hirsh
08e3999fa4 Merge more symbolic meta kernels and symint changes from branch (#86334) 
symintify split_with_sizes, dropout, fused_fake_obs_quant. meta for padding_2d ops

add meta_bernoulli_

meta kernel for at::gather

get pytorch_struct to pass: meta for scatter_add, fix backward

symintify split ops
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86334
Approved by: https://github.com/ezyang
 2022-10-06 23:29:04 +00:00
albanD
fd5085c445 Symintify getitem and add the required helper functions (#86207) 
Note that this might not cover every use of the function (we know it doesn't)
But this is enough to get few models passing.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86207
Approved by: https://github.com/ezyang, https://github.com/Chillee, https://github.com/bdhirsh
 2022-10-06 04:46:19 +00:00
PyTorch MergeBot
168ba066e3 Revert "Symintify getitem and add the required helper functions (#86207)" 
This reverts commit 17addb307e.

Reverted https://github.com/pytorch/pytorch/pull/86207 on behalf of https://github.com/malfet due to Broke lint, by double-registering `meta_index_put`, but no CI was run during the outage
 2022-10-05 22:42:56 +00:00
albanD
17addb307e Symintify getitem and add the required helper functions (#86207) 
Note that this might not cover every use of the function (we know it doesn't)
But this is enough to get few models passing.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86207
Approved by: https://github.com/ezyang
 2022-10-05 21:19:00 +00:00
Edward Z. Yang
e778fbf519 Revert "Revert "SymIntify cat and narrow (#86191)"" (#86289) 
This reverts commit fc94a2115b.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86289
Approved by: https://github.com/wconstab
 2022-10-05 20:51:28 +00:00
lezcano
b67e022833 Fix ref / decomposition index_add (#86266) 
The decomposition of `index_add` was using `slice(None)`, when it should
use just `None`.

The reference for index_add was also wrong, as `x[idx] += t` does not
use atomic add, so it does not work when several `idx`s point to the
same location.

This PR adds extra reference inputs to help test for this.

Fixes https://github.com/pytorch/torchdynamo/issues/1356
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86266
Approved by: https://github.com/ngimel
 2022-10-05 19:59:15 +00:00
PyTorch MergeBot
fc94a2115b Revert "SymIntify cat and narrow (#86191)" 
This reverts commit 63d8d4f6ec.

Reverted https://github.com/pytorch/pytorch/pull/86191 on behalf of https://github.com/seemethere due to Fails internal tests, see [D40106464](https://www.internalfb.com/diff/D40106464)
 2022-10-05 17:19:55 +00:00
Will Constable
63d8d4f6ec SymIntify cat and narrow (#86191) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86191
Approved by: https://github.com/ezyang
 2022-10-05 14:46:55 +00:00
lezcano
c609768896 Add refs for torch.unfold and a decomposition for its backward. (#85629) 
It's not clear to me what's the difference between `unfold` and `unfold_copy`, as this latter one is codegen'd

I also took this chance to clean the implementation of unfold and its reference
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85629
Approved by: https://github.com/mruberry
 2022-10-05 12:15:49 +00:00
Edward Z. Yang
d07b85393a SymInt fixes from symbolic-shapes branch (#86242) 
symintify a few inplace meta functions

symintify resize_(), nbytes(), functionalization input mutations

meta funcs for avg_pool2d_backward
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86242
Approved by: https://github.com/Chillee
 2022-10-05 04:52:02 +00:00
lezcano
1432b9978b Add ref for cumsum (#86229) 
As noted in the comment, this decomposition may not be as efficient as
specific implementations of it in different backends. Added here to then
benchmark it. Note that this is needed by TorchInductor https://github.com/pytorch/torchdynamo/issues/883
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86229
Approved by: https://github.com/ngimel
 2022-10-05 00:04:03 +00:00
Khushi
d6b030856b [primTorch] special: j0, j1, spherical_j0 (#86049) 
Adds prims and refs for special functions (bessel_j0, bessel_j1, spherical_bessel_j0). Thanks!

Pull Request resolved: https://github.com/pytorch/pytorch/pull/86049
Approved by: https://github.com/mruberry
 2022-10-04 18:21:46 +00:00
Horace He
b95e0fcc2c Forward fix land race (unexpected successes) (#86186) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86186
Approved by: https://github.com/ezyang
 2022-10-04 04:29:31 +00:00
Edward Z. Yang
79dd621f76 Symbolic shapes mega merge PR (Oct 3) (#86160) 
- TensorGeometry supports symint
- check_size supports symint
- functorch batch rule improved symint
- Some operator support for symint in LTC
- More supported operations on SymInt and SymFloat
- More symint support in backwards formulas

This merge includes code contributions from bdhirsh and anjali411.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86160
Approved by: https://github.com/Chillee
 2022-10-04 04:12:09 +00:00
Horace He
82d9592f1b Batch of symintifications to allow more models to pass in inference (#86104) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86104
Approved by: https://github.com/ezyang
 2022-10-04 04:01:58 +00:00
Edward Z. Yang
954660a308 Correctly error if you pass in tensors where size arguments expected (#86126) 
This also makes symintlist track intlist exception handling,
which eellison fixed.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86126
Approved by: https://github.com/eellison
 2022-10-03 20:18:41 +00:00
Edward Z. Yang
2aa9e0750a Symintified all functions, not including factory functions (#86078) 
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86078
Approved by: https://github.com/Chillee, https://github.com/albanD
 2022-10-03 20:12:32 +00:00
Horace He
bc993e39cc Unwrap SymInt => Proxy when being returned from the wrapped function make_fx traces (#86098) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86098
Approved by: https://github.com/ezyang
 2022-10-03 16:45:22 +00:00