Using the same repro from the issue (but with BatchNorm2D)
Rectifies native_batch_norm schema by splitting the schema into 2:
1. one will have NON-optional alias-able running_mean and running_var inputs
2. the other will just not have those parameters at all (no_stats variation)
**Calling for name suggestions!**
## test plan
I've added tests in test_functionalization.py as well as an entry in common_method_invocations.py for `native_batch_norm_legit`
CI should pass.
## next steps
Because of bc/fc reasons, we reroute native_batch_norm to call our new schemas ONLY through the python dispatcher, but in 2 weeks or so, we should make `native_batch_norm_legit` the official batch_norm.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/88697
Approved by: https://github.com/albanD
`diag` was unnecessarily implemented as a kernel rather than as a composite
function, which made it unnecessarily difficult (explicit backward + all it entails).
We also change a few uses of `diag` on 2D tensors for `diagonal()`. The
latter returns a view rather than creating a new tensor.
We also upgrade its meta implementation to a fully-fledged
decomposition
I tried implementing the backwards of `diagonal()` via `diag_scatter` (or better `diag_scatter_` to keep the perf) but functionalisation was failing and I was not sure how to fix this, so I moved on. It may be possible to simplify that one as well if @soulitzer or someone knows how to do this.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87180
Approved by: https://github.com/ngimel, https://github.com/albanD, https://github.com/mruberry
Change the names of the parameters in the `addmm` meta kernel to be more consistent with `mm`. Functionally, the only difference in behaviour should be that `addmm` meta kernel gets its options from the `input` tensor instead of from the `bias` parameter.
Fixes#84930
CC: @ezyang @ngimel @wconstab @ke1337 @glebk-cerebras
Pull Request resolved: https://github.com/pytorch/pytorch/pull/84960
Approved by: https://github.com/ezyang
switching the tensor constructor to use `lift_fresh` instead of `lift` silently broken LTC, because we didn't have a test for it. I added one here.
The fix is that LTC now also needs a kernel for `lift_fresh`.
The way this error manifests is that if you call `torch.tensor(..., device='lazy')`, we expect the result to be wrapped up a functional wrapper, and it wasn't - so calling any view ops on the tensor will now break. That wrapping is supposed to happen in `lift_fresh` now.
cc @antoniojkim
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81928
Approved by: https://github.com/antoniojkim, https://github.com/wconstab
This PR adds support for `SymInt`s in python. Namely,
* `THPVariable_size` now returns `sym_sizes()`
* python arg parser is modified to parse PyObjects into ints and `SymbolicIntNode`s
* pybind11 bindings for `SymbolicIntNode` are added, so size expressions can be traced
* a large number of tests added to demonstrate how to implement python symints.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78135
Approved by: https://github.com/ezyang
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75359
For some models in torchbench (e.g. pyhpc_isoneutral_mixing), dynamo will generate Fx graphs that has side effects . Those graphs may
- return an empty tuple
- change tensors passed in as forward method arguments in-place
This makes the Dynamo+LTC integration fail since we extract compiled graph based on the lazy tensors returned from the forward method. From an empty tuple, we extract nothing.
To solve this problem, we extract compile graph from `union(argument lazy tensors, returned lazy tensors)` instead. The inplace mutations applied to argument lazy tensors will be captured this way.
Test Plan:
```
pytest test/lazy/test_extract_compiled_graph.py
```
```
LTC_TS_CUDA=1 gpui time python torchbench.py --speedup-ltc -dcuda --nvfuser --randomize-input --only pyhpc_isoneutral_mixing
```
Reviewed By: ZolotukhinM
Differential Revision: D35478799
Pulled By: shunting314
fbshipit-source-id: 8116768fc50fe7630e481e6039319ddf5c6a9416
(cherry picked from commit 2e6531d2c80c35ae99c11d49ca01dcdb7fc032f2)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75292
- Follow the convention in [this doc](https://docs.google.com/document/d/1Vi96ITGoK7BW01ZEccexs4pvCQKF4_LdV8w7TfIWPvM/edit) to setup config for ltc force fallback ops.
- Pybinds are added to read/set the config.
- Use the added pybinds in the unit test which needs to force fallbacks.
Test Plan:
```
pytest test/lazy/test_extract_compiled_graph.py
```
Reviewed By: malfet
Differential Revision: D35417678
Pulled By: shunting314
fbshipit-source-id: 1e05b8c831174872d70257a0ddd958863d6ca80d
(cherry picked from commit 9366bde7ef20837dcf03b7d8e18f9017a58c80fa)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75046
Merge the code needed for dynamic+ltc integration from the staging branch to the master branch.
Test Plan:
Unit test
```
pytest test_extract_compiled_graph
```
test thru dynamo
```
LTC_TS_CUDA=1 time python torchbench.py --speedup-ltc -dcuda --nvfuser --randomize-input --only <model name>
```
Reviewed By: alanwaketan
Differential Revision: D35300646
Pulled By: shunting314
fbshipit-source-id: 09ed20d3bb8ef80e4b93ba87ea3356a07d2dccdb
(cherry picked from commit 2b56771cdfd2cfa825c65ee9fd42338fb372fb32)
Summary:
This adds a minimal set of python bindings for lazy tensor and the torchscript backend.
It targets the APIs that are used by the `test_ts_opinfo.py` test (which it also lands, from lazy_tensor_staging, where it is [lazy_tensor_core/test/test_lazy.py](https://github.com/pytorch/pytorch/blob/lazy_tensor_staging/lazy_tensor_core/test/test_lazy.py)).
We should land more python bindings obviously. I just wanted to focus on a minimal set that can also be tested, and use it to agree on how we organize the bindings, then others could easily contribute bindings on top of this infrastructure.
cc JackCaoG
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74508
Reviewed By: pbelevich
Differential Revision: D35032152
Pulled By: wconstab
fbshipit-source-id: 526505ab355b7ad27037ece0ff814b2a4b69f1e2
(cherry picked from commit b4f73dd147472cb38003204aff228087c0230fda)
