Jane Xu
8695f0cced
Rectify native_batch_norm schema by splitting it into two legit schemas ( #88697 )
...
Using the same repro from the issue (but with BatchNorm2D)
Rectifies native_batch_norm schema by splitting the schema into 2:
1. one will have NON-optional alias-able running_mean and running_var inputs
2. the other will just not have those parameters at all (no_stats variation)
**Calling for name suggestions!**
## test plan
I've added tests in test_functionalization.py as well as an entry in common_method_invocations.py for `native_batch_norm_legit`
CI should pass.
## next steps
Because of bc/fc reasons, we reroute native_batch_norm to call our new schemas ONLY through the python dispatcher, but in 2 weeks or so, we should make `native_batch_norm_legit` the official batch_norm.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/88697
Approved by: https://github.com/albanD
2022-11-23 23:23:17 +00:00
Henry Tu
02093da36c
Autogen native_batch_norm and native_batch_norm_backward ( #79637 )
...
This PR makes the `native_batch_norm` and `native_batch_norm_backward` ops autogen, and implements their respective shape inference functions.
Previously, these two ops were manually implemented.
cc: @ke1337 @antoniojkim @wconstab @desertfire
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79637
Approved by: https://github.com/Gamrix , https://github.com/desertfire
2022-06-24 23:29:33 +00:00
Edward Z. Yang
f7ee061638
Wconstab/reland pysymint ( #79795 )
...
rebased https://github.com/pytorch/pytorch/pull/79617/ to see if issues are reproducible.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79795
Approved by: https://github.com/malfet
2022-06-20 22:55:06 +00:00
PyTorch MergeBot
44436947bc
Revert "Reland PySymInt ( #79617 )"
...
This reverts commit 8ef6356f26https://github.com/pytorch/pytorch/pull/79617 on behalf of https://github.com/zengk95 due to this is breaking periodic jobs (and maybe pull) on trunk
2022-06-16 19:40:27 +00:00
Nikolay Korovaiko
8ef6356f26
Reland PySymInt ( #79617 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79617
Approved by: https://github.com/Chillee
2022-06-16 04:18:06 +00:00
PyTorch MergeBot
b8db0a0475
Revert "Python Bindings for SymInts ( #78135 )"
...
This reverts commit d332724071https://github.com/pytorch/pytorch/pull/78135 on behalf of https://github.com/ezyang due to broke torchvision tests
2022-06-15 13:52:14 +00:00
Nikolay Korovaiko
d332724071
Python Bindings for SymInts ( #78135 )
...
This PR adds support for `SymInt`s in python. Namely,
* `THPVariable_size` now returns `sym_sizes()`
* python arg parser is modified to parse PyObjects into ints and `SymbolicIntNode`s
* pybind11 bindings for `SymbolicIntNode` are added, so size expressions can be traced
* a large number of tests added to demonstrate how to implement python symints.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78135
Approved by: https://github.com/ezyang
2022-06-14 02:17:59 +00:00
Bin Bao
29189d2ba8
[LT] Add IR resuing support for manually-implemented ops
...
Summary: Add CanBeReused methods for manually-implemented ops and replace MakeNode with
ReuseOrMakeNode.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77616
Approved by: https://github.com/JackCaoG , https://github.com/wconstab
2022-05-26 16:04:47 +00:00
Nikolay Korovaiko
df1f9b9840
Implement sym_sizes to create proper IR for sym ints representing tensor sizes ( #77756 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77756
Approved by: https://github.com/desertfire
2022-05-20 05:39:03 +00:00
Bin Bao
25c6ebd12c
Revert "Revert "[LT] Codegen ReuseNode for supported ops""
...
Summary: Fixed a XLC build failure by generating an always-return-false
default CanBeReused method.
This reverts commit 3cade9d454https://github.com/pytorch/pytorch/pull/77513
Approved by: https://github.com/alanwaketan
2022-05-16 20:14:42 +00:00
PyTorch MergeBot
3cade9d454
Revert "[LT] Codegen ReuseNode for supported ops"
...
This reverts commit 6066e5929fhttps://github.com/pytorch/pytorch/pull/76738 on behalf of https://github.com/malfet
2022-05-14 00:33:10 +00:00
Bin Bao
6066e5929f
[LT] Codegen ReuseNode for supported ops
...
Summary:
1. Update the codegen script to add a TrieCache lookup (ReuseNode)
before creating a new IR node. The following is an example generated
code,
```
at::Tensor LazyNativeFunctions::add(const at::Tensor & self, const at::Tensor & other, const at::Scalar & alpha) {
...
torch::lazy::NodePtr node = torch::lazy::ReuseNode<AddTensor>(lazy_self->GetIrValue(), lazy_other->GetIrValue(), node_alpha);
if (!node) {
auto out_meta = at::meta::add(self, other, alpha);
std::vector<Shape> shapes{Shape(out_meta.scalar_type(), out_meta.sizes().vec())};
TORCH_INTERNAL_ASSERT(shapes.size() == 1);
if(symbolicShapeEnabled()){
std::vector<jit::IValue> inputs = { self, other, alpha };
char* schema_str = "aten::add.Tensor(Tensor self, Tensor other, *, Scalar alpha=1) -> Tensor";
applySymbolicShapesOnLT(schema_str, inputs, shapes);
}
node = torch::lazy::MakeNode<AddTensor>(lazy_self->GetIrValue(), lazy_other->GetIrValue(), node_alpha, std::move(shapes));
CacheNode(node);
}
...
}
```
2. TrieCache lookup depends on each IR node subclass to provide its own
comparison function. The following is an example generated code,
```
bool CanBeReused(const torch::lazy::Value& self, const torch::lazy::Value& other, const torch::lazy::Value& alpha) const {
size_t i = 0;
return (operand(i++) == self &&
operand(i++) == other &&
operand(i++) == alpha);
}
```
3. DeviceData is specially handled.
4. Non-codegen op changes are coming a separate PR.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76738
Approved by: https://github.com/JackCaoG , https://github.com/wconstab
2022-05-13 19:13:58 +00:00
