Michael Lazos
8bc38ae4e2
[reland2][dynamo] Revert "Revert "[reland][dynamo] use optimizers correctly in benchmar… ( #90956 )
...
…king (#87492 )" (#90746 )"
This reverts commit ff1bbc2773https://github.com/pytorch/pytorch/pull/90936 ), and the numeric mismatch was root-caused to three decomps (still investigating why those decomps cause this) see https://github.com/pytorch/torchdynamo/issues/1985 for more detail.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90956
Approved by: https://github.com/desertfire
2022-12-16 13:33:38 +00:00
Michael Lazos
f3da157ce3
Reset rng in hf before loading a model ( #90936 )
...
Reset the rng in hf before generating input and loading model, this makes the huggingface inputs+weights deterministic depending on the seed of the rng. This matches the behavior of the other test suites.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90936
Approved by: https://github.com/desertfire
2022-12-16 00:15:27 +00:00
Bin Bao
ff1bbc2773
Revert "[reland][dynamo] use optimizers correctly in benchmarking ( #87492 )" ( #90746 )
...
This reverts commit d91d7a3221https://github.com/pytorch/pytorch/pull/90746
Approved by: https://github.com/anijain2305
2022-12-13 11:37:16 +00:00
Animesh Jain
d91d7a3221
[reland][dynamo] use optimizers correctly in benchmarking ( #87492 )
...
Reland https://github.com/pytorch/pytorch/pull/87311
mlazos: updated to use SGD to not add a bunch of additional memory allocations (like Adam)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87492
Approved by: https://github.com/desertfire
2022-12-09 20:32:53 +00:00
Animesh Jain
3162a48a77
[dynamo][benchmarks] Call zero grad ( #90026 )
...
Hoping that it might reduce some flakiness
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90026
Approved by: https://github.com/williamwen42
2022-12-02 04:05:57 +00:00
Animesh Jain
cad5772c2c
[dashboard][huggingface] skip accuracy checks for really large models… ( #89273 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/89273
Approved by: https://github.com/desertfire
2022-11-19 00:22:45 +00:00
Animesh Jain
74610a1ced
[dynamo][benchmarks] HF - Fix seq len and batch sizes ( #89165 )
...
Fixes many models in https://github.com/pytorch/torchdynamo/issues/1842
Pull Request resolved: https://github.com/pytorch/pytorch/pull/89165
Approved by: https://github.com/ngimel
2022-11-17 06:14:24 +00:00
Animesh Jain
1b575782a0
[dynamo][benchmarks] use fresh inductor cache and raise batch size wherever possible ( #88044 )
...
cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx
Pull Request resolved: https://github.com/pytorch/pytorch/pull/88044
Approved by: https://github.com/ngimel
2022-10-30 17:10:17 +00:00
PyTorch MergeBot
f38a88c4dd
Revert "[dynamo] use optimizers correctly in benchmarking ( #87311 )"
...
This reverts commit 703c19008dhttps://github.com/pytorch/pytorch/pull/87311 on behalf of https://github.com/anijain2305 due to Bin (desertfire) is trying to get torchbench models in CI, and this PR prevents that. I will bring this back after models are in CI.
2022-10-20 22:01:51 +00:00
Animesh Jain
703c19008d
[dynamo] use optimizers correctly in benchmarking ( #87311 )
...
We were not setting optimizers correctly
* This hid the issue that we see here - https://github.com/pytorch/torchdynamo/issues/1687
* This has also revealed that we are activating profilers for every dynamo optimized model call. This could affect speedup
cc @jansel @lezcano @fdrocha
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87311
Approved by: https://github.com/mlazos , https://github.com/yanboliang
2022-10-20 05:46:25 +00:00
Animesh Jain
c30cfb07ab
[dynamo][dashboard] Run 2 iterations for the correctness runs ( #87104 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87104
Approved by: https://github.com/soumith
2022-10-18 15:53:40 +00:00
Jason Ansel
054a2fd6c2
Sync changes from pytorch/torchdynamo ( #87013 )
...
This updates to:
6380959be2https://github.com/pytorch/torchdynamo/blob/main/copy_to_core.sh
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87013
Approved by: https://github.com/voznesenskym
2022-10-15 21:00:57 +00:00
Jason Ansel
c7c09722ad
Move TorchDynamo into PyTorch core ( #86461 )
...
Context:
https://github.com/pytorch/torchdynamo/issues/1588
This PR moves [TorchDynamo](https://github.com/pytorch/torchdynamo ) and TorchInductor into PyTorch core.
- `torchdynamo` becomes `torch._dynamo`
- `torchinductor` becomes `torch._inductor`
This PR was generated by running `copy_to_core.sh` in https://github.com/pytorch/torchdynamo/pull/1538
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86461
Approved by: https://github.com/voznesenskym
2022-10-13 23:18:06 +00:00
