Michael Voznesensky
333e771394
Add benchmarks.py to run all benchmarks, add new file with all torchbench model names ( #94146 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94146
Approved by: https://github.com/ezyang
2023-02-08 01:18:38 +00:00
Edward Z. Yang
c52567ec18
Switch CI exclusions to use exact match. ( #92761 )
...
Since the CI exclusions are hard-coded in our script, we might as well require them to match exactly. This solved some head scratching where I was like, "this model is not obviously excluded, why is it not showing up in CI."
Signed-off-by: Edward Z. Yang <ezyang@meta.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92761
Approved by: https://github.com/jansel
2023-01-22 17:10:20 +00:00
blzheng
0c1777acec
Dynamo benchmark: add CPU specific changes ( #88477 )
...
This pr adds some CPU specific changes:
- Add support for IPEX backend
- https://github.com/pytorch/torchdynamo/issues/1618
- https://github.com/pytorch/torchdynamo/issues/1534
- Enable CPU launcher in runner.py.
- Fix the issue that some environment variables are not support on CPU
Pull Request resolved: https://github.com/pytorch/pytorch/pull/88477
Approved by: https://github.com/jgong5 , https://github.com/jansel
2023-01-07 09:26:06 +00:00
Natalia Gimelshein
c43209db4d
use libdevice for tanh ( #90889 )
...
Per title
I see slight differences in perf with this implementation, where standalone tanh is slightly slower for a tensor of 4000000
elements (20.4 us instead of 19.4us), other sizes are within noise.
@bertmaher could you check if it affects your benchmarks?
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90889
Approved by: https://github.com/bertmaher , https://github.com/anijain2305
2022-12-20 23:21:37 +00:00
Michael Lazos
7c524221ba
[reland3][dynamo] Revert "Revert "[reland][dynamo] use optimizers correctly in benchmar… ( #90956 )
...
…king (#87492 )" (#90746 )"
This reverts commit ff1bbc2773https://github.com/pytorch/pytorch/pull/90936 ), and the numeric mismatch was root-caused to three decomps (still investigating why those decomps cause this) see https://github.com/pytorch/torchdynamo/issues/1985 for more detail.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90956
Approved by: https://github.com/desertfire
2022-12-17 06:27:15 +00:00
PyTorch MergeBot
6bc6fb21db
Revert "[reland2][dynamo] Revert "Revert "[reland][dynamo] use optimizers correctly in benchmar… ( #90956 )"
...
This reverts commit 8bc38ae4e2https://github.com/pytorch/pytorch/pull/90956 on behalf of https://github.com/desertfire due to Causing TIMM model failures
2022-12-16 19:28:05 +00:00
Michael Lazos
8bc38ae4e2
[reland2][dynamo] Revert "Revert "[reland][dynamo] use optimizers correctly in benchmar… ( #90956 )
...
…king (#87492 )" (#90746 )"
This reverts commit ff1bbc2773https://github.com/pytorch/pytorch/pull/90936 ), and the numeric mismatch was root-caused to three decomps (still investigating why those decomps cause this) see https://github.com/pytorch/torchdynamo/issues/1985 for more detail.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90956
Approved by: https://github.com/desertfire
2022-12-16 13:33:38 +00:00
Michael Lazos
f3da157ce3
Reset rng in hf before loading a model ( #90936 )
...
Reset the rng in hf before generating input and loading model, this makes the huggingface inputs+weights deterministic depending on the seed of the rng. This matches the behavior of the other test suites.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90936
Approved by: https://github.com/desertfire
2022-12-16 00:15:27 +00:00
Bin Bao
ff1bbc2773
Revert "[reland][dynamo] use optimizers correctly in benchmarking ( #87492 )" ( #90746 )
...
This reverts commit d91d7a3221https://github.com/pytorch/pytorch/pull/90746
Approved by: https://github.com/anijain2305
2022-12-13 11:37:16 +00:00
Animesh Jain
d91d7a3221
[reland][dynamo] use optimizers correctly in benchmarking ( #87492 )
...
Reland https://github.com/pytorch/pytorch/pull/87311
mlazos: updated to use SGD to not add a bunch of additional memory allocations (like Adam)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87492
Approved by: https://github.com/desertfire
2022-12-09 20:32:53 +00:00
Animesh Jain
3162a48a77
[dynamo][benchmarks] Call zero grad ( #90026 )
...
Hoping that it might reduce some flakiness
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90026
Approved by: https://github.com/williamwen42
2022-12-02 04:05:57 +00:00
Animesh Jain
cad5772c2c
[dashboard][huggingface] skip accuracy checks for really large models… ( #89273 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/89273
Approved by: https://github.com/desertfire
2022-11-19 00:22:45 +00:00
Animesh Jain
74610a1ced
[dynamo][benchmarks] HF - Fix seq len and batch sizes ( #89165 )
...
Fixes many models in https://github.com/pytorch/torchdynamo/issues/1842
Pull Request resolved: https://github.com/pytorch/pytorch/pull/89165
Approved by: https://github.com/ngimel
2022-11-17 06:14:24 +00:00
Animesh Jain
1b575782a0
[dynamo][benchmarks] use fresh inductor cache and raise batch size wherever possible ( #88044 )
...
cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx
Pull Request resolved: https://github.com/pytorch/pytorch/pull/88044
Approved by: https://github.com/ngimel
2022-10-30 17:10:17 +00:00
PyTorch MergeBot
f38a88c4dd
Revert "[dynamo] use optimizers correctly in benchmarking ( #87311 )"
...
This reverts commit 703c19008dhttps://github.com/pytorch/pytorch/pull/87311 on behalf of https://github.com/anijain2305 due to Bin (desertfire) is trying to get torchbench models in CI, and this PR prevents that. I will bring this back after models are in CI.
2022-10-20 22:01:51 +00:00
Animesh Jain
703c19008d
[dynamo] use optimizers correctly in benchmarking ( #87311 )
...
We were not setting optimizers correctly
* This hid the issue that we see here - https://github.com/pytorch/torchdynamo/issues/1687
* This has also revealed that we are activating profilers for every dynamo optimized model call. This could affect speedup
cc @jansel @lezcano @fdrocha
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87311
Approved by: https://github.com/mlazos , https://github.com/yanboliang
2022-10-20 05:46:25 +00:00
Animesh Jain
c30cfb07ab
[dynamo][dashboard] Run 2 iterations for the correctness runs ( #87104 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87104
Approved by: https://github.com/soumith
2022-10-18 15:53:40 +00:00
Jason Ansel
054a2fd6c2
Sync changes from pytorch/torchdynamo ( #87013 )
...
This updates to:
6380959be2https://github.com/pytorch/torchdynamo/blob/main/copy_to_core.sh
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87013
Approved by: https://github.com/voznesenskym
2022-10-15 21:00:57 +00:00
Jason Ansel
c7c09722ad
Move TorchDynamo into PyTorch core ( #86461 )
...
Context:
https://github.com/pytorch/torchdynamo/issues/1588
This PR moves [TorchDynamo](https://github.com/pytorch/torchdynamo ) and TorchInductor into PyTorch core.
- `torchdynamo` becomes `torch._dynamo`
- `torchinductor` becomes `torch._inductor`
This PR was generated by running `copy_to_core.sh` in https://github.com/pytorch/torchdynamo/pull/1538
Pull Request resolved: https://github.com/pytorch/pytorch/pull/86461
Approved by: https://github.com/voznesenskym
2022-10-13 23:18:06 +00:00
