OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-06 12:20:52 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
1e37e5b836
pytorch/benchmarks
History
henrylhtsang 66300d3d55 [cutlass backend] try make cutlass backend benchmark more robust (#149015) 
Differential Revision: [D71006269](https://our.internmc.facebook.com/intern/diff/D71006269/)

I want to make sure the benchmark even if failed on some experiment can still print most of the results.

```
Experiment group: mm (3x3, 3x3) torch.bfloat16
+-----------------------+-------------------+----------------------+---------------------+
|         name          | forward_time (us) | compilation_time (s) | perf_over_aten (%)  |
+-----------------------+-------------------+----------------------+---------------------+
|         aten          | 6.175220478326082 |  0.5982149520423263  |         NA          |
|        triton         | 5.326753947883844 |  3.2067150759976357  | -13.739858089605114 |
| triton_persistent_tma | 5.340870004147291 |  3.279932268196717   | -13.51126615004617  |
|  cutlass_lvl_default  |        inf        |         inf          |         inf         |
|   cutlass_lvl_1111    |        inf        |         inf          |         inf         |
|   cutlass_lvl_2222    |        inf        |         inf          |         inf         |
|   cutlass_lvl_3333    |        inf        |         inf          |         inf         |
+-----------------------+-------------------+----------------------+---------------------+
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/149015
Approved by: https://github.com/chenyang78, https://github.com/jingsh
2025-03-12 18:59:49 +00:00
..
distributed Revert "Use absolute path path.resolve() -> path.absolute() (#129409)" 2025-01-04 14:17:20 +00:00
dynamo Add the max_autotune tests in the periodic jobs. (#143560) 2025-03-12 01:47:46 +00:00
fastrnns [BE]: Enable ruff rule SIM113 (#147290) 2025-02-16 22:41:16 +00:00
framework_overhead_benchmark Fix unused Python variables outside torch/ and test/ (#136359) 2024-12-11 17:10:23 +00:00
functional_autograd_benchmark [BE][CI] bump ruff to 0.9.2: multiline assert statements (#144546) 2025-02-27 20:46:16 +00:00
fuser Fix unused Python variables outside torch/ and test/ (#136359) 2024-12-11 17:10:23 +00:00
gpt_fast [BE][CI] bump ruff to 0.9.2: multiline assert statements (#144546) 2025-02-27 20:46:16 +00:00
inductor_backends [cutlass backend] try make cutlass backend benchmark more robust (#149015) 2025-03-12 18:59:49 +00:00
inference [BE][Easy][3/19] enforce style for empty lines in import segments in benchmarks/ (#129754) 2024-07-17 14:34:42 +00:00
instruction_counts [BE][CI] bump ruff to 0.9.2: multiline assert statements (#144546) 2025-02-27 20:46:16 +00:00
nested Fix unused Python variables outside torch/ and test/ (#136359) 2024-12-11 17:10:23 +00:00
operator_benchmark [BE][CI] bump ruff to 0.9.2: multiline assert statements (#144546) 2025-02-27 20:46:16 +00:00
overrides_benchmark [BE][Easy][3/19] enforce style for empty lines in import segments in benchmarks/ (#129754) 2024-07-17 14:34:42 +00:00
profiler_benchmark Apply TorchFix TOR203 fixes (#143691) 2024-12-23 18:21:03 +00:00
record_function_benchmark [Caffe2]Remove Caffe2 scripts and benchmarks (#126747) 2024-06-05 23:46:31 +00:00
serialization Fix unused Python variables outside torch/ and test/ (#136359) 2024-12-11 17:10:23 +00:00
sparse [BE][CI] bump ruff to 0.9.2: multiline assert statements (#144546) 2025-02-27 20:46:16 +00:00
static_runtime [StaticRuntime] Fix a bug that memory planner ignores subblocks (#146728) (#146855) 2025-02-11 13:59:54 +00:00
tensorexpr [BE][CI] bump ruff to 0.8.4 (#143753) 2024-12-24 12:24:10 +00:00
transformer Add sparsity (#148513) 2025-03-07 01:47:52 +00:00
compare-fastrnn-results.py [BE][Easy][3/19] enforce style for empty lines in import segments in benchmarks/ (#129754) 2024-07-17 14:34:42 +00:00
compare.sh
README.md
upload_scribe.py

README.md

PyTorch Benchmarks

This folder contains scripts that produce reproducible timings of various PyTorch features.

It also provides mechanisms to compare PyTorch with other frameworks.

Setup environment

Make sure you're on a machine with CUDA, torchvision, and pytorch installed. Install in the following order:

# Install torchvision. It comes with the pytorch stable release binary
conda install pytorch torchvision -c pytorch

# Install the latest pytorch master from source.
# It should supersede the installation from the release binary.
cd $PYTORCH_HOME
python setup.py build develop

# Check the pytorch installation version
python -c "import torch; print(torch.__version__)"

Benchmark List

Please refer to each subfolder to discover each benchmark suite. Links are provided where descriptions exist: