OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-06 12:20:52 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
master
pytorch/benchmarks/dynamo/genai_layers
History
Shunting Zhang 0db6bcc015 Fix accuracy for layernorm/rmsnorm benchmarking (#166005) 
Example command:
    python benchmarks/dynamo/genai_layers/benchmark.py --exit-on-accuracy-failure --tolerance=1e-2 rmsnorm_backward

Fix the accuracy problem for layernorm/rmsnorm fwd/bwd.
Also fix some quack calls (maybe due to quack API change)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/166005
Approved by: https://github.com/BoyuanFeng
2025-10-24 18:14:51 +00:00
..
benchmark.py Fix accuracy for layernorm/rmsnorm benchmarking (#166005) 2025-10-24 18:14:51 +00:00
kernels.py Fix accuracy for layernorm/rmsnorm benchmarking (#166005) 2025-10-24 18:14:51 +00:00
README.md
requirements.txt
utils.py Fix accuracy for layernorm/rmsnorm benchmarking (#166005) 2025-10-24 18:14:51 +00:00

README.md

GenAI Kernel Benchmark

This directory contains benchmarks for the GenAI kernels. It compares pytorch eager, pytorch compiler, quack, and liger.

Setup

Assuming pytorch is installed.

pip install -r requirements.txt

Run

  python benchmark.py --list                    # List all available benchmarks
  python benchmark.py --all                     # Run all benchmarks
  python benchmark.py cross_entropy_forward     # Run specific benchmark
  python benchmark.py softmax_forward softmax_backward  # Run multiple benchmarks

Add --visualize to plot graph for the benchmark results.