2 Commits

Author	SHA1	Message	Date
Will Constable	123b103bf1	Add dynamo_optimize_ddp arg to dist bench (#87768) ... cc @jansel @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 Pull Request resolved: https://github.com/pytorch/pytorch/pull/87768 Approved by: https://github.com/davidberard98	2022-10-26 19:29:35 +00:00
Will Constable	7a4d91cac4	Add distributed dynamo benchmarking utils (#87419) ... Util for convenient local benchmarking/debugging of distributed models. Not to be confused with the 'real' distributed benchmark script we use for torchbench experiments on slurm. Tries to be simple/hackable and let you use different combinations of DDP/FSDP with models and dynamo backends. Example usage `python benchmarks/dynamo/distributed.py --toy_model --dynamo inductor --ddp` `--dynamo` flag accepts normal dynamo backends (plus 'print' which literally prints graphs to screen) `--torchbench_model <model_name>` works in place of `--toy_model` `--fsdp` is WIP cc @jansel @lezcano @fdrocha @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 Pull Request resolved: https://github.com/pytorch/pytorch/pull/87419 Approved by: https://github.com/jansel	2022-10-24 17:39:57 +00:00