pytorch/docs/source/notes
Frank Lin 249e65b92d Graph-Safe RNG State Exchange for Tensor Parallelism (#114068)
See #113541

The PR allows for registering and controlling multiple RNG states using indices, ensuring cudagraph-safe operations, and includes both C++ and Python API changes to support this functionality.

cc  @eellison @anijain2305 @jansel @ezyang @ptrblck @csarofeen @mcarilli
Pull Request resolved: https://github.com/pytorch/pytorch/pull/114068
Approved by: https://github.com/ezyang, https://github.com/eqy, https://github.com/xuzhao9
2024-03-27 01:14:38 +00:00
..
amp_examples.rst [AMP] Use generic autocast in example, specify dtype (#79579) 2022-06-17 21:32:51 +00:00
autograd.rst Document complex optimizer semantic behavior (#121667) 2024-03-16 00:43:47 +00:00
broadcasting.rst Fixes docs (#51439) 2021-01-31 22:00:26 -08:00
cpu_threading_runtimes.svg
cpu_threading_torchscript_inference.rst
cpu_threading_torchscript_inference.svg Lint trailing newlines (#54737) 2021-03-30 13:09:52 -07:00
cuda.rst Graph-Safe RNG State Exchange for Tensor Parallelism (#114068) 2024-03-27 01:14:38 +00:00
ddp.rst Update DDP dynamo debug docs (#118295) 2024-01-29 14:58:26 +00:00
extending.func.rst Fix the example in the extending.func.rst (#109279) 2023-09-14 17:29:39 +00:00
extending.rst [BE] add parentheses to kwargs unpacking func(*args, **(kwargs or {})) (#115026) 2023-12-03 20:03:26 +00:00
faq.rst Update faq.rst so OOM section mentions checkpoint (#62709) 2021-08-05 07:40:08 -07:00
fsdp.rst [docs] start a new FSDP notes doc (#117323) 2024-01-22 15:46:35 +00:00
gradcheck.rst Add first draft of gradcheck note (#55966) 2021-04-27 14:33:42 -07:00
hip.rst Tweak formatting of note on macros (#89598) 2022-11-28 20:42:30 +00:00
large_scale_deployments.rst Move ThreadLocalDebugInfo to c10 (#37774) 2020-05-11 19:27:41 -07:00
modules.rst Replace master with main in links and docs/conf.py (#100176) 2023-05-02 18:20:32 +00:00
mps.rst update mps note with more details (#78669) 2022-06-02 20:53:19 +00:00
multiprocessing.rst [draft] Update Multiprocessing best practices with CPU device (#103229) 2023-06-25 06:26:40 +00:00
numerical_accuracy.rst [DOCS][CUDA] Update TF32 docs for sm90 (#111337) 2023-10-19 09:36:13 +00:00
randomness.rst Changed cublas repdocubility URL (#121534) 2024-03-08 23:46:21 +00:00
serialization.rst Document file format returned by torch.save (#118719) 2024-02-03 02:11:44 +00:00
windows.rst Remove remaining THC code (#69039) 2021-12-08 12:18:08 -08:00