pytorch/docs/source/notes
Frank Lin 249e65b92d Graph-Safe RNG State Exchange for Tensor Parallelism (#114068)
See #113541

The PR allows for registering and controlling multiple RNG states using indices, ensuring cudagraph-safe operations, and includes both C++ and Python API changes to support this functionality.

cc  @eellison @anijain2305 @jansel @ezyang @ptrblck @csarofeen @mcarilli
Pull Request resolved: https://github.com/pytorch/pytorch/pull/114068
Approved by: https://github.com/ezyang, https://github.com/eqy, https://github.com/xuzhao9
2024-03-27 01:14:38 +00:00
..
amp_examples.rst
autograd.rst
broadcasting.rst
cpu_threading_runtimes.svg
cpu_threading_torchscript_inference.rst
cpu_threading_torchscript_inference.svg
cuda.rst Graph-Safe RNG State Exchange for Tensor Parallelism (#114068) 2024-03-27 01:14:38 +00:00
ddp.rst
extending.func.rst
extending.rst
faq.rst
fsdp.rst
gradcheck.rst
hip.rst
large_scale_deployments.rst
modules.rst
mps.rst
multiprocessing.rst
numerical_accuracy.rst
randomness.rst
serialization.rst
windows.rst