2 Commits

Author	SHA1	Message	Date
Tristan Rice	7f4c3e7d2f	distributed/serialization: support zero sized tensors (#164198) ... Fixes ``` [4] ValueError: both buffer length (0) and count (-1) must not be 0 ``` Test plan: ``` pytest test/distributed/test_serialization.py ``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/164198 Approved by: https://github.com/amirafzali	2025-09-30 08:11:29 +00:00
Tristan Rice	23af9dde4d	distributed/serialization: add experimental streaming torch.save/load methods (#146555) ... Summary: This is intended for use with torchft when we need to do a streaming state dict transfer. This is strictly superior to the prior streaming method in torchft as this supports all tensor subclasses such as DTensor. This supports 100% of the inputs to torch.save/load but is not wire compatible nor intended to have any backwards compatibility. Security wise this fully supports weights_only and defaults to True. It does use pickle for some metadata but uses weights_only for the metadata. Adapted from: https://github.com/pytorch/torchft/pull/101 https://github.com/pytorch/torchft/pull/54 Test Plan: pytest test/distributed/test_serialization.py Pull Request resolved: https://github.com/pytorch/pytorch/pull/146555 Approved by: https://github.com/fegin, https://github.com/mikaylagawarecki Co-authored-by: Krishn Parasar <76171905+Krishn1412@users.noreply.github.com>	2025-02-07 18:08:11 +00:00