wz337 8140494afd [3/N][2D] Enable training with new 2D flow (#110034) ... Replacing https://github.com/pytorch/pytorch/pull/109553 as it gets reverted. This PR enables training with new 2D flow and adds associated test. In addition, this PR moves the tensor/parallel/_data_parallel_utils.py that are fsdp specific back to tensor/parallel/fsdp.py to avoid circular dependency for ddp.py and test/distributed/tensor/parallel/test_ddp_2d_parallel.py. state_dict related changes would be in later PRs. cc. @fegin, @fduwjj, @wanchaol, @awgu Pull Request resolved: https://github.com/pytorch/pytorch/pull/110034 Approved by: https://github.com/fduwjj		2023-09-26 09:14:15 +00:00
..
parallel	[3/N][2D] Enable training with new 2D flow (#110034)	2023-09-26 09:14:15 +00:00
__init__.py