fduwjj aa84078c6c [PTD][TP] Add BWD support for colwise embedding sharding (#104820) ... Originally, we didn't enable BWD for colwise embedding because we thought it was just for inference, but it turns out that we do need it for training. So, let's enable it for now and unit test is also added. Pull Request resolved: https://github.com/pytorch/pytorch/pull/104820 Approved by: https://github.com/fegin		2023-07-10 22:33:20 +00:00
..
_composable	[FSDP][3/N] Unify fully_shard auto wrap (#104408)	2023-07-08 12:40:12 +00:00
_shard	enforce dtype (reland) (#102996)	2023-06-06 00:35:04 +00:00
_sharded_tensor
_sharding_spec
_spmd	Add meta registrations and distributed decomps: _foreach_div_.Scalar, sqrt_.default (#104779)	2023-07-10 17:38:46 +00:00
_tensor	[PTD][TP] Add BWD support for colwise embedding sharding (#104820)	2023-07-10 22:33:20 +00:00
_tools
algorithms	[CheckpointWrapper] Warn on reentrant use (#102890)	2023-06-04 18:31:22 +00:00
autograd
benchmarks
checkpoint	[DCP][fsspec] Consolidate OSS FsspecWriter/Reader and internal FsspecWriter/Reader (#104724)	2023-07-10 19:31:01 +00:00
elastic	[BE] Fix flake8 B027 errors - missing abstractmethod decorator (#100715)	2023-05-09 17:28:48 +00:00
examples
fsdp	[FSDP][optim_state_dict] The correct way to initialize optimizer states if the corresponding param is empty (#104765)	2023-07-10 08:00:55 +00:00
launcher
nn	Added option to always call nn.Module global/non-global forward hooks (#104278)	2023-07-10 18:58:07 +00:00
optim	[FSDP][optim_state_dict] The correct way to initialize optimizer states if the corresponding param is empty (#104765)	2023-07-10 08:00:55 +00:00
pipeline	Enable ruff in lintrunner (#99785)	2023-04-24 16:18:44 +00:00
rpc	[BE]: enable PLE error codes in ruff and fix bugs (#101079)	2023-05-11 23:57:25 +00:00
tensor	[DTensor][TP][Random] Introduce TensorParallelRNGTracker to integrate parallel RNG state with Tensor Parallel (#103910)	2023-06-30 08:06:41 +00:00
__init__.py	[c10d] Record time spent for init_process_group, new_group, _store_based_barrier (#101912)	2023-05-24 09:36:34 +00:00
_composable_state.py
_functional_collectives_impl.py	Refactor funcol for readability and dynamo tracing (#104387)	2023-07-06 23:29:49 +00:00
_functional_collectives.py	Refactor funcol for readability and dynamo tracing (#104387)	2023-07-06 23:29:49 +00:00
argparse_util.py
c10d_logger.py	[c10d] Record time spent for init_process_group, new_group, _store_based_barrier (#101912)	2023-05-24 09:36:34 +00:00
collective_utils.py	Initial commit of collective_utils (#101037)	2023-06-27 02:15:16 +00:00
constants.py
CONTRIBUTING.md
distributed_c10d.py	Implement coalesced reduce_scatter_tensor (#103561)	2023-06-15 20:11:12 +00:00
launch.py
logging_handlers.py
remote_device.py
rendezvous.py
run.py
utils.py	[FSDP/DDP] Unify _cast_forward_inputs (#102680)	2023-06-04 18:31:21 +00:00