Lucas Pasqualin
|
bcb6e5aa72
|
[DCP] Support partial load (#122829)
Adds ability to load a subset of keys directly from a checkpoint, avoiding the need to initialize state dict first
Differential Revision: [D55441391](https://our.internmc.facebook.com/intern/diff/D55441391/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/122829
Approved by: https://github.com/fegin
|
2024-04-02 19:22:22 +00:00 |
|
Lucas Pasqualin
|
909d73d8cb
|
[DCP] Removes no_dist and coordinator_rank from public DCP API's (#121317)
[DCP] Removes `no_dist` and `coordinator_rank` from public DCP API's
Differential Revision: [D54591181](https://our.internmc.facebook.com/intern/diff/D54591181/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/121317
Approved by: https://github.com/fegin
|
2024-03-08 02:14:12 +00:00 |
|
Lucas Pasqualin
|
eb1145436a
|
[DCP] Adds main in format utils (#120128)
Adds main in format utils. Usage:
`python -m torch.distributed.checkpoint.format_utils dcp_to_torch dcp_dir torch_file.pt`
or
`python -m torch.distributed.checkpoint.format_utils torch_to_dcp torch_file.pt dcp_dir`
Differential Revision: [D53791355](https://our.internmc.facebook.com/intern/diff/D53791355/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/120128
Approved by: https://github.com/fegin, https://github.com/wz337
|
2024-03-07 01:18:17 +00:00 |
|
Lucas Pasqualin
|
9d5dea7812
|
[DCP] Adds storage reader and planner classes for online loading/sharding of models in torch.save format (#119816)
as title
Differential Revision: [D53718041](https://our.internmc.facebook.com/intern/diff/D53718041/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/119816
Approved by: https://github.com/fegin
|
2024-03-01 00:21:05 +00:00 |
|
Lucas Pasqualin
|
1c1028ac49
|
[DCP] Adds utility for converting torch save to dcp (#119815)
as title
Differential Revision: [D53718040](https://our.internmc.facebook.com/intern/diff/D53718040/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/119815
Approved by: https://github.com/fegin
ghstack dependencies: #119813, #119814
|
2024-02-22 17:22:11 +00:00 |
|
Lucas Pasqualin
|
1ab441a7dd
|
[DCP] Adds utility for converting dcp to torch save format (#119814)
as title
Differential Revision: [D53718042](https://our.internmc.facebook.com/intern/diff/D53718042/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/119814
Approved by: https://github.com/fegin
ghstack dependencies: #119813
|
2024-02-22 16:55:58 +00:00 |
|