Wanchao Liang
d31707a257
Get rid of dim_groups attribute from DeviceMesh ( #103105 )
...
This PR get rids of the dim_groups attribute from DeviceMesh, the main
motivation behind this is that we should let c10d store the process
groups during its creation instead of DeviceMesh, DeviceMesh should just
handle ranks correctly.
This could enable DTensor becomes picklable! (torch.save/load could be
possible), which I will give it a try in the next PR
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103105
Approved by: https://github.com/XilunWu , https://github.com/fduwjj
2023-06-09 04:11:15 +00:00
fduwjj
d4380edb9b
[TP] Add API logging for TP high level API ( #102209 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102209
Approved by: https://github.com/wz337 , https://github.com/wanchaol
2023-05-25 03:33:00 +00:00
Wanchao Liang
261eb46ddd
[dtensor] refactor get_coordiniate ( #95457 )
...
This refactor get_coordinate to return a optional[list] instead of
directly the coordinate on dim, this is so that we can check if the
rank is inside the mesh easily
Differential Revision: [D43643579](https://our.internmc.facebook.com/intern/diff/D43643579 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/95457
Approved by: https://github.com/XilunWu
2023-02-28 17:54:26 +00:00
fduwjj
39511697d4
[PT-D][BE] Update 2D parallelism API name and docs ( #94771 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/94771
Approved by: https://github.com/wanchaol
2023-02-14 08:13:15 +00:00
Wanchao Liang
9b5e6b029f
[tp] umft distributed.tensor.parallel ( #89969 )
...
cmd: `ufmt format torch/distributed/tensor`
Pull Request resolved: https://github.com/pytorch/pytorch/pull/89969
Approved by: https://github.com/fduwjj
2022-12-01 20:58:16 +00:00
Wanchao Liang
4451eb24e6
Move tensor_parallel out to distributed.tensor folder ( #89878 )
...
This PR moves tensor parallel from torch.distributed._tensor.parallel
to torch.distributed.tensor.parallel, to prepare for beta release
Pull Request resolved: https://github.com/pytorch/pytorch/pull/89878
Approved by: https://github.com/fduwjj
2022-11-30 22:13:10 +00:00
