Commit Graph

3 Commits

Author SHA1 Message Date
Vitaly Fedyunin
883f8ef62e [DataLoader] DataLoader now automatically apply sharding to DataPipes
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78631

Approved by: https://github.com/ejguan, https://github.com/NivekT
2022-06-02 17:40:29 +00:00
Vitaly Fedyunin
d90012689f [DataPipe] Control shuffle settings from DataLoader2 (#65756)
Summary:
Makes `shuffle` DataPipe sensitive to DataLoader(2) `shuffle` kwarg.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/65756

Reviewed By: albanD

Differential Revision: D31344867

Pulled By: VitalyFedyunin

fbshipit-source-id: e0084e0ac193ac784d6298328ca1222745681347
2021-12-14 07:35:26 -08:00
Vitaly Fedyunin
fd13e925ec Adding backward compatibility for sharding support in old DataLoader (#61237)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/61237

Test Plan: Imported from OSS

Reviewed By: ejguan

Differential Revision: D29588832

Pulled By: VitalyFedyunin

fbshipit-source-id: 3bfa4417f6a04450f656ecf28fc95322d2cf076a
2021-07-12 14:53:45 -07:00