OSSForks/pytorch

mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00

History

Howard Huang 75109682b6 [Pipelining] Refactor Interleaved1F1B and ZeroBubble (#137783 ) NOTE: this PR removes `ScheduleFlexibleInterleaved1F1B`, let me know if theres any concerns. `ScheduleFlexibleInterleaved1F1B` is a superset of `Interleaved1F1B` and uses most of the same implementation, but relaxes the condition that `n_microbatches % pp_size == 0`. This is refactors the implementation into `Interleaved1F1B` and then removes it since it is confusing to have both schedules with similar names. This also refactors the zero bubble logic to belong in the `ZeroBubble` schedule class. Pull Request resolved: https://github.com/pytorch/pytorch/pull/137783 Approved by: https://github.com/wconstab		2024-10-16 03:05:14 +00:00
..
__init__.py	[Pipelining] Refactor Interleaved1F1B and ZeroBubble (#137783 )	2024-10-16 03:05:14 +00:00
_backward.py	[PP] Remove modifications to autograd nodes in ZB (#136678 )	2024-09-27 07:07:58 +00:00
_debug.py	Flip default value for mypy disallow_untyped_defs [6/11] (#127843 )	2024-06-08 18:49:29 +00:00
_IR.py	[BE]: Update mypy to 1.11.2 (#133816 )	2024-09-16 19:44:11 +00:00
_unflatten.py	[alt] fix unroll in successive unflatten (#137646 )	2024-10-12 15:53:52 +00:00
_utils.py	[BE][Easy][18/19] enforce style for empty lines in import segments in `torch/d*/` (#129770 )	2024-08-01 04:22:50 +00:00
microbatch.py	[BE] typing for decorators - fx/_compatibility (part 1) (#134202 )	2024-08-22 17:07:33 +00:00
README.md	[pipelining] redirect README to document (#128205 )	2024-06-07 19:34:52 +00:00
schedules.py	[Pipelining] Refactor Interleaved1F1B and ZeroBubble (#137783 )	2024-10-16 03:05:14 +00:00
stage.py	[pipelining] rename prev_/next_stage vars to clarify (#137739 )	2024-10-14 20:12:18 +00:00

README.md

Pipeline Parallelism for PyTorch

torch.distributed.pipelining is a package for implementing pipeline parallelism on your model.

Our documentation is available here.