OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 00:21:07 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
ac3effd150
pytorch/docs/source
History
Banit Agrawal ac3effd150 [PyTorch GPU Allocator] Better use of blocks with rounding of allocation sizes (#74213) 
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/74213

In the current CUDACachingAllocator, the sizes are rounded up in multiple of blocks size of 512, so this works for smaller sizes. However for large sizes, we can have lots of different size blocks in the larger pool. This is problematic when we have variable batch sizes 1001, 1021, 1023 -> all will go to different block size and will create different size of blocks. This will create lots of unused blocks and will waste GPU memory capacity.

This diff adds a rounding approach to allocation size. It rounds up the size to nearest power-of-2 divisions and the power2-division can be changed with env variable setting.

   For example, if we need to round-up  size of1200 and if number of divisions is 4,
   the size 1200 lies between 1024 and 2048 and if we do 4 divisions between
   them, the values are 1024, 1280, 1536, and 1792. So the function will
   return 1280 as the nearest ceiling of power-2 division.

env setting:
   export PYTORCH_CUDA_ALLOC_CONF=roundup_power2_divisions:4
ghstack-source-id: 151446017

Reviewed By: ezyang

Differential Revision: D34868036

fbshipit-source-id: 494785add16e6b37c920dcb5a2b81d4c637b554a
(cherry picked from commit 548454ccacbd8700e7ffd2d762e40b4ba37abbae)
2022-03-16 02:53:53 +00:00
..
_static clarify the documentation of torch.meshgrid (#62977) 2021-08-18 04:01:22 -07:00
_templates DOC: Merge extraheader block from theme instead of override (#70187) 2022-01-05 06:42:38 -08:00
community Update persons of interest for ONNX (#72072) 2022-02-16 23:01:13 +00:00
elastic (torchelastic) make --max_restarts explicit in the quickstart and runner docs (#65838) 2021-09-29 19:29:01 -07:00
notes [PyTorch GPU Allocator] Better use of blocks with rounding of allocation sizes (#74213) 2022-03-16 02:53:53 +00:00
rpc Support Union in TorchScript (#64234) 2021-09-03 06:12:24 -07:00
scripts [docs] Add images to some activation functions (#65415) 2021-09-22 11:05:29 -07:00
amp.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
autograd.rst Targeted documentation updates in autograd.functional (#72111) 2022-02-02 03:19:31 +00:00
backends.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
benchmark_utils.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
bottleneck.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
checkpoint.rst
complex_numbers.rst Grammatical update of tech docs (#61547) 2021-07-14 14:01:59 -07:00
conf.py Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
config_mod.rst rename config module file to work with gh pages better 2022-03-10 20:41:44 +00:00
cpp_extension.rst Check clang++/g++ version when compiling CUDA extensions (#63230) 2022-02-24 08:32:32 +00:00
cpp_index.rst
cuda.rst Document torch.cuda.ExternalStream, torch.cuda.caching_allocator_alloc and torch.cuda.caching_allocator_delete (#70126) 2022-01-12 15:44:40 -08:00
cudnn_persistent_rnn.rst Remove orphan from cuDNN persistent note (#65160) 2021-09-21 11:09:47 -07:00
cudnn_rnn_determinism.rst
data.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
ddp_comm_hooks.rst [DDP Comm Hook] Add debugging communication hooks to ddp_comm_hooks.rst (#64352) 2021-09-01 17:37:19 -07:00
deploy.rst [deploy] docs (#69251) 2021-12-01 21:55:18 -08:00
distributed.algorithms.join.rst Add tutorial link (#62785) 2021-08-05 17:28:02 -07:00
distributed.elastic.rst
distributed.optim.rst [distributed][docs] Delete distributed optimimzer section from RPC and add reference to namespace docs page (#68068) 2021-11-09 15:01:54 -08:00
distributed.rst [PyTorch Distributed] Update documentation about NCCL environment variables (#74006) 2022-03-11 23:57:17 +00:00
distributions.rst [Reinstate] Wishart distribution (#70377) 2021-12-30 11:41:46 -08:00
dlpack.rst
docutils.conf
fft.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
fsdp.rst make fsdp folder to be public (#72084) 2022-02-02 15:50:14 +00:00
futures.rst
fx.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
hub.rst Add more details to the known limitations section of torchhub docs (#69970) 2021-12-16 02:43:48 -08:00
index.rst rename config module file to work with gh pages better 2022-03-10 20:41:44 +00:00
jit_builtin_functions.rst
jit_language_reference_v2.rst Add Union type to TorchScript Language Ref (#69514) 2021-12-07 12:53:54 -08:00
jit_language_reference.rst fix typos in jit_language_reference.rst (#68706) 2021-11-22 19:09:06 -08:00
jit_python_reference.rst [JIT] improve documentation (#57991) 2021-05-19 11:47:32 -07:00
jit_unsupported.rst
jit.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
linalg.rst [Array API] Add linalg.diagonal (#70599) 2022-01-26 08:08:32 +00:00
math-quantizer-equation.png
mobile_optimizer.rst
model_zoo.rst
monitor.rst torch/monitor: merge Interval and FixedCount stats (#72009) 2022-01-30 23:21:59 +00:00
multiprocessing.rst
name_inference.rst Abladawood patch 1 (#58496) 2021-05-20 10:32:18 -07:00
named_tensor.rst
nested.rst Minimal NestedTensor (#72881) 2022-03-02 16:31:51 +00:00
nn.functional.rst Revert D34154832: [pytorch][PR] Add multi_head_attention_forward to functional rst docs 2022-02-11 05:08:46 +00:00
nn.init.rst
nn.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
onnx.rst [ONNX] Minor doc update (#69501) (#69550) 2022-02-11 22:05:15 +00:00
optim.rst To add SequentialLR to PyTorch Core Schedulers (#64037) 2021-09-09 09:36:32 -07:00
package.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
pipeline.rst Minor changes in documentation (#68557) 2021-11-18 17:57:16 -08:00
profiler.rst Add low level torch.profiler.kineto_profile base class (#63302) 2021-12-14 14:47:43 -08:00
quantization-support.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
quantization.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
random.rst
rpc.rst Add note in RPC docs about retries. (#73601) 2022-03-03 00:29:31 +00:00
sparse.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
special.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
storage.rst
tensor_attributes.rst fix wrong indexing of class names in docs 2022-03-02 22:21:21 +00:00
tensor_view.rst Correcting a minor typo: "Users should pay" instead of "Users should be pay" (#72500) 2022-02-08 23:08:25 +00:00
tensorboard.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
tensors.rst pytorch: fix typo in quantization docs (#73511) 2022-02-28 23:11:52 +00:00
testing.rst promote torch.testing to stable (#73348) 2022-02-25 06:30:31 +00:00
torch.ao.ns._numeric_suite_fx.rst Quantization docs: add pages for Numeric Suite (Eager and FX) (#66380) 2021-10-11 18:47:58 -07:00
torch.ao.ns._numeric_suite.rst Quantization docs: add pages for Numeric Suite (Eager and FX) (#66380) 2021-10-11 18:47:58 -07:00
torch.overrides.rst
torch.rst Cleanup all module references in doc (#73983) 2022-03-10 22:26:29 +00:00
type_info.rst [Docs] Mention torch.bfloat16 in torch.finfo (#68496) 2021-11-18 17:52:41 -08:00