mirror of
https://github.com/zebrajr/pytorch.git
synced 2025-12-06 12:20:52 +01:00
d22c4cc353
MemPool is a separate pool of memory handled by the caching allocator. This PR adds the option let the caching allocator try to use this pool as a last resort instead of OOMing by associating a use_on_oom bool with each MemPool.
Usage:
Users can optionally specify a ``use_on_oom`` bool (which is False by default) during MemPool creation. If true, then the CUDACachingAllocator will be able to use memory in this pool as a last resort instead of OOMing.
```
pool = torch.cuda.MemPool(allocator, use_on_oom=True)
with torch.cuda.use_mem_pool(pool):
a = torch.randn(40 * 1024 * 1024, dtype=torch.uint8, device="cuda")
del a
# at the memory limit, this will succeed by using pool's memory in order to avoid the oom
b = torch.randn(40 * 1024 * 1024, dtype=torch.uint8, device="cuda")
```
Testing:
```
python test/test_cuda.py -k test_mempool_limited_memory_with_allocator
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/151487
Approved by: https://github.com/eqy, https://github.com/syed-ahmed, https://github.com/ngimel
|
||
|---|---|---|
| .. | ||
| _dynamo | ||
| __init__.pyi.in | ||
| _aoti.pyi | ||
| _autograd.pyi | ||
| _cpu.pyi | ||
| _cudnn.pyi | ||
| _cusparselt.pyi | ||
| _distributed_autograd.pyi | ||
| _distributed_c10d.pyi | ||
| _distributed_rpc_testing.pyi | ||
| _distributed_rpc.pyi | ||
| _export.pyi | ||
| _functions.pyi | ||
| _functorch.pyi | ||
| _instruction_counter.pyi | ||
| _itt.pyi | ||
| _lazy_ts_backend.pyi | ||
| _lazy.pyi | ||
| _monitor.pyi | ||
| _nn.pyi.in | ||
| _nvtx.pyi | ||
| _onnx.pyi | ||
| _profiler.pyi | ||
| _VariableFunctions.pyi.in | ||
| _verbose.pyi | ||
| build.bzl | ||
| return_types.pyi.in | ||