pytorch/torch/csrc/jit/python
Luca Wehrstedt 5a238eb96e Fix deadlock in Future due to lock inversion with GIL (#58382)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58382

Calling markCompleted on a Future now first acquires the Future's mutex (as usual) but then sometimes tries to acquire the GIL during the DataPtr extraction while still holding the Future's mutex. (This happens when the value passed to markCompleted is a Python object). This can cause a deadlock if someone else calls any of the other methods of Future while holding the GIL.

There are two solutions to this: avoid holding the Future's mutex when extracting DataPtrs, and avoid holding the GIL while invoking the Future's method. In this PR I'm going for the latter, because it's a very simple immediate fix, but I believe this is brittle and that we should probably also consider the former fix.
ghstack-source-id: 129105358

Test Plan: The repro in https://github.com/pytorch/pytorch/issues/58239 now doesn't deadlock.

Reviewed By: mrshenli

Differential Revision: D28472816

fbshipit-source-id: 1bc9bca426dd004f9eb2568db1ffd38f014450e2
2021-05-17 10:53:19 -07:00
..
init.cpp Fix deadlock in Future due to lock inversion with GIL (#58382) 2021-05-17 10:53:19 -07:00
init.h
module_python.h
pybind_utils.cpp Revert D27448156: irange for size_t 2021-04-03 19:14:00 -07:00
pybind_utils.h Pass reference to parent future in callbacks (#57635) 2021-05-07 03:59:18 -07:00
pybind.h Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_arg_flatten.cpp Replace all direct cdata access with THPVariable_Unpack (#55799) 2021-04-15 08:57:04 -07:00
python_arg_flatten.h Move torch/csrc/utils/hash.h to c10/util/hash.h. (#42503) 2020-08-29 17:47:00 -07:00
python_custom_class.cpp Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_custom_class.h
python_interpreter.cpp Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_ir.cpp [ONNX] Support registering custom export for prim::PythonOp from torch.autograd.Function (#55630) (#57600) 2021-05-13 13:42:49 -07:00
python_ir.h Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_ivalue.h Make DataPtr extraction in CUDAFuture faster for Python values (#56918) 2021-05-06 01:12:53 -07:00
python_sugared_value.cpp Add cuda device synchronization support in JIT (#55469) 2021-04-14 09:13:07 -07:00
python_sugared_value.h Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_tracer.cpp [Usability] Capture argument names for traced functions and modules (#51775) 2021-02-10 18:28:08 -08:00
python_tracer.h [Usability] Capture argument names for traced functions and modules (#51775) 2021-02-10 18:28:08 -08:00
python_tree_views.cpp Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
python_tree_views.h
script_init.cpp [torch.package/TorchScript] logic to enable sharing of tensors on load (#57573) 2021-05-14 08:21:43 -07:00
script_init.h
update_graph_executor_opt.cpp Make PyTorch code-base clang-tidy compliant (#56892) 2021-04-28 14:10:25 -07:00
update_graph_executor_opt.h