mirror of
https://github.com/zebrajr/pytorch.git
synced 2025-12-07 12:21:27 +01:00
91ea2cd5a7
Summary: This PR addresses some numerical issues of Sigmoid/StickBreakingTransform, where these transforms give +-inf when the unconstrained values move to +-20 areas. For example, with ``` t = torch.distributions.SigmoidTransform() x = torch.tensor(20.) t.inv(t(x)), t.log_abs_det_jacobian(x, t(x)) ``` current behaviour the inverse will return `inf` and logdet return `-inf` while this PR makes it to `15.9424` and `-15.9424`. And for ``` t = torch.distributions.StickBreakingTransform() x = torch.tensor([20., 20.]) t.inv(t(x)), t.log_abs_det_jacobian(x, t(x)) ``` current value is `(inf, nan)` and `-inf` for logdet, while this PR makes it `[16.6355, 71.3942]` and `-47.8272` for logdet. Although these finite values are wrong and seems unavoidable, it is better than returning `inf` or `nan` in my opinion. This is useful in HMC where despite that the grad will be zero when the unconstrained parameter moves to unstable area (due to clipping), velocity variable will force the parameter move to another area which by chance can move the parameter out of unstable area. But inf/nan can be useful to stop doing inference early. So the changes in this PR might be inappropriate. I also fix some small issues of `_Simplex` and `_RealVector` constraints where batch shape of the input is not respected when checking validation. Pull Request resolved: https://github.com/pytorch/pytorch/pull/20288 Differential Revision: D15742047 Pulled By: ezyang fbshipit-source-id: b427ed1752c41327abb3957f98d4b289307a7d17 |
||
|---|---|---|
| .. | ||
| bottleneck | ||
| cpp | ||
| cpp_extensions | ||
| custom_operator | ||
| data | ||
| error_messages | ||
| expect | ||
| jit | ||
| onnx | ||
| optim | ||
| test_module | ||
| common_cuda.py | ||
| common_methods_invocations.py | ||
| common_nn.py | ||
| common_utils.py | ||
| expecttest.py | ||
| HowToWriteTestsUsingFileCheck.md | ||
| hypothesis_utils.py | ||
| jit_utils.py | ||
| run_test.py | ||
| test_autograd.py | ||
| test_c10d.py | ||
| test_cpp_extensions.py | ||
| test_cuda_primary_ctx.py | ||
| test_cuda.py | ||
| test_dataloader.py | ||
| test_distributed.py | ||
| test_distributions.py | ||
| test_docs_coverage.py | ||
| test_expecttest.py | ||
| test_fake_quant.py | ||
| test_indexing_cuda.py | ||
| test_indexing.py | ||
| test_jit_fuser.py | ||
| test_jit_py3.py | ||
| test_jit.py | ||
| test_logging.py | ||
| test_mkldnn.py | ||
| test_multiprocessing_spawn.py | ||
| test_multiprocessing.py | ||
| test_namedtensor.py | ||
| test_namedtuple_return_api.py | ||
| test_nccl.py | ||
| test_nn_quantized.py | ||
| test_nn.py | ||
| test_numba_integration.py | ||
| test_optim.py | ||
| test_quantized.py | ||
| test_quantizer.py | ||
| test_sparse.py | ||
| test_tensorboard.py | ||
| test_thd_distributed.py | ||
| test_torch.py | ||
| test_type_hints.py | ||
| test_type_info.py | ||
| test_utils.py | ||