Summary:
**Expected**: When I run `torch.distributions.HalfCauchy(torch.tensor(1.0), validate_args=True).log_prob(-1)`, I expect a `ValueErro` because that is the behavior of other distributions (e.g. Beta, Bernoulli).
**Actual**: No run-time error is thrown, but a `-inf` log prob is returned.
Fixes https://github.com/pytorch/pytorch/issues/50404
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/pytorch/pytorch/50403)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50403
Reviewed By: mrshenli
Differential Revision: D25907131
Pulled By: neerajprad
fbshipit-source-id: ceb63537e5850809c8b32cf9db0c99043f381edf
Summary:
After we merged https://github.com/pytorch/pytorch/pull/48743, we noticed that some existing code that subclasses `torch.Distribution` started throwing `NotImplemenedError` since the constraints required for validation checks were not implemented.
```sh
File "torch/distributions/distribution.py", line 40, in __init__
for param, constraint in self.arg_constraints.items():
File "torch/distributions/distribution.py", line 92, in arg_constraints
raise NotImplementedError
```
This PR throws a UserWarning for such cases instead and gives a better warning message.
cc. Balandat
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50302
Reviewed By: Balandat, xuzhao9
Differential Revision: D25857315
Pulled By: neerajprad
fbshipit-source-id: 0ff9f81aad97a0a184735b1fe3a5d42025c8bcdf
Summary:
Fixes https://github.com/pytorch/pytorch/issues/47123
Follows https://github.com/pyro-ppl/pyro/pull/2701
This turns on `Distribution` validation by default. The motivation is to favor beginners by providing helpful error messages. Advanced users focused on speed can disable validation by calling
```py
torch.distributions.Distribution.set_default_validate_args(False)
```
or by disabling individual distribution validation via `MyDistribution(..., validate_args=False)`.
In practice I have found many beginners forget or do not know about validation. Therefore I have [enabled it by default](https://github.com/pyro-ppl/pyro/pull/2701) in Pyro. I believe PyTorch could also benefit from this change. Indeed validation caught a number of bugs in `.icdf()` methods, in tests, and in PPL benchmarks, all of which have been fixed in this PR.
## Release concerns
- This may slightly slow down some models. Concerned users may disable validation.
- This may cause new `ValueErrors` in models that rely on unsupported behavior, e.g. `Categorical.log_prob()` applied to continuous-valued tensors (only {0,1}-valued tensors are supported).
We should clearly note this change in release notes.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/48743
Reviewed By: heitorschueroff
Differential Revision: D25304247
Pulled By: neerajprad
fbshipit-source-id: 8d50f28441321ae691f848c55f71aa80cb356b41
Summary:
Fixes https://github.com/pytorch/pytorch/issues/44530
As explained in the issue description, CatTransform does not work with event_dim > 0.
This PR fixes this. If this gets approved I am hoping to do the same for StackTransform as well.
fritzo Can you take a look at this ?
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49111
Reviewed By: neerajprad
Differential Revision: D25526005
Pulled By: ezyang
fbshipit-source-id: e14430093f550d5e0da7a311f9cd44796807830f
Summary:
As a follow up to https://github.com/pytorch/pytorch/issues/48041, this adds the `LKJCholesky` distribution that samples the Cholesky factor of positive definite correlation matrices.
This also relaxes the check on `tril_matrix_to_vec` so that it works for 2x2 matrices with `diag=-2`.
cc. fehiepsi
Pull Request resolved: https://github.com/pytorch/pytorch/pull/48798
Reviewed By: zhangguanheng66
Differential Revision: D25364635
Pulled By: neerajprad
fbshipit-source-id: 4abf8d83086b0ad45c5096760114a2c57e555602
Summary:
This adds a transform to convert a real vector of (D * (D-1))/2 dimension into the cholesky factor of a D x D correlation matrix. This follows the implementation in [NumPyro](https://github.com/pyro-ppl/numpyro/blob/master/numpyro/distributions/transforms.py) by fehiepsi. This is needed for the LKJDistribution which will be added in a subsequent PR.
Also in line with the ongoing effort to refactor distributions test, this moves the transforms test into its own file that uses pytest with parametrized fixtures.
For review:
fehiepsi - could you help review the math?
fritzo - do you have any suggestions for what to do about the event dimension (more details are in the comment below)?
ezyang - could you review the changes in `run_test.py`? Instead of a separate `PYTEST_TESTS`, I have clubbed these tests in `USE_PYTEST_LIST` to avoid duplicate logic. The only difference is that we do not anymore check if pytest is not installed and exclude the tests in the list. I figured that if existing tests are already using pytest, this should not matter.
TODOs (probably not all can be satisfied at the same time):
- [x] Use operations that are JIT friendly, i.e. the transform works with different sized input under JIT.
- [x] Resolve test failures - currently `arange(scalar_tensor)` fails on certain backends but this is needed for JIT. Maybe we should only support same sized tensor under JIT?
- [x] Add tests to check that the transform gives correct gradients and is in agreement with the `log_det_jacobian`.
- [x] Add `input_event_dim` and `output_event_dim` to `CorrCholeskyTransform`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/48041
Reviewed By: zhangguanheng66
Differential Revision: D25262505
Pulled By: neerajprad
fbshipit-source-id: 5a57e1c19d8230b53592437590b9169bdf2f71e9
Summary:
Add reparameterization support to the `OneHotCategorical` distribution. Samples are reparameterized based on the straight-through gradient estimator, which is proposed in the paper [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://arxiv.org/abs/1308.3432).
Pull Request resolved: https://github.com/pytorch/pytorch/pull/46610
Reviewed By: neerajprad
Differential Revision: D25272883
Pulled By: ezyang
fbshipit-source-id: 8364408fe108a29620694caeac377a06f0dcdd84
Summary:
In response to https://github.com/pytorch/pytorch/issues/11578. This is a test run to see if CI (and other internal systems) works fine with pytest style tests.
- Creates a separate `distributions` directory within `test`.
- For testing, this rewrites the `constraint` tests as parameterized tests in pytest. I don't plan to convert any other tests to pytest style, but only expose this option for adding new tests, if required.
If this is a success, we can move `EXAMPLES` in `test_distributions` into a separate file that can be imported by both pytest and unittest style tests. cc. fritzo
Pull Request resolved: https://github.com/pytorch/pytorch/pull/45648
Reviewed By: ezyang, colesbury
Differential Revision: D24080248
Pulled By: neerajprad
fbshipit-source-id: 1f2e7d169c3c291a3051d0cece17851560fe9ea9
