Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/16373
motivation: https://github.com/pytorch/pytorch/pull/12407
This is a manual diff.
most of the fixes should be:
```
auto* Y = Output(0);
Y->Resize(dims);
Y->raw_mutable_data(dtype);
```
-->
```
auto* Y = Output(0, dims, at::dtype(dtype));
```
But there might be other cases.
Reviewed By: dzhulgakov
Differential Revision: D13725460
fbshipit-source-id: 649a4b0e42f62cda1a60171dd9fa3e440dc9dca1
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/9939
Pull Request resolved: https://github.com/facebookresearch/weakly-supervised-action-detection/pull/13
Pull Request resolved: https://github.com/pytorch/translate/pull/166
Pull Request resolved: https://github.com/pytorch/pytorch/pull/9125
Closes https://github.com/pytorch/pytorch/pull/9125
Use inheritance for polymorphism, and remove template parameter
This is to change the templating in call sites, the core implementations will change later
Before Caffe2 Tensor class was compile-time fixed to bind to a particular device/context. With this change, we're making it a runtime property (stored inside the tensor), but preserve the same semantics. For example, one has to specify device type in order to create a Tensor - there are no uninitialized tensors. More specifically the changes are:
1. We added an extra argument *DeviceType* to most of the constructors of the tensor, e.g. (Tensor(DeviceType type)),
2. Semantics of constructor Tensor(const Tensor<SrcContext>& src, ContextForCopy* context); is changed, in this constructor, the second context is passed in to enable us to call the templated Copy function, it could be in a different context as source and target previously, now we'll enforce that the context should have same device type as src, if it is provided.
3. To preserve 'get-or-construct' semantics of Blob, we added specialized getter Blob::GetMutableTensor that verifies both that Blob contains a Tensor and that it's of a correct type
4. Specifically, Tensor type is not default-constructible any more (as we don't have unknown device tensors) and thus some of the code handling STL containers needs to change
Note: Some changes are postponed just to keep this diff a bit smaller. Please see `TODO`s.
Reviewed By: ezyang, houseroad
Differential Revision: D9024330
fbshipit-source-id: e0b8295d2dc6ebe2963383ded5af799ad17164ba
Summary:
Pull Request resolved: https://github.com/facebookresearch/weakly-supervised-action-detection/pull/13
Pull Request resolved: https://github.com/pytorch/translate/pull/166
Pull Request resolved: https://github.com/pytorch/pytorch/pull/9125
Closes https://github.com/pytorch/pytorch/pull/9125
Use inheritance for polymorphism, and remove template parameter
This is to change the templating in call sites, the core implementations will change later
Before Caffe2 Tensor class was compile-time fixed to bind to a particular device/context. With this change, we're making it a runtime property (stored inside the tensor), but preserve the same semantics. For example, one has to specify device type in order to create a Tensor - there are no uninitialized tensors. More specifically the changes are:
1. We added an extra argument *DeviceType* to most of the constructors of the tensor, e.g. (Tensor(DeviceType type)),
2. Semantics of constructor Tensor(const Tensor<SrcContext>& src, ContextForCopy* context); is changed, in this constructor, the second context is passed in to enable us to call the templated Copy function, it could be in a different context as source and target previously, now we'll enforce that the context should have same device type as src, if it is provided.
3. To preserve 'get-or-construct' semantics of Blob, we added specialized getter Blob::GetMutableTensor that verifies both that Blob contains a Tensor and that it's of a correct type
4. Specifically, Tensor type is not default-constructible any more (as we don't have unknown device tensors) and thus some of the code handling STL containers needs to change
Note: Some changes are postponed just to keep this diff a bit smaller. Please see `TODO`s.
Reviewed By: xw285cornell
Differential Revision: D8121878
fbshipit-source-id: 4a5e9a677ba4ac82095df959851a054c81eccf81
Summary:
* Likely need to test this so bad formatting can't be added in the future, but cleaning all operators so we at least have good examples.
* Formatting between our internal Facebook operator catalog and external caffe2.ai catalog are still slightly different. We'll work on this.
Closes https://github.com/caffe2/caffe2/pull/1846
Reviewed By: pjh5
Differential Revision: D6848570
Pulled By: orionr
fbshipit-source-id: b9bc0bfccb243d0440bd7b2406858cad8dc37e92
Summary: Allow the application of sequence-length masking to be replicated along one or more minor axes. See task for details.
Reviewed By: jamesr66a
Differential Revision: D6090835
fbshipit-source-id: 9064232aa9b93246c582b6e0bae73be5dbe09e98
Summary:
Added support for batching to SequenceMaskOp.
Let b be the batch dim and k be the axis dim. (We enforce that b < k.) Write the dimensions of the input tensor as [a_1, ..., a_b, ..., a_k, ...]. We first collapse our tensor down to 3D, with dimensions [P, Q, D], where: P = a_1 * ... * a_b, Q=a_{b+1} * ... * a_{k-1}, and D=a_k * a_{k+1} * ... * a_n. Then we mask each slice [i, :, : ] of this 3D tensor (note that each slice is a Q times D tensor w/ dimension 2)
Reviewed By: jamesr66a
Differential Revision: D5733382
fbshipit-source-id: e7a314d9fe6e6691a75112edbee8ba6e8ea8e396
Summary:
This exhibits the problem in NMT training where some out of bound data seems to
have silently written over bound, and causing random segfaults elsewhere in the
code. This itself does not solve the problem, but will trigger us to then fix the out
of bound issues.
Differential Revision: D5832646
fbshipit-source-id: 5eb259e4584e5341ef3f19362f98f0a9554e9aec
Summary: A bit safer, and also suppresses compiler warning.
Reviewed By: bddppq
Differential Revision: D5803080
fbshipit-source-id: d8c782c936a8fdaded4ae209b212378e78606ffb
Summary: This can be used for local attention to mask elements outside of a window
Reviewed By: jamesr66a
Differential Revision: D5643677
fbshipit-source-id: 92b33866258ccc7307d5bcf08234610aa3fb152d
Summary:
Implement forward pass for a SequenceMaskOp to replace https://github.com/caffe2/caffe2/blob/master/caffe2/python/attention.py#L54-L72.
This implements two modes: a sequence-length based mode and a matrix triangle mode.
Reviewed By: akyrola
Differential Revision: D5615493
fbshipit-source-id: a2ce4a8e655d9b720049010a7856be052c5567eb
Summary: The booleanmask supports another output with sorted indices
Differential Revision: D4984255
fbshipit-source-id: becb10d7fe989bb2f6488c901766a45369613eb7
Summary:
A CPU implementation for unmask operator in caffe2.
There's also a small bug in mask operator, fix it as well.
Reviewed By: ender-wieczorek
Differential Revision: D4896351
fbshipit-source-id: 887d1beb66fe93ea2da1c4e165fce2e026907726
