PyTorch MergeBot
c006c8b50e
Revert "markDynamoStrictTest some more ( #115885 )"
...
This reverts commit 55ce4693ffhttps://github.com/pytorch/pytorch/pull/115885 on behalf of https://github.com/atalman due to OSSCI oncall, broke inductor ([comment](https://github.com/pytorch/pytorch/pull/115885#issuecomment-1858409669 ))
2023-12-15 19:51:24 +00:00
rzou
55ce4693ff
markDynamoStrictTest some more ( #115885 )
...
Featuring
test_native_mha.py
test_nn.py
test_prims.py
test_schema_check.py
test_serialization.py
test_show_pickle.py
test_sort_and_select.py
Pull Request resolved: https://github.com/pytorch/pytorch/pull/115885
Approved by: https://github.com/voznesenskym
ghstack dependencies: #115845 , #115855 , #115856 , #115857 , #115858 , #115870 , #115871 , #115879
2023-12-15 13:19:52 +00:00
Driss Guessous
e24ce484ed
Use scaled_dot_product_attention within attention.cpp ( #87312 )
...
# Summary
Use the private _scaled_dot_product_attention to support _native_multiheaded_attention. _SDP provides access to fused kernels when certain conditions are meant enabling a speed up for MHA.
cc @cpuhrsch @jbschlosser @bhosmer @mikaylagawarecki
Pull Request resolved: https://github.com/pytorch/pytorch/pull/87312
Approved by: https://github.com/cpuhrsch
2022-10-31 04:06:31 +00:00
Mikayla Gawarecki
afaee00fec
Add python nested_tensor and as_nested_tensor constructors in torch.nested ( #85593 )
...
Remove `torch.nested_tensor` which has erroneous behavior wrt gradients (could be either leaf or not leaf). Introduce `torch.nested.nested_tensor` and `torch.nested.as_nested_tensor` in the vein of `torch.tensor` and `torch.as_tensor`. Done in nested `__init__.py` for now but can move to pybind in future (when we want to load from numpy/nested lists ).
Discussed offline with @cpuhrsch and pybind constructor (https://github.com/pytorch/pytorch/pull/85536 ) was more gnarly than expected, so we can move to that when we do need loading from numpy etc.
Differential Revision: [D39806622](https://our.internmc.facebook.com/intern/diff/D39806622 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85593
Approved by: https://github.com/drisspg , https://github.com/cpuhrsch
2022-09-28 20:15:02 +00:00
Mikayla Gawarecki
77f1f98479
Re-introduce torch.Tensor.to_padded_tensor ( #85293 )
...
Differential Revision: [D39629004](https://our.internmc.facebook.com/intern/diff/D39629004 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85293
Approved by: https://github.com/cpuhrsch
2022-09-21 18:45:56 +00:00
Mikayla Gawarecki
e217b30b0f
Add torch.nested namespace ( #84102 )
...
First step towards #83775
- only `to_padded_tensor` is moved to the nested namespace for now
- following the schema used for `special`, `fft`, `linalg` and other namespaces, nested functions are registered in native_functions.yaml as `nested_{function_name}` and are bound to the desired Python name in
`torch/nested/__init__.py`, and the desired C++ name in `torch/csrc/api/include/torch/nested.h`.
~~**Question**: should we keep the documentation for `Tensor.to_padded_tensor` or can this deleted since it is shared by `torch.nested.to_padded_tensor`?~~
[generated nested docs](https://docs-preview.pytorch.org/84102/nested.html?highlight=nested#module-torch.nested )
Differential Revision: [D39361148](https://our.internmc.facebook.com/intern/diff/D39361148 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/84102
Approved by: https://github.com/drisspg
2022-09-12 16:31:05 +00:00
Nicolas Macchioni
b236352036
Add mask identifier for multiplexed src_mask/src_key_padding_mask in BT ( #81947 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81947
Transformer fastpath multiplexes two arguments, src_mask [seq_len x seq_len] and src_key_padding_mask [batch_size x seq_len], and later deduces the type based on mask shape.
In the event that batch_size == seq_len, any src_mask is wrongly interpreted as a src_key padding_mask. This is fixed by requiring a mask_type identifier be supplied whenever batch_size == seq_len.
Additionally, added support for src_mask in masked_softmax CPU path.
Test Plan: existing unit tests + new unit tests (batch_size == seq_len)
Differential Revision: D37932240
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81947
Approved by: https://github.com/zrphercule
2022-08-09 23:42:16 +00:00
Scott Wolchok
7e4730d017
[PyTorch] Round T up to next multiple of 8 in NestedTensor case
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77903
Code comment should explain why; in brief, it lets us use Tensor cores.
Differential Revision: [D36527773](https://our.internmc.facebook.com/intern/diff/D36527773/ )
Approved by: https://github.com/ngimel , https://github.com/cpuhrsch
2022-05-25 20:34:19 +00:00
Scott Wolchok
e816e17655
[PyTorch] Add native fast path for transformer encoder inference ( #76333 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76333
The current PyTorch multi-head attention and transformer
implementations are slow. This should speed them up for inference.
ghstack-source-id: 154737857
(Note: this ignores all push blocking failures!)
Test Plan: CI
Reviewed By: cpuhrsch
Differential Revision: D35239925
fbshipit-source-id: 5a7eb8ff79bc6afb4b7d45075ddb2a24a6e2df28
2022-04-26 12:58:03 -04:00
Jon Janzen
2387efd356
Revert "[PyTorch] Add native fast path for transformer encoder inference"
...
This reverts commit b369b89f23https://github.com/pytorch/pytorch/pull/75809#issuecomment-1108717166
2022-04-25 11:40:02 -04:00
Scott Wolchok
b369b89f23
[PyTorch] Add native fast path for transformer encoder inference
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/75809
The current PyTorch multi-head attention and transformer
implementations are slow. This should speed them up for inference.
Differential Revision: [D35239925](https://our.internmc.facebook.com/intern/diff/D35239925/ )
**NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D35239925/ )!
Approved by: https://github.com/ezyang
2022-04-25 06:11:36 +00:00
