Mikayla Gawarecki
487b6d40ec
Add RMSNorm module ( #121364 )
...
Similar to dbeed9724b/torchmultimodal/modules/layers/normalizations.py (L51)dbeed9724b/torchmultimodal/modules/layers/normalizations.py (L73)https://our.internmc.facebook.com/intern/diff/ )
Differential Revision: [D55485840](https://our.internmc.facebook.com/intern/diff/D55485840 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/121364
Approved by: https://github.com/albanD
2024-03-29 18:05:28 +00:00
PyTorch MergeBot
8698121636
Revert "Add RMSNorm module ( #121364 )"
...
This reverts commit a7306de0dchttps://github.com/pytorch/pytorch/pull/121364 on behalf of https://github.com/atalman due to Broke internal tests ([comment](https://github.com/pytorch/pytorch/pull/121364#issuecomment-2025502007 ))
2024-03-28 15:31:10 +00:00
Mikayla Gawarecki
a7306de0dc
Add RMSNorm module ( #121364 )
...
Similar to dbeed9724b/torchmultimodal/modules/layers/normalizations.py (L51)dbeed9724b/torchmultimodal/modules/layers/normalizations.py (L73)https://github.com/pytorch/pytorch/pull/121364
Approved by: https://github.com/albanD
2024-03-27 21:39:30 +00:00
Wongboo
68f74dd162
Add python and C++ support for LPPool3d ( #114199 )
...
Add python and C++ support for LPPool3d to Fixes #114114
Pull Request resolved: https://github.com/pytorch/pytorch/pull/114199
Approved by: https://github.com/mikaylagawarecki
2023-12-08 18:18:44 +00:00
lkct
fd4beb7a05
Better function annotations for nn.functional ( #102918 )
...
Fixes #102768
- Provides proper function declarations in generated `torch/nn/functional.pyi`.
- Moves some functions from manually defined in `functional.pyi.in` to generated code, in order to single-source the signature.
- Includes some of the functions in `torch._C._nn` into its `.pyi.in`, but not exhaustive (only what's already there).
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102918
Approved by: https://github.com/drisspg , https://github.com/malfet
2023-06-16 19:48:04 +00:00
Xuehai Pan
1fd119948e
[3/3] Update .pyi Python stub files and enable 'UFMT' linter ( #95268 )
...
Changes:
- #95200
1. Recognize `.py.in` and `.pyi.in` files as Python in VS Code for a better development experience.
2. Fix deep setting merge in `tools/vscode_settings.py`.
- #95267
3. Use `Namedtuple` rather than `namedtuple + __annotations__` for `torch.nn.utils.rnn.PackedSequence_`:
`namedtuple + __annotations__`:
```python
PackedSequence_ = namedtuple('PackedSequence_',
['data', 'batch_sizes', 'sorted_indices', 'unsorted_indices'])
# type annotation for PackedSequence_ to make it compatible with TorchScript
PackedSequence_.__annotations__ = {'data': torch.Tensor, 'batch_sizes': torch.Tensor,
'sorted_indices': Optional[torch.Tensor],
'unsorted_indices': Optional[torch.Tensor]}
```
`Namedtuple`: Python 3.6+
```python
class PackedSequence_(NamedTuple):
data: torch.Tensor
batch_sizes: torch.Tensor
sorted_indices: Optional[torch.Tensor]
unsorted_indices: Optional[torch.Tensor]
```
- => this PR: #95268
4. Sort import statements and remove unnecessary imports in `.pyi`, `.pyi.in` files.
5. Format `.pyi`, `.pyi.in` files and remove unnecessary ellipsis `...` in type stubs.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/95268
Approved by: https://github.com/huydhn
2023-03-01 23:50:56 +00:00
Michael Gschwind
7265f60ad0
Regularize mask handling for attn_mask and key_padding_mask ( #92733 )
...
Summary:
Regularize mask handling for attn_mask and key_padding_mask
* Update documentation to remove reference to byte masks (which were deprecated long ago)
* Introduce check and warn about deprecation if attn_mask and key_padding_mask types mismatch
* Convert all masks to float before combining
* Combine by adding
Test Plan: sandcastle & github CI
Differential Revision: D42653215
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92733
Approved by: https://github.com/ngimel , https://github.com/drisspg
2023-01-24 14:12:05 +00:00
Michael Gschwind
512ec181ec
Introduce causal mask ( #90508 )
...
Summary: Introduce causal mask
This PR introduces a causal mask option _causal_mask (as well as causal mask detection if attn_mask is provided), since current custom kernels do not support arbitrary masks.
Test Plan: sandcastle & github ci/cd
Differential Revision: D41723137
Pull Request resolved: https://github.com/pytorch/pytorch/pull/90508
Approved by: https://github.com/albanD
2022-12-16 21:39:42 +00:00
Joel Benjamin Schlosser
2d73c8e6e0
Add Dropout1d module
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79545
Approved by: https://github.com/ngimel , https://github.com/albanD
2022-06-15 14:39:07 +00:00
Yuge Zhang
3ac27e78ca
Fix typehint of multi_head_attention_forward
...
Fixes #76169
Pull Request resolved: https://github.com/pytorch/pytorch/pull/76170
Approved by: https://github.com/jbschlosser
2022-04-27 13:47:43 +00:00
Ryan Spring
4f8b986e28
Implement Tanh Gelu Approximation ( #61439 )
...
Summary:
1. Implements https://github.com/pytorch/pytorch/issues/39853
2. Adds approximate boolean flag to Gelu
3. Enables Tanh Gelu approximation
4. Adds double backward support for Gelu
5. Enable Tanh Gelu in NvFuser
```
def gelu(x, approximate : str = 'none'):
if approximate == 'tanh':
# sqrt(2/pi) = 0.7978845608028654
return 0.5 * x * (1.0 + torch.tanh(0.7978845608028654 * (x + 0.044715 * torch.pow(x, 3.0))))
else:
return x * normcdf(x)
```
Linking XLA PR - https://github.com/pytorch/xla/pull/3039
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61439
Reviewed By: VitalyFedyunin
Differential Revision: D33894937
Pulled By: jbschlosser
fbshipit-source-id: b65e8fb6ea66168af8f34f45ed50e92737a33851
(cherry picked from commit 6e986f91a9
2022-02-14 03:40:32 +00:00
Nikita Shulga
cedf37d933
Back out "Revert D34043182: [pytorch][PR] Added missing antialias argument to functional.pyi.in"
...
Summary:
Original commit changeset: 4ce347cb0f30
Original Phabricator Diff: D34043182 (8315c9b885046b1dbb72
2022-02-08 01:37:22 +00:00
Nikita Shulga
127bf42ee7
Revert D34043182: [pytorch][PR] Added missing antialias argument to functional.pyi.in
...
Test Plan: revert-hammer
Differential Revision:
D34043182 (8315c9b8858315c9b88593eef03aa6
2022-02-08 01:11:39 +00:00
vfdev-5
8315c9b885
Added missing antialias argument to functional.pyi.in ( #72420 )
...
Summary:
Description:
- Added missing antialias argument to functional.pyi.in
- mypy is happy if checking `interpolate` method with antialias argument
Related torchvision issue: https://github.com/pytorch/vision/pull/5329
Pull Request resolved: https://github.com/pytorch/pytorch/pull/72420
Reviewed By: mruberry
Differential Revision: D34043182
Pulled By: albanD
fbshipit-source-id: ca64a8f0d2516c1be5b060c1c24e0b1ed2127b96
(cherry picked from commit 7c8a90cbfa
2022-02-07 20:44:59 +00:00
Nikita Shulga
74c44ba9d6
Revert D33850228: [pytorch][PR] Implement Tanh Gelu Approximation
...
Test Plan: revert-hammer
Differential Revision:
D33850228 (23d03025dc23d03025dcc9efb58223
2022-01-31 17:44:19 +00:00
Ryan Spring
23d03025dc
Implement Tanh Gelu Approximation ( #61439 )
...
Summary:
1. Implements https://github.com/pytorch/pytorch/issues/39853
2. Adds approximate boolean flag to Gelu
3. Enables Tanh Gelu approximation
4. Adds double backward support for Gelu
5. Enable Tanh Gelu in NvFuser
```
def gelu(x, approximate : str = 'none'):
if approximate == 'tanh':
# sqrt(2/pi) = 0.7978845608028654
return 0.5 * x * (1.0 + torch.tanh(0.7978845608028654 * (x + 0.044715 * torch.pow(x, 3.0))))
else:
return x * normcdf(x)
```
Linking XLA PR - https://github.com/pytorch/xla/pull/3039
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61439
Reviewed By: cpuhrsch
Differential Revision: D33850228
Pulled By: jbschlosser
fbshipit-source-id: 3cc33fb298e480d7ecc5c67716da019d60c6ab33
(cherry picked from commit 3a53b3e94f
2022-01-31 17:07:45 +00:00
Joel Schlosser
cb823d9f07
Revert D33744717: [pytorch][PR] Implement Tanh Gelu Approximation
...
Test Plan: revert-hammer
Differential Revision:
D33744717 (f499ab9ceff499ab9cefe9fb2d1db1
2022-01-28 18:35:01 +00:00
Ryan Spring
f499ab9cef
Implement Tanh Gelu Approximation ( #61439 )
...
Summary:
1. Implements https://github.com/pytorch/pytorch/issues/39853
2. Adds approximate boolean flag to Gelu
3. Enables Tanh Gelu approximation
4. Adds double backward support for Gelu
5. Enable Tanh Gelu in NvFuser
```
def gelu(x, approximate : str = 'none'):
if approximate == 'tanh':
# sqrt(2/pi) = 0.7978845608028654
return 0.5 * x * (1.0 + torch.tanh(0.7978845608028654 * (x + 0.044715 * torch.pow(x, 3.0))))
else:
return x * normcdf(x)
```
Linking XLA PR - https://github.com/pytorch/xla/pull/3039
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61439
Reviewed By: mikaylagawarecki
Differential Revision: D33744717
Pulled By: jbschlosser
fbshipit-source-id: d64532a562ed53247bb4fa52bb16722634d5c187
(cherry picked from commit 4713dd9cca
2022-01-28 16:59:09 +00:00
Joel Schlosser
e6befbe85c
Add flag to optionally average output attention weights across heads ( #70055 )
...
Summary:
Fixes https://github.com/pytorch/pytorch/issues/47583
Pull Request resolved: https://github.com/pytorch/pytorch/pull/70055
Reviewed By: bhosmer
Differential Revision: D33457866
Pulled By: jbschlosser
fbshipit-source-id: 17746b3668b0148c1e1ed8333227b7c42f1e3bf5
2022-01-06 17:32:37 -08:00
Pearu Peterson
48771d1c7f
[BC-breaking] Change dtype of softmax to support TorchScript and MyPy ( #68336 )
...
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/68336
Test Plan: Imported from OSS
Reviewed By: VitalyFedyunin
Differential Revision: D32470965
Pulled By: cpuhrsch
fbshipit-source-id: 254b62db155321e6a139bda9600722c948f946d3
2021-11-18 11:26:14 -08:00
vfdev
62ca5a81c0
Exposed recompute_scale_factor into nn.Upsample ( #66419 )
...
Summary:
Description:
- Exposed recompute_scale_factor into nn.Upsample such that recompute_scale_factor=True option could be used
Context: https://github.com/pytorch/pytorch/pull/64501#discussion_r710205190
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66419
Reviewed By: gchanan
Differential Revision: D31731276
Pulled By: jbschlosser
fbshipit-source-id: 2118489e6f5bc1142f2a64323f4cfd095a9f3c42
2021-10-20 07:59:25 -07:00
Thomas J. Fan
d3bcba5f85
ENH Adds label_smoothing to cross entropy loss ( #63122 )
...
Summary:
Fixes https://github.com/pytorch/pytorch/issues/7455
Partially resolves pytorch/vision#4281
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63122
Reviewed By: iramazanli
Differential Revision: D30586076
Pulled By: jbschlosser
fbshipit-source-id: 06afc3aa1f8b9edb07fe9ed68c58968ad1926924
2021-08-29 23:33:04 -07:00
Adnios
09a8f22bf9
Add mish activation function ( #58648 )
...
Summary:
See issus: https://github.com/pytorch/pytorch/issues/58375
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58648
Reviewed By: gchanan
Differential Revision: D28625390
Pulled By: jbschlosser
fbshipit-source-id: 23ea2eb7d5b3dc89c6809ff6581b90ee742149f4
2021-05-25 10:36:21 -07:00
Kurt Mohler
3fe4718d16
Add padding_idx argument to EmbeddingBag ( #49237 )
...
Summary:
This PR adds a `padding_idx` parameter to `nn.EmbeddingBag` and `nn.functional.embedding_bag`. As with `nn.Embedding`'s `padding_idx` argument, if an embedding's index is equal to `padding_idx` it is ignored, so it is not included in the reduction.
This PR does not add support for `padding_idx` for quantized or ONNX `EmbeddingBag` for opset10/11 (opset9 is supported). In these cases, an error is thrown if `padding_idx` is provided.
Fixes https://github.com/pytorch/pytorch/issues/3194
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49237
Reviewed By: walterddr, VitalyFedyunin
Differential Revision: D26948258
Pulled By: jbschlosser
fbshipit-source-id: 3ca672f7e768941f3261ab405fc7597c97ce3dfc
2021-04-14 09:38:01 -07:00
Joel Schlosser
e86476f736
Huber loss ( #50553 )
...
Summary:
Fixes https://github.com/pytorch/pytorch/issues/48595 .
## Background
This PR implements HuberLoss, which differs from SmoothL1Loss by a factor of beta. The current implementation does not share logic between the two. Feedback is welcome for the optimal way to minimize code duplication while remaining performant.
I've done some early [benchmarking](https://pytorch.org/tutorials/recipes/recipes/benchmark.html#collecting-instruction-counts-with-callgrind ) with Huber calling in to the Smooth L1 kernel and scaling afterwards; for the simple test case I used, instruction counts are as follows:
```
Huber loss calls dedicated Huber kernel: 2,795,300
Huber loss calls Smooth L1 kernel and scales afterwards: 4,523,612
```
With these numbers, instruction counts are ~62% higher when using the pre-existing Smooth L1 kernel.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50553
Test Plan:
```
python test/test_nn.py TestNN.test_HuberLoss
python test/test_nn.py TestNN.test_HuberLoss_delta
python test/test_nn.py TestNN.test_huber_loss_invalid_delta
python test/test_nn.py TestNNDeviceTypeCPU.test_smooth_l1_loss_vs_huber_loss_cpu
python test/test_nn.py TestNNDeviceTypeCUDA.test_smooth_l1_loss_vs_huber_loss_cuda
python test/test_nn.py TestNNDeviceTypeCPU.test_invalid_reduction_strings_cpu
python test/test_nn.py TestNNDeviceTypeCUDA.test_invalid_reduction_strings_cuda
python test/test_nn.py TestNN.test_loss_equal_input_target_shape
python test/test_nn.py TestNN.test_pointwise_loss_broadcast
python test/test_overrides.py
python test/test_jit.py TestJitGeneratedFunctional.test_nn_huber_loss
python test/test_type_hints.py
python test/test_cpp_api_parity.py
build/bin/test_api
```
## Documentation
<img width="677" alt="Screen Shot 2021-01-14 at 4 25 08 PM" src="https://user-images.githubusercontent.com/75754324/104651224-5a445980-5685-11eb-884b-14ea517958c2.png ">
<img width="677" alt="Screen Shot 2021-01-14 at 4 24 35 PM" src="https://user-images.githubusercontent.com/75754324/104651190-4e589780-5685-11eb-974d-8c63a89c050e.png ">
<img width="661" alt="Screen Shot 2021-01-14 at 4 24 45 PM" src="https://user-images.githubusercontent.com/75754324/104651198-50225b00-5685-11eb-958e-136b36f6f8a8.png ">
<img width="869" alt="Screen Shot 2021-01-14 at 4 25 27 PM" src="https://user-images.githubusercontent.com/75754324/104651208-53b5e200-5685-11eb-9fe4-5ff433aa13c5.png ">
<img width="862" alt="Screen Shot 2021-01-14 at 4 25 48 PM" src="https://user-images.githubusercontent.com/75754324/104651209-53b5e200-5685-11eb-8051-b0cfddcb07d3.png ">
Reviewed By: H-Huang
Differential Revision: D26734071
Pulled By: jbschlosser
fbshipit-source-id: c98c1b5f32a16f7a2a4e04bdce678080eceed5d5
2021-03-02 17:30:45 -08:00
M.L. Croci
8eb90d4865
Add Gaussian NLL Loss ( #50886 )
...
Summary:
Fixes https://github.com/pytorch/pytorch/issues/48520 .
cc albanD (This is a clean retry PR https://github.com/pytorch/pytorch/issues/49807 )
Pull Request resolved: https://github.com/pytorch/pytorch/pull/50886
Reviewed By: ejguan
Differential Revision: D26007435
Pulled By: albanD
fbshipit-source-id: 88fe91b40dea6f72e093e6301f0f04fcc842d2f0
2021-01-22 06:56:49 -08:00
Guilherme Leobas
4411b5ac57
add type annotations to torch.nn.modules.normalization ( #49035 )
...
Summary:
Fixes https://github.com/pytorch/pytorch/issues/49034
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49035
Test Plan:
Imported from GitHub, without a `Test Plan:` line.
Force rebased to deal with merge conflicts
Reviewed By: zhangguanheng66
Differential Revision: D25767065
Pulled By: walterddr
fbshipit-source-id: ffb904e449f137825824e3f43f3775a55e9b011b
2021-01-12 07:40:15 -08:00
Guilherme Leobas
aa18d17455
add type annotations to torch.nn.modules.fold ( #49479 )
...
Summary:
closes gh-49478
Fixes https://github.com/pytorch/pytorch/issues/49478
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49479
Reviewed By: mruberry
Differential Revision: D25723838
Pulled By: walterddr
fbshipit-source-id: 45c4cbd6f147b6dc4a5f5419c17578c49c201022
2021-01-08 13:52:14 -08:00
Loi Ly
74c055b240
Fix mypy type hint for AdaptiveAvgPool2,3d, AdaptiveMaxPool2,3d ( #49963 )
...
Summary:
Fixes https://github.com/pytorch/pytorch/issues/49918
Pull Request resolved: https://github.com/pytorch/pytorch/pull/49963
Reviewed By: mrshenli, heitorschueroff
Differential Revision: D25760110
Pulled By: ezyang
fbshipit-source-id: aeb655b784689544000ea3b948f7d6d025aee441
2021-01-06 09:47:15 -08:00
Guilherme Leobas
ca8b9437ab
Add type annotations for a few torch.nn.modules ( #46013 )
...
Summary:
Fixes https://github.com/pytorch/pytorch/issues/46012
Pull Request resolved: https://github.com/pytorch/pytorch/pull/46013
Reviewed By: gchanan
Differential Revision: D25012419
Pulled By: ezyang
fbshipit-source-id: 9fd8ad9fa3122efa294a08010171cb7ddf752778
2020-11-18 07:44:59 -08:00
Guilherme Leobas
9b52654620
annotate a few torch.nn.modules.* modules ( #45772 )
...
Summary:
Fixes https://github.com/pytorch/pytorch/issues/45771
Pull Request resolved: https://github.com/pytorch/pytorch/pull/45772
Reviewed By: mruberry
Differential Revision: D24682013
Pulled By: albanD
fbshipit-source-id: e32bc4fe9c586c079f7070924a874c70f3d127fa
2020-11-02 13:04:59 -08:00
Mike Ruberry
ef885c10d8
[pytorch] Add triplet margin loss with custom distance ( #43680 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43680
As discussed [here](https://github.com/pytorch/pytorch/issues/43342 ),
adding in a Python-only implementation of the triplet-margin loss that takes a
custom distance function. Still discussing whether this is necessary to add to
PyTorch Core.
Test Plan:
python test/run_tests.py
Imported from OSS
Reviewed By: albanD
Differential Revision: D23363898
fbshipit-source-id: 1cafc05abecdbe7812b41deaa1e50ea11239d0cb
2020-09-22 11:35:52 -07:00
Nikita Shulga
442684cb25
Enable typechecks for torch.nn.modules.[activation|upsampling] ( #44093 )
...
Summary:
Add missing `hardsigmoid`, `silu`, `hardswish` and `multi_head_attention_forward` to functional.pyi.in
Embed some typing annotations into functional.py
Pull Request resolved: https://github.com/pytorch/pytorch/pull/44093
Reviewed By: ezyang
Differential Revision: D23494384
Pulled By: malfet
fbshipit-source-id: 27023c16ff5951ceaebb78799c4629efa25f7c5c
2020-09-03 13:20:04 -07:00
Ralf Gommers
573940f8d7
Fix type annotation errors in torch.functional ( #43446 )
...
Summary:
Closes gh-42968
Pull Request resolved: https://github.com/pytorch/pytorch/pull/43446
Reviewed By: albanD
Differential Revision: D23280962
Pulled By: malfet
fbshipit-source-id: de5386a95a20ecc814c39cbec3e4252112340b3a
2020-08-26 08:27:59 -07:00
Edward Yang
6edf340338
Delete torch/__init__.pyi, deferring to direct extension stubs ( #38157 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/38157
This removes the error prone process of assembling `torch/__init__.pyi`
(and frequently forgetting to expose things), since now we can simply
rely on the true source file to get things done. Most of the old
codegen in gen_pyi.py is now rerouted to various files:
- `torch/_C/__init__.pyi` (the dumping pile of all misc bindings)
- `torch/_C/_nn.pyi` (NN function bindings)
- `torch/_C/_VariableFunctions.pyi` (torch function bindings)
`torch.types` grew a bunch more definitions that previously where
defined in `torch/__init__.pyi`
Some miscellaneous changes
- Fixed a bug where we treat single TensorList argument as implying
varargs are accepted. This is actually only supported on IntList.
This means we can correctly generate a stub for dequantize.
- Add missing manual stub for nonzero
- Switched torch/onnx/operators.py to directly refer to _C module,
since apparently mypy doesn't think that methods prefixed with
underscores get reexported. This may be a recurring theme; maybe
we need to find a better way to solve it.
Because I was really lazy, I dumped namedtuple definitions in both
`torch._C` and `torch._C._VariableFunctions`. This is definitely wrong.
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Test Plan: Imported from OSS
Differential Revision: D21497400
Pulled By: ezyang
fbshipit-source-id: 07b126141c82efaca37be27c07255cb2b9b3f064
2020-05-11 07:20:13 -07:00
Nik Ved
35cdb78522
Make kl_div accept target in log space ( #34586 )
...
Summary:
Fixes [32520](https://github.com/pytorch/pytorch/issues/32520 ), implements [34536](https://github.com/pytorch/pytorch/issues/34536 ).
Here are some benchmarks:
```python
import torch
import torch.nn.functional as F
from IPython import get_ipython
ipython = get_ipython()
torch.set_num_threads(1)
for d in [5, 10, 20, 50, 100, 1000]:
i = torch.rand(d, d)
t = torch.rand(d, d)
print(f"Size: {d}x{d}")
ipython.magic("timeit F.kl_div(i, t, reduction='none', log_target=False)")
ipython.magic("timeit F.kl_div(i, t.log(), reduction='none', log_target=True)")
```
Output:
```
Size: 5x5
16 µs ± 33 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
8.24 µs ± 17.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
Size: 10x10
16.7 µs ± 17.5 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
8.7 µs ± 20.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
Size: 20x20
17.7 µs ± 47.5 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
9.7 µs ± 28.8 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
Size: 50x50
23.6 µs ± 60.1 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
15 µs ± 33.7 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
Size: 100x100
42.8 µs ± 223 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
34 µs ± 17.2 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
Size: 1000x1000
3.9 ms ± 1.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
3.45 ms ± 364 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/34586
Differential Revision: D20652726
Pulled By: ezyang
fbshipit-source-id: 480697b4cd01341bbeee7514a8b812705a0600ea
2020-04-01 12:26:58 -07:00
Sameer Deshmukh
602394e996
verify input sizes for instance norm and group norm ( #29082 )
...
Summary:
Fix for https://github.com/pytorch/pytorch/issues/19250
Pull Request resolved: https://github.com/pytorch/pytorch/pull/29082
Differential Revision: D19373507
Pulled By: ezyang
fbshipit-source-id: 231a79280f4cd7db2c26218a60869356a124bf77
2020-01-27 09:05:56 -08:00
Edgar Riba
510a122d27
add missing align_corners annotation ( #32492 )
...
Summary:
adds the missing annotation in grid_sample and affine_grid functional
Pull Request resolved: https://github.com/pytorch/pytorch/pull/32492
Differential Revision: D19516550
Pulled By: ezyang
fbshipit-source-id: 064c8c99bf6eae6744237c0b151b3ce4c82ada96
2020-01-22 11:29:07 -08:00
Xiaomeng Yang
2460dced8f
Add torch.nn.GELU for GELU activation ( #28944 )
...
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/28944
Add torch.nn.GELU for GELU activation
Test Plan: buck test mode/dev-nosan //caffe2/test:nn -- "GELU"
Reviewed By: hl475, houseroad
Differential Revision: D18240946
fbshipit-source-id: 6284b30def9bd4c12bf7fb2ed08b1b2f0310bb78
2019-11-03 21:55:05 -08:00
Jon Malmaud
bfeff1eb8f
Stubs for torch.nn ( #19089 )
...
Summary:
Closes https://github.com/pytorch/pytorch/issues/18724
Pull Request resolved: https://github.com/pytorch/pytorch/pull/19089
Differential Revision: D16073654
Pulled By: ezyang
fbshipit-source-id: 5642179651ce45ab7c5a46cc1fcc4fd6b37fa71c
2019-07-01 09:50:17 -07:00
