OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00
Code Issues Actions 37 Packages Projects Releases Wiki Activity
64,482 Commits 8,304 Branches 150 Tags 1.5 GiB
1b51d29b66
Commit Graph

345 Commits

Author SHA1 Message Date
Abhishek Pathak
8860e48994 [MPS] Handle compatible inputs to where (#85946) 
Inputs with different number of dimensions but compatible shapes were being rejected

e.g. x.shape = [10,1,10]
y.shape = [10,10]
cond.shape = [10,10,1]
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85946
Approved by: https://github.com/malfet
 2022-10-03 18:12:48 +00:00
Nikita Shulga
b9b24c31fd [MPS] Fix non-contig to contig tensor copy (#86056) 
This handles a rare case when MPS tensor is constructed from non-contiguous CPU tensor.
Fixes https://github.com/pytorch/pytorch/issues/85967

Pull Request resolved: https://github.com/pytorch/pytorch/pull/86056
Approved by: https://github.com/janeyx99
 2022-10-02 20:13:05 +00:00
Ramin Azarmehr
bf667c63e7 Fix the error with constant_pad_nd for 4D+ padding (#85991) 
- We warn the user and fall back to default implementation for 4D+ constant padding

Fixes #84535

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85991
Approved by: https://github.com/kulinseth
 2022-10-01 00:33:23 +00:00
Ramin Azarmehr
334686bde7 Fix the dimension of padding to match the input's dimension (#85990) 
Fixes #85143

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85990
Approved by: https://github.com/malfet, https://github.com/kulinseth
 2022-09-30 22:57:57 +00:00
Ramin Azarmehr
a4cc63991a [MPS] Enable caching for random ops with Philox engine (#85833) 
Also Fix type cast issue in Bernoulli (Fixes #85611)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85833
Approved by: https://github.com/kulinseth, https://github.com/malfet
 2022-09-30 22:40:50 +00:00
Denis Vieriu
be327ec08f [MPS] Fix base shape size for view ops in case of multiple slices (#85934) 
Fixes https://github.com/pytorch/pytorch/issues/84364, https://github.com/pytorch/pytorch/issues/85592

Fixes bug for view ops where the base shape would be incorectly determined.
E.g for the following tensor `torch.tensor([0.5, 0.5], device="mps")[1][None]`, we could consider the base shape of the parent tensor as 1, while the actual base shape is 2.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85934
Approved by: https://github.com/kulinseth
 2022-09-30 18:51:43 +00:00
Abhishek Pathak
81b366a9dd [MPS] Handle scalar input for scatter and gather (#85842) 
Issue noticed in test consistency - "Indexing dim 0 is out of bounds of tensor"
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85842
Approved by: https://github.com/kulinseth
 2022-09-30 00:24:16 +00:00
Abhishek Pathak
62a4fd7907 [MPS] Handle output shape for empty input in binary ops (#85836) 
Output of input shape [0,1,2] should be [0,1,2], not [0] i.e. delay returning from empty input condition to resize/reshape the output accordingly
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85836
Approved by: https://github.com/DenisVieriu97, https://github.com/kulinseth
 2022-09-30 00:19:14 +00:00
Denis Vieriu
6a14fcb922 [MPS] Add support for aten::masked_select on mps (#119) (#85818) 
Reuse the `index.Tensor_out` implementation since it's already expanding the bool/byte indices to long tensors.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85818
Approved by: https://github.com/kulinseth
 2022-09-29 23:23:00 +00:00
Denis Vieriu
ce4f187f15 [MPS] Add tensor::index_put op (#85672) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85672
Approved by: https://github.com/malfet
 2022-09-28 00:47:52 +00:00
Kulin Seth
7ff6a00a9a [MPS] Handle 1D weight in linear layer (#85752) 
Fixes https://github.com/pytorch/pytorch/issues/84591

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85752
Approved by: https://github.com/malfet
 2022-09-28 00:43:11 +00:00
Abhishek Pathak
e746fff8ba [MPS] Enable adaptive avg pool 2d with larger output size (#85726) 
* Handle adpative pool 2d forward and backward when ouptut size is larger than input size
* Disallow larger output size if not a multiple of input size
Fixes: https://github.com/pytorch/pytorch/issues/80732
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85726
Approved by: https://github.com/malfet
 2022-09-27 19:08:22 +00:00
Nikita Shulga
1367f2409f [MPS] Fix placeholder behavior for transposed view (#85689) 
Looks like the expectation in that code were that `.clone` will return contiguous tensor, so explicitly specify memory format

Fixes https://github.com/pytorch/pytorch/issues/85675 and https://github.com/pytorch/pytorch/issues/85224

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85689
Approved by: https://github.com/kulinseth
 2022-09-27 15:44:53 +00:00
Abhishek Pathak
7a5449f148 [MPS] Clamp op - fix shape issues (#114) (#85673) 
* Handle shape mismatch
* Handle case where 1 occurs in input shape; fix fill_new_shapes
* Move clamp ops to allowlist

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85673
Approved by: https://github.com/malfet
 2022-09-27 01:54:42 +00:00
George Qi
686555b663 [maskedtensor] port torch/_masked into torch/masked (#85515) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/85515
Approved by: https://github.com/cpuhrsch
 2022-09-26 23:41:13 +00:00
Abhishek Pathak
f0570354dd [MPS] Fix memory error in var (#85571) 
* Fix memory corruption + wrong handling of negative dims
* Use vector for shape

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85571
Approved by: https://github.com/malfet
 2022-09-25 19:03:58 +00:00
Kulin Seth
e1ed485c65 [MPS] Handle reduction of scalars in edge-cases (#83743) 
The issue was found as part of fixing Test consistency issues.
Test case coming up.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/83743
Approved by: https://github.com/razarmehr, https://github.com/malfet
 2022-09-20 17:53:45 +00:00
Kulin Seth
077db3de92 [MPS] Fix conv1d backwards crash for channels last case (#85283) 
Fixes pytorch#84511

Use the same logic as in the forward pass for the backward pass (in case of channels last, manually set the shape to NHWC)

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85283
Approved by: https://github.com/malfet, https://github.com/razarmehr
 2022-09-20 06:19:40 +00:00
Nikita Shulga
1a6cf6ea88 [MPS] Fix int rounding div crash on M1 (#85016) 
Fixes  https://github.com/pytorch/pytorch/issues/84995

Pull Request resolved: https://github.com/pytorch/pytorch/pull/85016
Approved by: https://github.com/kulinseth
 2022-09-14 23:40:20 +00:00
Denis Vieriu
4247cc98a2 [MPS] Fix mps to cpu casting from a smaller dtype to a bigger dtype (#84928) 
Fixes #82566 , #80800

- mps->cpu casts from a smaller dtype to a bigger dtype mps->mps cast from smaller/bigger dtype to another dtype in case of scatter
- For mps->cpu copies where we don't have a source/destination offset, we can save the cast result directly in the destTensor, so we can skip the additional overhead of the blit.
- In case we can return the data without doing the blit, we need to check if it's blocking call, case in which we'd need a synchronize(SyncType::COMMIT_AND_WAIT); call (previously this was done by the blit).

Pull Request resolved: https://github.com/pytorch/pytorch/pull/84928
Approved by: https://github.com/razarmehr
 2022-09-14 17:24:25 +00:00
Nikita Shulga
9b16bf04af Fix MPS test sanity (#84889) 
Follow up after https://github.com/pytorch/pytorch/pull/84834

Pull Request resolved: https://github.com/pytorch/pytorch/pull/84889
Approved by: https://github.com/tugsbayasgalan, https://github.com/janeyx99, https://github.com/ZainRizvi
 2022-09-12 22:25:26 +00:00
Abhishek Pathak
bccc26f365 [MPS] Handle casting for div operation (#84742) 
* Handle casting for div operation
* Update divmode test to test for rounding mode in div

cc. @lhoenig
Pull Request resolved: https://github.com/pytorch/pytorch/pull/84742
Approved by: https://github.com/razarmehr
 2022-09-10 03:10:04 +00:00
Peter Bell
b9793a66b5 Fix linalg.norm sample inputs function and related failures (#84452) 
Due to an indentation error, the return statement happens after just 1
loop of `for test_size in test_sizes` so only one shape was ever
tested.

This also revealed several cases where the provided shapes don't work
so I've disabled the generation of those sample inputs.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/84452
Approved by: https://github.com/Lezcano, https://github.com/zou3519
 2022-09-08 17:07:58 +00:00
soulitzer
bfdfeecd15 Add per-op MPS gradient tests and update skips (#84242) 
Follow up:
- ~Remove non-float dtypes from allow-list for gradients~
- ~Map dtypes to short-hand so there aren't so many lines, i.e. float16 should be f16.~
- ~There were a lot of linting issues that flake8 wouldn't format for me, so I reformatted with black. This makes the diff a little trickier to parse.~

Observations:
- there are entries in the allow-list that weren't there before
- some forward that we previously passing now fail with requires_grad=True
- Because the allow list does not know about variants, a special skip was added for that in the block list

Pull Request resolved: https://github.com/pytorch/pytorch/pull/84242
Approved by: https://github.com/kulinseth, https://github.com/malfet
 2022-09-01 16:41:52 +00:00
Ramin Azarmehr
d1be36ceab [MPS] Fix the index error in constant_pad_nd() with single-dimension input (#83745) 
* Fix the index error in constant_pad_nd() with single-dimension input (#83343)
- Also added a test case in test_mps for it

* Move padding code into new file Pad.mm

Fixes https://github.com/pytorch/pytorch/issues/83343

Pull Request resolved: https://github.com/pytorch/pytorch/pull/83745
Approved by: https://github.com/razarmehr
 2022-08-22 17:07:09 +00:00
Denis Vieriu
a6b75bb099 [MPS] Fix placeholder case for missing gather graph (#83744) 
Fixes https://github.com/pytorch/pytorch/issues/82543, https://github.com/pytorch/pytorch/issues/83230

The current Placeholder code relies to find a gather graph in order to make the data contiguous, otherwise we'll try calling into tensor.contiguous() directly, which for slice elements, won't do anything.

E.g consider the following basic case where we index a 2 element tensor:
```
tensor_list = torch.tensor([1.2, 1.0], device="mps")

for scalar in tensor_list:
  r_mps = torch.ceil(scalar)
  r_cpu = torch.ceil(scalar.to("cpu"))
  self.assertEqual(r_mps.cpu(), r_cpu)
```

The second element 1.0 is a contiguous view tensor (similar to slicing), but it has no gather graph created behind. In the placeholder, we won't be able to find the graph, thus relying on the fallback case where we call _tensor = src.contiguous();. For an already contiguous tensor, this won't do anything, thus we end up creating the NDArray with all the values of the tensor (1.2 and 1.0 instead of just 1.0). Doing clone instead of contiguous will actually perform a blit behind and take into consideration the storage_offset of the view when performing the copy.

Similarly, the following basic case is also failing because of this issue:

```
x = torch.tensor([1.0, 0.49], device="mps")
print(x) # prints 1.0 and 0.0
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/83744
Approved by: https://github.com/razarmehr
 2022-08-22 17:05:53 +00:00
Nikita Shulga
ff533b1efa [MPS] Fix torch.full for uint8 (#83697) 
By creating uint32  tensor and then downcasting it to uint8

Workaround https://github.com/pytorch/pytorch/issues/83692

Pull Request resolved: https://github.com/pytorch/pytorch/pull/83697
Approved by: https://github.com/albanD
 2022-08-18 21:59:15 +00:00
Kulin Seth
ce7177f88a [MPS] Register index.Tensor_out (#82507) 
* Add more tests from test_indexing into test_mps
* Cache the indexing library on the MPSDevice
Pull Request resolved: https://github.com/pytorch/pytorch/pull/82507
Approved by: https://github.com/malfet
 2022-08-18 06:03:16 +00:00
Kulin Seth
31d4b6f52a [MPS] Fix conv1D and conv2D with non-matching strides/paddings (#83522) 
* Add reference to the github issue in test_mps.py

Fixes https://github.com/pytorch/pytorch/issues/83180, https://github.com/pytorch/pytorch/issues/82921, https://github.com/pytorch/pytorch/issues/82711, https://github.com/pytorch/pytorch/issues/82563

Pull Request resolved: https://github.com/pytorch/pytorch/pull/83522
Approved by: https://github.com/albanD, https://github.com/malfet
 2022-08-17 00:26:41 +00:00
Kulin Seth
02cfefb48c [MPS] Fix for matmul errors in test consistency (#83124) 
* Handle empty input with non-empty output
* Remove transpose options from mm op

Pull Request resolved: https://github.com/pytorch/pytorch/pull/83124
Approved by: https://github.com/malfet
 2022-08-12 23:28:38 +00:00
Kulin Seth
b3dea3e413 Add the Conv1D support for NHWC format. (#83121) 
Fixes https://github.com/pytorch/pytorch/issues/81557

cc @DenisVieriu97

Pull Request resolved: https://github.com/pytorch/pytorch/pull/83121
Approved by: https://github.com/malfet
 2022-08-10 14:30:20 +00:00
Nikita Shulga
dcf5188561 [MPS] Fix scatter for bool type (#82685) 
By typecasting bool tensors to int8 before scattering

See https://github.com/pytorch/pytorch/issues/82663

Pull Request resolved: https://github.com/pytorch/pytorch/pull/82685
Approved by: https://github.com/albanD, https://github.com/kulinseth
 2022-08-03 14:54:47 +00:00
Nikita Shulga
420c576809 [MPS] Unary ops over empty tensors should be no-op (#82650) 
Fixes https://github.com/pytorch/pytorch/issues/82531

Pull Request resolved: https://github.com/pytorch/pytorch/pull/82650
Approved by: https://github.com/kulinseth
 2022-08-02 21:15:37 +00:00
Nikita Shulga
bdd0a4a84c [MPS] Fix torch.full for boolean types (#82575) 
By creating int8 tensor and casting it to bool later

Workaround for MPSGraph deficiency reported in https://github.com/pytorch/pytorch/issues/82427

Pull Request resolved: https://github.com/pytorch/pytorch/pull/82575
Approved by: https://github.com/kulinseth
 2022-08-01 19:42:24 +00:00
qqaatw
7496937e3f [MPS] Add prelu (#82401) 
### Description
Adds `prelu` MPS support.

I'm wondering whether the generic test framework implemented in `test_mps.py` checks the correctness of gradients.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/82401
Approved by: https://github.com/kulinseth
 2022-07-30 05:16:25 +00:00
Ramin Azarmehr
38b4114278 [MPS] Add MPS implementation for constant_pad_nd() (#75) (#82366) 
MPS has a native implementation of the constant pad nd. Adding that instead of going through the view ops helps improve performance in several benchmarks in torchbench.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/82366
Approved by: https://github.com/malfet, https://github.com/razarmehr
 2022-07-29 16:34:07 +00:00
Abhishek Pathak
d7210e6129 [MPS] Fixes for MPS testConsistency (#81735) 
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/81735
Approved by: https://github.com/razarmehr, https://github.com/albanD
 2022-07-20 16:31:44 +00:00
Kulin Seth
596bb41163 [MPS] Get the correct size of the view tensor when copying from cpu to mps (#81730) 
Fixes: https://github.com/pytorch/pytorch/issues/81567, https://github.com/pytorch/pytorch/issues/80844
* Get the correct size of the view tensor when copying from cpu to mps

* Use 'computeStorageNbytesContiguous' to get the size just when src is a view

* Add asserts and tests to check for storage_offset

* Add testcase for https://github.com/pytorch/pytorch/issues/80844

* Replace assert_allclose with assertEqual

* Replace TORCH_CHECK with TORCH_INTERNAL_ASSERT
Pull Request resolved: https://github.com/pytorch/pytorch/pull/81730
Approved by: https://github.com/razarmehr, https://github.com/albanD
 2022-07-20 14:27:54 +00:00
qqaatw
1caa25ebcb [MPS] Add aten::index_add.out (#79935) 
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79935
Approved by: https://github.com/kulinseth
 2022-07-14 23:40:00 +00:00
Denis Vieriu
e3b98ba7a4 [MPS] Handle Boolean inputs in the cat op (#81480) 
Fixes https://github.com/pytorch/pytorch/issues/80850

Pull Request resolved: https://github.com/pytorch/pytorch/pull/81480
Approved by: https://github.com/razarmehr
 2022-07-14 22:00:57 +00:00
Denis Vieriu
0adc2e35f2 Add testcase for MPS issue #80856 (#81455) 
Testcase for #80856.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/81455
Approved by: https://github.com/kulinseth
 2022-07-14 19:54:18 +00:00
Kulin Seth
067c8067a3 [MPS]: Added op upsample_nearest1d (#81303) 
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/81303
Approved by: https://github.com/malfet
 2022-07-13 21:39:50 +00:00
Abhishek Pathak
ae83e44c5f [MPS] Handle 1D inputs for NLL (#81290) 
* Add test for NLL 1d
* Fix forward NLL for 1D case
* Handle NLL backward for 1d

Pull Request resolved: https://github.com/pytorch/pytorch/pull/81290
Approved by: https://github.com/razarmehr
 2022-07-12 19:46:59 +00:00
qqaatw
b0b24b4285 [MPS] Fix LSTM batch_first output transposed (#80597) 
The output of LSTM with `batch_first` should be transposed back to batch first format.

Fixes #80306

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80597
Approved by: https://github.com/kulinseth
 2022-07-07 07:18:00 +00:00
qqaatw
2458b3cd5f [MPS] Add argmin (#80828) 
This PR

1. adds argmin
2. refactors `reduction_type` in `ReduceOps.mm` with enum.

Co-authored by Kulin Seth <kulinseth@gmail.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/80828
Approved by: https://github.com/malfet
 2022-07-07 00:04:49 +00:00
PyTorch MergeBot
255649984c Revert "[MPS] Add argmin (#80828)" 
This reverts commit 37ebcc8a80.

Reverted https://github.com/pytorch/pytorch/pull/80828 on behalf of https://github.com/suo due to Broke Mac tests on master: 37ebcc8a80
 2022-07-06 16:20:18 +00:00
qqaatw
c48e964a04 [MPS] Add huber loss (#80163) 
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80163
Approved by: https://github.com/kulinseth, https://github.com/malfet
 2022-07-06 15:36:47 +00:00
qqaatw
37ebcc8a80 [MPS] Add argmin (#80828) 
This PR

1. adds argmin
2. refactors `reduction_type` in `ReduceOps.mm` with enum.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/80828
Approved by: https://github.com/malfet
 2022-07-06 15:14:04 +00:00
qqaatw
8745118ccb [MPS] Add softplus backward (#79873) 
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79873
Approved by: https://github.com/malfet
 2022-07-06 06:13:21 +00:00
Kulin Seth
5436134f32 [MPS] Move the View ops to a separate file and reduce the number of graphs created (#80491) 
This is dependent on the PR to go in first: https://github.com/pytorch/pytorch/pull/79939

Remove the data_ptr from the View Graph key which reduces the number of
graphs created significantly.

Don't wait when copying from MPS to MPS tensors

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80491
Approved by: https://github.com/malfet
 2022-07-06 03:39:20 +00:00
Kulin Seth
76cff18242 [MPS] Add test consistency from OpInfo based tests from PR 78504 (#79532) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79532
Approved by: https://github.com/albanD, https://github.com/malfet
 2022-07-04 06:41:39 +00:00
Ramin Azarmehr
0e3953fc52 MPS: Fix handling of 1D tensors in linear backward (#80759) 
Fixes #https://github.com/pytorch/pytorch/issues/79784

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80759
Approved by: https://github.com/ezyang
 2022-07-04 02:06:14 +00:00
Kulin Seth
b744e1c8ef Add scatter support for view operations (#79939) 
* Add scatter support for view operations; #78074, #78886, #79672
* Update test_slicing_replace_column to properly test different sizes
* Handle in-place changes for binary ops; add new testcase
* Add new view ops testing scatter; add MPSDebugConfig.h config file for debugging purposes
* Merge gatherViewTensor and scatterViewTensor into a generic function
* Add scatter on demand in scatterViewOperation instead of caching it into a generic graph
* Create separate graphs for scatter and gather;
* Create scatter graph at scatter time

Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79939
Approved by: https://github.com/razarmehr
 2022-07-01 15:10:56 +00:00
PyTorch MergeBot
b1943e01e2 Revert "[MPS] Add test consistency from OpInfo based tests from PR 78504 (#79532)" 
This reverts commit c71886e048.

Reverted https://github.com/pytorch/pytorch/pull/79532 on behalf of https://github.com/malfet due to Unintended submodules updates
 2022-06-30 16:37:11 +00:00
qqaatw
ae6f07e7d5 [MPS] Fix std/var cache issue (#80502) 
Use `getTensorsStringKey` which has tensor shape info added as part of the key to prevent cache lookup issue when the shape of input tensor is changed.

Fixes #80499

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80502
Approved by: https://github.com/malfet, https://github.com/kulinseth
 2022-06-30 12:56:56 +00:00
qqaatw
c980fc3d3c [MPS] Add glu (#79866) 
Adds mps op for `aten::glu.out`.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79866
Approved by: https://github.com/kulinseth, https://github.com/albanD
 2022-06-30 08:58:42 +00:00
Kulin Seth
c71886e048 [MPS] Add test consistency from OpInfo based tests from PR 78504 (#79532) 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79532
Approved by: https://github.com/albanD
 2022-06-30 01:50:17 +00:00
qqaatw
5943aaa0c4 [MPS] Add logical ops (#80216) 
This PR adds `logical_not`, `logical_and`, `logical_or`, `logical_xor`.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/80216
Approved by: https://github.com/albanD, https://github.com/kulinseth
 2022-06-29 02:44:35 +00:00
qqaatw
c4da23ed1b [MPS] Add flip (#80214) 
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80214
Approved by: https://github.com/DenisVieriu97, https://github.com/albanD
 2022-06-28 19:51:45 +00:00
qqaatw
e1b15b7a04 [MPS] add aten::normal.Tensor_float aten::normal.float_Tensor aten::normal.Tensor_Tensor (#80297) 
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80297
Approved by: https://github.com/albanD, https://github.com/kulinseth
 2022-06-28 15:19:39 +00:00
Nikita Shulga
f11cce309b [MPS] Add equal operator (#80195) 
Which is, in essence is composite of `eq`->`all`->`item`
`native/mps/operators/Equal.cpp` is an almost verbatim copy of `native/cuda/Equal.cpp`

Fix codegen by generating MPSFunctions headers

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80195
Approved by: https://github.com/albanD
 2022-06-25 12:40:52 +00:00
Nikita Shulga
06f874e276 [MPS] Fix binary ops between int32 tensor with int64 scalar (#80220) 
For some reason, tensor *op* scalar does not follow the normal binary promotion rules
So cast output tensor to expected type if needed
It seems that one should have casted input tensors to expected output tensor type, but it does not really work for boolean binary ops, so...
Add output tensor type/shape to cached graph key
Extend `TestMPS. test_add_scalars` to test for this regression

Fixes #79835

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80220
Approved by: https://github.com/albanD
 2022-06-25 02:21:34 +00:00
qqaatw
ff44bfa1ea [MPS] Add L1 loss test (#80010) 
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80010
Approved by: https://github.com/albanD
 2022-06-24 17:18:31 +00:00
Nikita Shulga
4390546f86 [MPS] Fix torch.uint8 support (#80049) 
`ScalarType.Byte` should be cast to `MPSDataTypeUInt8`
And support for `torch.int8` as well as test those conversions in `TestMPS.test_to`

Fixes #80006

Pull Request resolved: https://github.com/pytorch/pytorch/pull/80049
Approved by: https://github.com/albanD
 2022-06-22 18:41:21 +00:00
Abhishek Pathak
074dc7465e MPS: Add amax and amin Ops with tests (#79682) 
* Add amax and amin with tests

Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79682
Approved by: https://github.com/albanD
 2022-06-18 00:14:05 +00:00
Kulin Seth
4615f6aa97 [MPS]: Add fix for squeezed input axes handling in BCE loss (#79676) 
Fixes #79527

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79676
Approved by: https://github.com/razarmehr, https://github.com/albanD
 2022-06-16 20:21:31 +00:00
Kulin Seth
355a1c8c3f MPS: TopK raise an error if K>16 (#79677) 
* Error out in TopK when k>16.
* Add a test case too.

Fixes #78915

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79677
Approved by: https://github.com/albanD
 2022-06-16 16:06:45 +00:00
Nikita Shulga
81cd276d61 [MPS] Support stride of stride 
Fixes https://github.com/pytorch/pytorch/issues/79181

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79521

Approved by: https://github.com/kulinseth
 2022-06-14 18:49:44 +00:00
Alban Desmaison
0a651a231d Add full support for serialization of MPS Tensors (#79465) 
Fix https://github.com/pytorch/pytorch/issues/79384
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79465
Approved by: https://github.com/kulinseth, https://github.com/malfet
 2022-06-14 17:54:30 +00:00
PyTorch MergeBot
ce6ce74703 Revert "Add full support for serialization of MPS Tensors (#79465)" 
This reverts commit 64c2a275c4.

Reverted https://github.com/pytorch/pytorch/pull/79465 on behalf of https://github.com/zengk95 due to this broke X linux-xenial-py3.7-clang7-onnx / test (default, 1, 2, linux.2xlarge). Not sure why since it passed on pull.
 2022-06-14 16:42:36 +00:00
Alban Desmaison
64c2a275c4 Add full support for serialization of MPS Tensors (#79465) 
Fix https://github.com/pytorch/pytorch/issues/79384
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79465
Approved by: https://github.com/kulinseth, https://github.com/malfet
 2022-06-14 14:20:09 +00:00
Kulin Seth
77b6885a22 MPS: add layer_norm_backward (#79189) 
Layernorm backward

Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79189
Approved by: https://github.com/razarmehr, https://github.com/albanD
 2022-06-10 13:25:41 +00:00
Kulin Seth
83239351c5 MPS: add exponential op (#79188) 
Add exponential distribution

Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/79188
Approved by: https://github.com/razarmehr, https://github.com/albanD
 2022-06-10 13:16:21 +00:00
Kulin Seth
50f7b40ad9 MPS: Binary cast fix by proper type promotion and remove spurious copy warning (#79185) 
Fixes #78019, #78020
Fixes https://github.com/pytorch/pytorch/pull/79185
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79185
Approved by: https://github.com/albanD, https://github.com/razarmehr
 2022-06-09 17:33:06 +00:00
Nikita Shulga
97594a24b4 Print output during MPS test import tests (#79163) 
Simplify `test_no_warnings_on_input` to simply capture any output.
Copy its implementation to `test_testing.py` as this is not specific to MPS
Pull Request resolved: https://github.com/pytorch/pytorch/pull/79163
Approved by: https://github.com/janeyx99, https://github.com/kulinseth
 2022-06-09 13:07:05 +00:00
Philip Meier
32593ef2dd move MPS compat into common comparison machinery (#77836) 
Addresses https://github.com/pytorch/pytorch/issues/77144#issuecomment-1128168082.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/77836
Approved by: https://github.com/albanD
 2022-06-08 08:09:18 +00:00
Kulin Seth
a6347f5467 MPS: Fixes (#78930) 
Cast integer to float in UnaryOps
Add tensor dtype in key generation
Enable FP16 scalars and use placeholder for alpha tensor in add/sum ops

Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78930
Approved by: https://github.com/albanD
 2022-06-07 18:22:10 +00:00
Nikita Shulga
55cac22cdf [MPS] Add arange_mps_out implementation (#78789) 
Mostly by factoring out shader logic from `linspace_out_mps` implementation
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78789
Approved by: https://github.com/albanD, https://github.com/kulinseth
 2022-06-03 21:54:41 +00:00
Kulin Seth
4858c56334 MPS: Fix issues with view tensors and linspace. (#78690) 
Fixes: #https://github.com/pytorch/pytorch/issues/78642, https://github.com/pytorch/pytorch/issues/78511
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78690
Approved by: https://github.com/razarmehr, https://github.com/DenisVieriu97
 2022-06-02 06:17:19 +00:00
Kulin Seth
a3bdafece3 MPS: add linespace op (#78570) 
Fixes #ISSUE_NUMBER

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78570
Approved by: https://github.com/malfet
 2022-06-01 13:47:14 +00:00
Ramin Azarmehr
aa62b3e003 Add test case for issue: https://github.com/pytorch/pytorch/issues/77851 (#78547) 
The test works fine now.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78547
Approved by: https://github.com/kulinseth
 2022-05-31 19:15:45 +00:00
Rohan Mitchell
f42b42d3eb MPS: Implement aten::count_nonzero.dim_IntList (#78169) 
- See: #77764

Implements the `aten::count_nonzero.dim_IntList` operator (as used by [torch.count_nonzero](https://pytorch.org/docs/stable/generated/torch.count_nonzero.html)) for [MPS](https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/).

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78169
Approved by: https://github.com/malfet, https://github.com/kulinseth, https://github.com/albanD
 2022-05-31 18:23:25 +00:00
Kulin Seth
017b0ae943 MPS: Fix crashes in view tensors due to buffer size mismatch (#78496) 
Fixes #78247, #77886

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78496
Approved by: https://github.com/albanD, https://github.com/malfet
 2022-05-31 02:09:03 +00:00
Alban Desmaison
bde246fcc6 Speed up test_mps from 9min to 25s 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78488

Approved by: https://github.com/kulinseth
 2022-05-30 18:16:53 +00:00
Alban Desmaison
02551a0025 Remove prints and add proper asserts 
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78454

Approved by: https://github.com/kulinseth
 2022-05-30 18:16:53 +00:00
Kulin Seth
d63db52349 MPS: Fixes the as_strided_mps implementation for contiguous view operations (#78440) 
Fixes https://github.com/pytorch/pytorch/issues/78107; https://github.com/pytorch/pytorch/issues/77750

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78440
Approved by: https://github.com/malfet
 2022-05-28 14:41:56 +00:00
Nikita Shulga
437ecfc461 [MPS] Fix copy_kernel_mps (#78428) 
By passing `storage_offset` of source and destination Tensors
This fixes following simple usecase:
```
python3` -c "import torch;x=torch.zeros(3, 3, device='mps'); x[1, 1]=1;print(x)"
```

Add test to validate it would not regress in the future

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78428
Approved by: https://github.com/kulinseth
 2022-05-27 20:46:53 +00:00
Kulin Seth
8552acbd74 MPS: Eye op (#78408) 
This can be used as a reference PR was to add Op in MPS backend.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78408
Approved by: https://github.com/albanD
 2022-05-27 17:07:02 +00:00
Kulin Seth
2e32d5fcd8 MPS: Add adaptive max pool2d op (#78410) 
Adaptive max pool 2d forward and backward with test

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78410
Approved by: https://github.com/albanD
 2022-05-27 11:59:07 +00:00
Nikita Shulga
705082656a Fix typo in testname (#78258) 
`test_linear2D_no_bias_backwarwd` -> `test_linear2D_no_bias_backward`

Pull Request resolved: https://github.com/pytorch/pytorch/pull/78258
Approved by: https://github.com/kulinseth, https://github.com/janeyx99
 2022-05-25 16:23:10 +00:00
Lukas Hoenig
a52bfe2c5d Convert MPS Tensor data using MPSGraph API (#78092) 
Fixes #78091
If you are already working on this, simply disregard this or take what may be helpful. This is my attempt at MPS-native Tensor datatype conversion. It works for everything tested ~~but is currently only implemented for MPS-to-MPS copy, not MPS-to-X or X-to-MPS, but the same approach could easily be used~~.

Before:
```python
In [5]: pt.full((40,), -10.3, device="mps")
Out[5]:
tensor([-10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000], device='mps:0')

In [6]: pt.full((40,), -10.3, device="mps").int()
Out[6]:
tensor([-1054552883, -1054552883, -1054552883, -1054552883, -1054552883,
        -1054552883, -1054552883, -1054552883, -1054552883, -1054552883,
        -1054552883, -1054552883, -1054552883, -1054552883, -1054552883,
        -1054552883, -1054552883, -1054552883, -1054552883, -1054552883,
        -1054552883, -1054552883, -1054552883, -1054552883, -1054552883,
        -1054552883, -1054552883, -1054552883, -1054552883, -1054552883,
        -1054552883, -1054552883, -1054552883, -1054552883, -1054552883,
        -1054552883, -1054552883, -1054552883, -1054552883, -1054552883],
       device='mps:0', dtype=torch.int32)

In [7]: pt.full((40,), -10.3, device="mps").int().float()
Out[7]:
tensor([-10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000], device='mps:0')

In [8]: pt.full((40,), -10.3, device="mps").int().float().bool()
Out[8]:
tensor([ True, False, False,  True,  True, False, False,  True,  True, False,
        False,  True,  True, False, False,  True,  True, False, False,  True,
         True, False, False,  True,  True, False, False,  True,  True, False,
        False,  True,  True, False, False,  True,  True, False, False,  True],
       device='mps:0')
```

After:
```python
In [3]: pt.full((40,), -10.3, device="mps")
Out[3]:
tensor([-10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000, -10.3000,
        -10.3000, -10.3000, -10.3000, -10.3000, -10.3000], device='mps:0')

In [4]: pt.full((40,), -10.3, device="mps").int()
Out[4]:
tensor([-10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10,
        -10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10,
        -10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10, -10],
       device='mps:0', dtype=torch.int32)

In [5]: pt.full((40,), -10.3, device="mps").int().float()
Out[5]:
tensor([-10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10.,
        -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10.,
        -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10., -10.,
        -10., -10., -10., -10.], device='mps:0')

In [6]: pt.full((40,), -10.3, device="mps").int().float().bool()
Out[6]:
tensor([True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True, True, True, True, True, True, True, True, True,
        True, True, True, True], device='mps:0')
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/78092
Approved by: https://github.com/kulinseth, https://github.com/malfet
 2022-05-24 20:09:45 +00:00
Alban Desmaison
04ac80c73a Fix a few issues on assert/double error/legacy constructor (#77966) 
Fixes https://github.com/pytorch/pytorch/issues/77960, https://github.com/pytorch/pytorch/issues/77957, https://github.com/pytorch/pytorch/issues/77781
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77966
Approved by: https://github.com/soulitzer, https://github.com/kulinseth
 2022-05-20 20:25:12 +00:00
Kulin Seth
3d83321b44 MPS Fixes: copy operations, addmm and baddmm (#77791) 
Fixes for the copy operations and GEMM operations on MPS backend.

Fixes https://github.com/pytorch/pytorch/issues/77819
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77791
Approved by: https://github.com/albanD
 2022-05-20 03:18:11 +00:00
Kulin Seth
978304fc9c MPS: fixes (#77462) 
- Fix the is_available flag for x86 machines
- Fix the tensor creation for older MacOS platforms
- Addmm fixes for transposition

Pull Request resolved: https://github.com/pytorch/pytorch/pull/77462
Approved by: https://github.com/albanD
 2022-05-14 13:33:16 +00:00
Kulin Seth
e011a8e18b Enable PyTorch operations on MPS Backend. (#77343) 
Add PyTorch operations to MPS backend.

- https://github.com/pytorch/pytorch/issues/77394
Pull Request resolved: https://github.com/pytorch/pytorch/pull/77343
Approved by: https://github.com/albanD
 2022-05-13 18:28:53 +00:00