PyTorch MergeBot
|
0cb5bc3b04
|
Revert "Move tensor grouping to ATen (#100007)"
This reverts commit 74b7a6c75e.
Reverted https://github.com/pytorch/pytorch/pull/100007 on behalf of https://github.com/izaitsevfb due to Breaks internal builds, see D46629727 ([comment](https://github.com/pytorch/pytorch/pull/100007#issuecomment-1587861598))
|
2023-06-12 18:30:33 +00:00 |
|
Masaki Kozuki
|
74b7a6c75e
|
Move tensor grouping to ATen (#100007)
rel: #94344
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100007
Approved by: https://github.com/janeyx99
|
2023-06-09 15:44:46 +00:00 |
|
shibo19
|
c24b61bc20
|
Enable torch._C._get_privateuse1_backend_name in Dynamo tracing (#103141)
Fixes https://github.com/pytorch/pytorch/issues/103125
torch._C._get_privateuse1_backend_name() will cause graph break, so I add it to the functions.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103141
Approved by: https://github.com/yanboliang
|
2023-06-09 09:19:33 +00:00 |
|
shibo19
|
e4a42bcf56
|
add foreach support for custom device (#102047)
Fixes #ISSUE_NUMBER
for custom device, we want to support foreach, so I add a func that we could set other device type, and the default value is cuda.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102047
Approved by: https://github.com/janeyx99
|
2023-06-07 13:59:20 +00:00 |
|
PyTorch MergeBot
|
9d77949b9e
|
Revert "add foreach support for custom device (#102047)"
This reverts commit b088ff4677.
Reverted https://github.com/pytorch/pytorch/pull/102047 on behalf of https://github.com/malfet due to Broke inductor, see b088ff4677 ([comment](https://github.com/pytorch/pytorch/pull/102047#issuecomment-1572368942))
|
2023-06-01 16:33:03 +00:00 |
|
shibo19
|
b088ff4677
|
add foreach support for custom device (#102047)
Fixes #ISSUE_NUMBER
for custom device, we want to support foreach, so I add a func that we could set other device type, and the default value is cuda.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102047
Approved by: https://github.com/janeyx99
|
2023-06-01 06:22:44 +00:00 |
|
Aaron Gokaslan
|
e2a3817dfd
|
[BE] Enable C419 rule for any all shortcircuiting (#99890)
Apparently https://github.com/pytorch/pytorch/pull/78142 made torch.JIT allow for simple generator expressions which allows us to enable rules that replace unnecessary list comprehensions with generators in any/all. This was originally part of #99280 but I split it off into this PR so that it can be easily reverted should anything break.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/99890
Approved by: https://github.com/justinchuby, https://github.com/kit1980, https://github.com/malfet
|
2023-04-25 15:02:13 +00:00 |
|
Jane Xu
|
8c9f745af1
|
[foreach] guard default support on native tensors only (#92923)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92923
Approved by: https://github.com/ngimel, https://github.com/crcrpar
|
2023-01-26 04:52:58 +00:00 |
|
milesial
|
e4d83d54a6
|
Foreach gradient clipping (#91846)
Faster gradient clipping using the foreach functions
```
[------------------------ (tensors, scalar) -------------------------]
| without foreach | with foreach | apex
1 threads: ----------------------------------------------------------------------
10 tensors of size 4 | 120.5 | 61.1 | 50.3
100 tensors of size 4 | 946.2 | 239.5 | 136.3
1000 tensors of size 4 | 9808.5 | 2151.1 | 1006.9
10000 tensors of size 4 | 96871.2 | 22637.4 | 10119.1
10 tensors of size 16 | 121.0 | 64.1 | 52.5
100 tensors of size 16 | 993.4 | 252.6 | 136.7
1000 tensors of size 16 | 9427.7 | 2151.2 | 1049.5
10000 tensors of size 16 | 97437.1 | 22203.1 | 10340.0
10 tensors of size 256 | 118.9 | 62.3 | 51.5
100 tensors of size 256 | 955.2 | 243.1 | 134.2
1000 tensors of size 256 | 9374.9 | 2140.7 | 1009.6
10000 tensors of size 256 | 95302.5 | 21849.4 | 10215.5
10 tensors of size 65536 | 118.5 | 62.4 | 51.1
100 tensors of size 65536 | 1740.7 | 243.3 | 225.3
1000 tensors of size 65536 | 17364.1 | 2228.7 | 2004.5
10000 tensors of size 65536 | 177510.1 | 25410.4 | 20678.2
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/91846
Approved by: https://github.com/janeyx99
|
2023-01-20 21:43:29 +00:00 |
|
Jane Xu
|
a41f00ed70
|
[optim][sgd] group tensors in foreach to maximize perf (#92338)
Make foreach faster for SGD
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92338
Approved by: https://github.com/albanD
|
2023-01-18 04:02:41 +00:00 |
|
Jane Xu
|
ed7885c254
|
[utils][foreach] Add group tensor by device and dtype util (#92014)
Add util that will be commonly used throughout optim
Pull Request resolved: https://github.com/pytorch/pytorch/pull/92014
Approved by: https://github.com/albanD
|
2023-01-11 23:37:20 +00:00 |
|