haozhe.zhu 1c3fe84033 [optim] add fused_adagrad support for CPU device (#124905) ... Support fused_sgd_kernel support for CPU. ## Bench result: 32 core/sockets ICX Test Scripts: https://gist.github.com/zhuhaozhe/79e842e0a6e25d6d7fa1e4598807272c https://gist.github.com/zhuhaozhe/b4c6998a509dcea1796dd05b3005c969 ``` Tensor Size: 262144, Num Tensor 4, Num Threads: 1 _single_tensor_adagrad time: 0.2500 seconds _fused_adagrad time: 0.0933 seconds Tensor Size: 4194304, Num Tensor 32, Num Threads: 32 _single_tensor_adagrad time: 2.8819 seconds _fused_adagrad time: 1.7591 seconds ``` ## Test Plan: ``` python test_optim.py -k test_fused_matches_forloop python test_optim.py -k test_fused_large_tensor python test_optim.py -k test_can_load_older_state_dict python test_optim.py -k test_grad_scaling_autocast_fused_optimizers python test_torch.py -k test_grad_scaling_autocast_fused python test_torch.py -k test_params_invalidated_with_grads_invalidated_between_unscale_and_step ``` Co-authored-by: Jane (Yuan) Xu <31798555+janeyx99@users.noreply.github.com> Pull Request resolved: https://github.com/pytorch/pytorch/pull/124905 Approved by: https://github.com/jgong5, https://github.com/janeyx99		2024-05-13 01:16:20 +00:00
..
_internal	[optim] add fused_adagrad support for CPU device (#124905)	2024-05-13 01:16:20 +00:00
__init__.py	document torch.testing.assert_allclose (#89526)	2022-12-01 11:22:50 +00:00
_comparison.py	[BE] enable ruff rule RSE and remove useless parentheses in raise statements (#124261)	2024-04-17 19:29:34 +00:00
_creation.py	additional support for float8_e4m3fnuz and _e5m2fnuz (#115214)	2024-01-22 18:33:41 +00:00