drisspg 4e29f01bf2 Remove sdp_kernel and replace with sdpa_kernel in attention namespace (#114689) ... # Summary Simplification of Backend Selection This PR deprecates the `torch.backends/cuda/sdp_kernel` context manager and replaces it with a new context manager `torch.nn.attention.sdpa_kernel`. This context manager also changes the api for this context manager. For `sdp_kernel` one would specify the backend choice by taking the negation of what kernel they would like to run. The purpose of this backend manager was to only to be a debugging tool, "turn off the math backend" and see if you can run one of the fused implementations. Problems: - This pattern makes sense if majority of users don't care to know anything about the backends that can be run. However, if users are seeking to use this context manager then they are explicitly trying to run a specific backend. - This is not scalable. We are working on adding the cudnn backend and this API makes it so so that more implementations will need to be turned off if user wants to explicitly run a given backend. - Discoverability of the current context manager. It is somewhat un-intutive that this backend manager is in backends/cuda/init when this now also controls the CPU fused kernel behavior. I think centralizing to attention namespace will be helpful. Other concerns: - Typically backends (kernels) for operators are entirely hidden from users and implementation details of the framework. We have exposed this to users already, albeit not by default and with beta warnings. Does making backends choices even more explicit lead to problems when we potentially want to remove existing backends, (perhaps inputs shapes will get covered by newer backends). A nice side effect is now that we aren't using the `BACKEND_MAP` in test_transformers many, many dynamo failures are passing for CPU tests. Pull Request resolved: https://github.com/pytorch/pytorch/pull/114689 Approved by: https://github.com/cpuhrsch		2024-01-24 22:28:04 +00:00
..
_static	Removing HTA documentation (#116513)	2023-12-28 23:04:23 +00:00
_templates	Remove sdp_kernel and replace with sdpa_kernel in attention namespace (#114689)	2024-01-24 22:28:04 +00:00
community	Add thiagocrepaldi as person of interest for onnx exporter (#113402)	2023-11-10 15:19:58 +00:00
elastic	[TorchElastic] Support for overprovisioning in C10 based rendezvous (#117066)	2024-01-18 01:16:55 +00:00
notes	[docs] start a new FSDP notes doc (#117323)	2024-01-22 15:46:35 +00:00
rpc
scripts	Fixed typo in build_activation_images.py (#117458)	2024-01-15 03:27:40 +00:00
amp.rst	Add torch.utils.deterministic.fill_uninitialized_memory flag (#111377)	2023-11-01 16:10:09 +00:00
autograd.rst	Allow specifiying inputs as GradientEdge in autograd APIs (#110867)	2023-10-12 04:08:44 +00:00
backends.rst	Remove sdp_kernel and replace with sdpa_kernel in attention namespace (#114689)	2024-01-24 22:28:04 +00:00
benchmark_utils.rst
bottleneck.rst
checkpoint.rst	Add set_checkpoint_debug_enabled that overrides local setting (#110728)	2023-10-11 02:12:31 +00:00
complex_numbers.rst	Update mentions of deprecated functions if complex_numbers.rst (#113391)	2023-11-09 22:32:26 +00:00
cond.rst	[HigherOrderOp] expose torch.cond (#110293)	2023-10-07 20:39:52 +00:00
conf.py	Initial torchbind support in PT2 (#117697)	2024-01-19 06:28:20 +00:00
config_mod.rst
cpp_extension.rst
cpp_index.rst
cpu.rst	Add current_device() to torch.cpu (#110987)	2023-10-11 05:13:10 +00:00
cuda._sanitizer.rst
cuda.rst	Doc test non packages (#110568)	2023-10-06 14:16:01 +00:00
cudnn_persistent_rnn.rst
cudnn_rnn_determinism.rst
data.rst	Revert "reseed all Generators in Dataloader's _worker_loop() -- via GC (#107131)"	2023-08-23 17:08:07 +00:00
ddp_comm_hooks.rst	[DOCS][DDP]Fix the simple of saving and reloading PowerSGD state and hook. (#102721)	2023-06-10 00:15:00 +00:00
deploy.rst
deterministic.rst	Add torch.utils.deterministic.fill_uninitialized_memory flag (#111377)	2023-11-01 16:10:09 +00:00
distributed.algorithms.join.rst
distributed.checkpoint.rst	adds async save, makes checkpointer private (#116293)	2023-12-22 05:22:39 +00:00
distributed.elastic.rst
distributed.optim.rst
distributed.rst	adds async save, makes checkpointer private (#116293)	2023-12-22 05:22:39 +00:00
distributed.tensor.parallel.rst	[tp] add clarification to doc and improve TP examples (#117618)	2024-01-22 18:56:50 +00:00
distributions.rst	Add inverse gamma distribution and fix sign bug in PowerTransform. (#104501)	2023-11-01 02:26:25 +00:00
dlpack.rst
docutils.conf
export.ir_spec.rst	[export][refactor][4/n] Make equality_constraints optional (#116233)	2024-01-05 00:50:52 +00:00
export.rst	[fx] Add hooks to intercept node replacements. (#117825)	2024-01-23 22:28:40 +00:00
fft.rst
fsdp.rst	[FSDP][state_dict] Expose optimizer state_dict config (#105949)	2023-08-21 07:29:49 +00:00
func.api.rst
func.batch_norm.rst	Fix typo under docs directory (#97202)	2023-03-21 01:24:10 +00:00
func.migrating.rst
func.rst
func.ux_limitations.rst
func.whirlwind_tour.rst
futures.rst
fx.rst	Split SymNode into its own file (#112037)	2023-10-26 23:32:27 +00:00
hub.rst
index.rst	Remove sdp_kernel and replace with sdpa_kernel in attention namespace (#114689)	2024-01-24 22:28:04 +00:00
jit_builtin_functions.rst
jit_language_reference_v2.rst	Fix typo under docs directory (#97202)	2023-03-21 01:24:10 +00:00
jit_language_reference.rst
jit_python_reference.rst
jit_unsupported.rst	Add support for torch.Generator type in TorchScript (#110413)	2023-11-21 23:07:21 +00:00
jit_utils.rst
jit.rst	Doc test non packages (#110568)	2023-10-06 14:16:01 +00:00
library.rst	Rewrite torch.library's documentation (#111310)	2023-10-23 23:02:41 +00:00
linalg.rst
logging.rst	[Easy] log graphs in compiled_autograd if TORCH_LOGS=compiled_autograd (#108991)	2023-09-12 00:15:02 +00:00
masked.rst	Doc test non packages (#110568)	2023-10-06 14:16:01 +00:00
math-quantizer-equation.png
mobile_optimizer.rst
model_zoo.rst
monitor.rst
mps.rst	Doc test non packages (#110568)	2023-10-06 14:16:01 +00:00
multiprocessing.rst	Doc test non packages (#110568)	2023-10-06 14:16:01 +00:00
name_inference.rst	[docs] Properly link register_post_accumulate_grad_hook docs (#108157)	2023-08-29 22:13:33 +00:00
named_tensor.rst	fixing named tensor unflatten example (#106921)	2023-08-22 18:00:10 +00:00
nested.rst	Replace master with main in links and docs/conf.py (#100176)	2023-05-02 18:20:32 +00:00
nn.attention.bias.rst	Remove sdp_kernel and replace with sdpa_kernel in attention namespace (#114689)	2024-01-24 22:28:04 +00:00
nn.attention.rst	Remove sdp_kernel and replace with sdpa_kernel in attention namespace (#114689)	2024-01-24 22:28:04 +00:00
nn.functional.rst	Add python and C++ support for LPPool3d (#114199)	2023-12-08 18:18:44 +00:00
nn.init.rst
nn.rst	Remove sdp_kernel and replace with sdpa_kernel in attention namespace (#114689)	2024-01-24 22:28:04 +00:00
onnx_dynamo_onnxruntime_backend.rst	Follow-up #108379 (#108905)	2023-09-09 01:38:36 +00:00
onnx_dynamo.rst	[ez][doc] Fix sample code in onnx_dynamo.rst (#114770)	2023-11-29 19:27:52 +00:00
onnx_torchscript_supported_aten_ops.rst	Refactor torch.onnx documentation (#108379)	2023-09-08 18:23:48 +00:00
onnx_torchscript.rst	Follow-up #108379 (#108905)	2023-09-09 01:38:36 +00:00
onnx.rst	Doc test non packages (#110568)	2023-10-06 14:16:01 +00:00
optim.rst	Added example regarding weight_decay distinction with per-parameter API (#117436)	2024-01-22 21:26:02 +00:00
package.rst	Doc test non packages (#110568)	2023-10-06 14:16:01 +00:00
pipeline.rst	docs: Linking ResNeXt PyTorch Hub Pipeline (#98689)	2023-04-11 02:20:26 +00:00
profiler.rst	Doc test non packages (#110568)	2023-10-06 14:16:01 +00:00
quantization-accuracy-debugging.rst
quantization-backend-configuration.rst
quantization-support.rst	[quant][pt2e] Add generate_numeric_debug_handle pass (#114315)	2023-12-01 03:38:17 +00:00
quantization.rst	[ao] updating embedding_bag support for fx and eager (#107623)	2023-11-21 03:54:00 +00:00
random.rst
rpc.rst	[BE] RPC is missing RRef docs (#106902)	2023-08-10 16:26:27 +00:00
signal.rst
sparse.rst	Fix typo in https://pytorch.org/docs/stable/sparse.html (#115282)	2023-12-08 18:31:33 +00:00
special.rst
storage.rst
tensor_attributes.rst
tensor_view.rst
tensorboard.rst
tensors.rst	add float8 types to dtypes table (#117375)	2024-01-15 00:23:07 +00:00
testing.rst
torch_cuda_memory.rst	Fix typo under docs directory (#110359)	2023-10-03 16:36:05 +00:00
torch.ao.ns._numeric_suite_fx.rst
torch.ao.ns._numeric_suite.rst
torch.compiler_aot_inductor.rst	[AOTI][refactor] Organize model runner files (#116022)	2023-12-20 15:35:34 +00:00
torch.compiler_api.rst	Add cudagraph_mark_step_begin in torch.compiler, reference in error message (#111722)	2023-10-25 21:53:21 +00:00
torch.compiler_best_practices_for_backends.rst	Restructure torch.compile docs (#105376)	2023-07-28 20:58:57 +00:00
torch.compiler_cudagraph_trees.rst	[docs] add mode="reduce-overhead" into torch.compile to enable cuda g… (#116529)	2024-01-05 22:54:20 +00:00
torch.compiler_custom_backends.rst	[docs, dynamo] fix typos in dynamo custom backend docs (#115444)	2023-12-08 23:58:26 +00:00
torch.compiler_deepdive.rst	[Dynamo]Expose bytecode hooks and add example usage for decompilation in docs (#110714)	2023-10-13 12:36:00 +00:00
torch.compiler_dynamic_shapes.rst	Update dynamic shapes documentation (#109764)	2023-09-21 13:53:43 +00:00
torch.compiler_fake_tensor.rst	Restructure torch.compile docs (#105376)	2023-07-28 20:58:57 +00:00
torch.compiler_faq.rst	Add a wrapper to transform a NumPy function into a PyTorch function (#114610)	2024-01-02 18:35:29 +00:00
torch.compiler_fine_grain_apis.rst	Restructure torch.compile docs (#105376)	2023-07-28 20:58:57 +00:00
torch.compiler_get_started.rst	[Reland2] [inductor][BE] split triton_meta and inductor_meta (#112351)	2023-11-02 00:40:12 +00:00
torch.compiler_guards_overview.rst	Do not use a specific LOC in link (#108957)	2023-09-13 19:21:45 +00:00
torch.compiler_inductor_profiling.rst	Restructure torch.compile docs (#105376)	2023-07-28 20:58:57 +00:00
torch.compiler_ir.rst	[export] torch.export landing page (#108783)	2023-09-10 01:40:42 +00:00
torch.compiler_nn_module.rst	Revert "Reland 3rd try [finishing colesbury's PR 100642] Guard on nn.Module dicts and type (#109323)" + Forward fixes + test (#110964)	2023-10-11 05:16:47 +00:00
torch.compiler_performance_dashboard.rst	Restructure torch.compile docs (#105376)	2023-07-28 20:58:57 +00:00
torch.compiler_profiling_torch_compile.rst	Restructure torch.compile docs (#105376)	2023-07-28 20:58:57 +00:00
torch.compiler_transformations.rst	Fix typo under docs directory (#110359)	2023-10-03 16:36:05 +00:00
torch.compiler_troubleshooting.rst	Update torch.compiler_troubleshooting.rst (#114530)	2023-11-25 23:15:47 +00:00
torch.compiler.rst	[docs] Fix torch.compile "tensorrt" backend docs (#113711)	2023-11-15 08:42:53 +00:00
torch.overrides.rst	Doc test non packages (#110568)	2023-10-06 14:16:01 +00:00
torch.rst	Some tiny modification about torch.set/get_default_device (#116014)	2023-12-19 05:08:06 +00:00
type_info.rst
utils.rst	New swap function (#111747)	2023-12-08 18:49:35 +00:00