pytorch/torch
Jerry Zhang 1b51d29b66 [quant][pt2e] Enable constant folding for quantize ops (#109343)
Summary:
This PR added constant folding for quantize ops so that instead of storing fp32 weight in the
quantized model, we'll get int8/int16 etc. weight

Test Plan:
python test/test_quantization.py TestQuantizePT2E.test_fold_quantize

also will verify in executorch later

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D49399210](https://our.internmc.facebook.com/intern/diff/D49399210)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/109343
Approved by: https://github.com/kimishpatel, https://github.com/jgong5
2023-09-27 06:04:45 +00:00
..
_awaits
_C Rename torch._C._TensorBase to TensorBase (#109940) 2023-09-25 19:10:22 +00:00
_C_flatbuffer
_custom_op Add torch.library.impl_abstract (#109912) 2023-09-26 01:59:50 +00:00
_decomp [core IR] Add a core decomposition for aten.all (#110093) 2023-09-27 01:31:41 +00:00
_dispatch
_dynamo Revert "[Dynamo] Match closures by code ID (#109427)" 2023-09-26 18:54:36 +00:00
_export [export] Add dynamic_shapes to _export.aot_compile (#110101) 2023-09-27 04:10:22 +00:00
_functorch Revert "Reland "Update AOTAutograd to use FunctionalTensorMode instead of C++ functionalization (#106406)" (#109906)" 2023-09-26 12:10:25 +00:00
_higher_order_ops torch.compile + selective activation checkpointing (#105489) 2023-09-21 16:24:11 +00:00
_inductor [quant][pt2e] Enable constant folding for quantize ops (#109343) 2023-09-27 06:04:45 +00:00
_lazy
_library Add torch.library.impl_abstract (#109912) 2023-09-26 01:59:50 +00:00
_logging [BE]: Replace undocumented constant in logging (#109434) 2023-09-16 20:17:32 +00:00
_numpy Dispatch numpy.take_along_axis to torch.take_along_dim (#108880) 2023-09-13 23:13:09 +00:00
_prims fix infinite loop with primtorch and .to(meta) (#109632) 2023-09-22 07:09:04 +00:00
_prims_common Use _check_is_size for validate_dim_length (#109849) 2023-09-26 23:33:31 +00:00
_refs [core IR] Add a core decomposition for aten.all (#110093) 2023-09-27 01:31:41 +00:00
_subclasses Add masked_select abstract impl (#110103) 2023-09-27 04:07:58 +00:00
amp Unblock float16 dtype for xla autocasting (#109554) 2023-09-21 03:19:44 +00:00
ao [quant][pt2e] Enable constant folding for quantize ops (#109343) 2023-09-27 06:04:45 +00:00
autograd Setup_context does not contain default values of forward() (#108561) 2023-09-19 16:23:52 +00:00
backends [BE]: enable ruff rules PLR1722 and PLW3301 (#109461) 2023-09-18 02:07:21 +00:00
compiler
contrib
cpu
csrc [inductor] support _scaled_dot_product_flash_attention fallback (#110085) 2023-09-27 00:09:56 +00:00
cuda Improve torch.cuda.amp type hints (#108630) 2023-09-08 06:06:25 +00:00
distributed [dtensor][7/n] remove reduction rule (#109144) 2023-09-26 22:24:50 +00:00
distributions Spelling fix (#108490) 2023-09-04 16:59:35 +00:00
export [export] Verifier for exported program (#109519) 2023-09-26 18:47:43 +00:00
fft
func
futures
fx Add test for ShapeEnv recording fallback. (#109944) 2023-09-27 00:50:14 +00:00
jit Python 3.10 Union operator | support for JIT (#109293) 2023-09-25 15:35:54 +00:00
legacy
lib [RELAND] Remove some unnecessary <iostream> includes from headers (#108150) 2023-09-20 21:55:15 +00:00
linalg fix matrix_power documentation bug (#108585) 2023-09-05 22:08:46 +00:00
masked
monitor
mps
multiprocessing
nested Add NestedTensor python subclass (#108314) 2023-09-11 18:29:20 +00:00
nn Improved DDP checkpoint documentation (#106985) 2023-09-25 22:54:24 +00:00
onnx Revert "[ONNX] Remove the depreacated function _export (#109763)" 2023-09-25 17:47:21 +00:00
optim [torch] add use_buffers to swa_utils interface (#109078) 2023-09-19 21:30:59 +00:00
package removing some redundant str splits (#106089) 2023-09-01 00:22:58 +00:00
profiler [profiler] move _enable_dynamo_cache_lookup_profiler (#107720) 2023-08-23 23:41:35 +00:00
quantization
signal
sparse Minor fixes in semi-structured sparse code (#105595) 2023-09-25 14:06:08 +00:00
special
testing Add masked_select abstract impl (#110103) 2023-09-27 04:07:58 +00:00
utils Revert "Reland "Update AOTAutograd to use FunctionalTensorMode instead of C++ functionalization (#106406)" (#109906)" 2023-09-26 12:10:25 +00:00
__config__.py
__future__.py
__init__.py Add torch.library.impl_abstract (#109912) 2023-09-26 01:59:50 +00:00
_appdirs.py
_classes.py
_compile.py
_custom_ops.py Add torch.library.impl_abstract (#109912) 2023-09-26 01:59:50 +00:00
_deploy.py
_guards.py [RELAND] Force synced KJT to trace unbacked SymInt (#108960) (#109216) 2023-09-18 14:39:44 +00:00
_jit_internal.py Python 3.10 Union operator | support for JIT (#109293) 2023-09-25 15:35:54 +00:00
_linalg_utils.py
_lobpcg.py [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
_lowrank.py
_meta_registrations.py Fix python decomps for OpOverloadPackets and add tests (#107707) 2023-09-25 20:53:30 +00:00
_namedtensor_internals.py [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
_ops.py python functionalization: support higher order ops (#108656) 2023-09-20 04:37:31 +00:00
_python_dispatcher.py
_sources.py
_storage_docs.py
_tensor_docs.py Rename torch._C._TensorBase to TensorBase (#109940) 2023-09-25 19:10:22 +00:00
_tensor_str.py
_tensor.py Rename torch._C._TensorBase to TensorBase (#109940) 2023-09-25 19:10:22 +00:00
_torch_docs.py Improved the docs for torch.std, torch.var, torch.std_mean, torch.var_mean and torch.cov (#109326) 2023-09-19 20:47:24 +00:00
_utils_internal.py [TORCH_LIBRARY] Add impl_abstract_pystub (#109529) 2023-09-22 04:55:36 +00:00
_utils.py Hide __getattr__ from type checkers (#109683) 2023-09-21 17:01:23 +00:00
_VF.py
_vmap_internals.py
_weights_only_unpickler.py
abi-check.cpp
CMakeLists.txt Clean up CMake target linking (#109959) 2023-09-25 01:37:14 +00:00
custom_class_detail.h
custom_class.h [RELAND] Remove some unnecessary <iostream> includes from headers (#108150) 2023-09-20 21:55:15 +00:00
extension.h reduce header file to boost cpp_wrapper build. (#107585) 2023-08-22 11:58:47 +00:00
functional.py Improve docs for torch.unique dim argument (#108292) 2023-09-02 11:09:09 +00:00
hub.py Default permissions for torch.hub downloads (#82869) 2023-08-24 15:48:24 +00:00
library.h [TORCH_LIBRARY] Add impl_abstract_pystub (#109529) 2023-09-22 04:55:36 +00:00
library.py Add torch.library.impl_abstract (#109912) 2023-09-26 01:59:50 +00:00
overrides.py Disabled UserWarnings for some public functions in torch.overrides (#109890) 2023-09-23 20:40:04 +00:00
py.typed
quasirandom.py [BE]: Update ruff to 0.285 (#107519) 2023-08-22 23:16:38 +00:00
random.py Revert "Re-land: Break graph on manual_seed. (#108647)" 2023-09-08 21:18:00 +00:00
README.txt
return_types.py
script.h
serialization.py Fix hpu deserialization bug (#109499) 2023-09-19 00:10:51 +00:00
storage.py
torch_version.py
types.py [BE]: Apply PYI autofixes to various types (#107521) 2023-08-20 02:42:21 +00:00
version.py.tpl

Note [TH abstraction violation]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TH/THC provide some hpp headers, which are proper C++ headers rather than
C headers.  These headers serve double duty as *internal implementation
detail* headers, whose contents should largely not be used by external
clients.

Ideally, we would not install these headers at all; instead, you should
use public functions (in headers like `THTensor.h`, NOT `THTensor.hpp`)
to manipulate these structs.  However, there are a few places
in torch/csrc where we violate this abstraction.  They are marked with
a pointer to this note.  Each of those sites will have to be refactored
when we refactor the guts of THTensor and related structures.