OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
253059356f
pytorch/docs/source/notes/libtorch_stable_abi.md
Scott Wolchok 2f74cffab2 Remove reinterpret_casts with undefined behavior from stable/library.h (#151595) 
There is a list of valid uses of `reinterpret_cast` (see https://en.cppreference.com/w/cpp/language/reinterpret_cast), and the use here was not on the list, hence undefined behavior. Implement what we meant using memcpy, which is well-defined.

Differential Revision: [D73200791](https://our.internmc.facebook.com/intern/diff/D73200791/)
Pull Request resolved: https://github.com/pytorch/pytorch/pull/151595
Approved by: https://github.com/janeyx99
2025-04-22 20:24:47 +00:00

2.7 KiB
Raw Blame History

LibTorch Stable ABI

This note will eventually contain more details on how to use the APIs in torch/csrc/stable. For the moment, it contains a table of internal representations:

  1. type in custom extension: type used within the end user custom library.
  2. StableIValue representation: a stable conversion of the type to liaison between the user model vs libtorch.so in an ABI-stable manner.
  3. type in libtorch: type used within libtorch.so (or any code binary locked with libtorch).
  4. Schema Type: type as described by the schema, which we hail as the source of truth for both ATen ops in native_functions.yaml and for user defined custom operators registered to the dispatcher via TORCH_LIBRARY or torch.library.
type in custom extension StableIValue representation type in libtorch Schema Type
std::optional<S> raw bitwise copy into leading bytes of uint64_t of pointer to a new StableIValue representing S std::optional<T> Type?
std::nullopt nullptr IValue() None
RAIIATH raw bitwise copy of underlying AtenTensorHandle into leading bytes of uint64_t at::Tensor Tensor
int32_t raw bitwise copy into leading bytes of uint64_t at::ScalarType ScalarType
int32_t raw bitwise copy into leading bytes of uint64_t at::Layout Layout
int32_t raw bitwise copy into leading bytes of uint64_t at::MemoryFormat MemoryFormat
bool raw bitwise copy into leading bytes of uint64_t bool bool
int64_t raw bitwise copy into leading bytes of uint64_t int64_t int
double raw bitwise copy into leading bytes of uint64_t double float
? ? c10::Device Device
? ? c10::Stream Stream
? ? c10::complex complex
? ? at::Scalar Scalar
? ? std::string/const char*/ivalue::ConstantString str
? ? at::Storage Storage
? ? at::Generator Generator
? ? c10::List<T> Type[]
? ? ivalue::Tuple<T> (Type, ...)
? ? c10::SymInt SymInt
? ? c10::SymFloat SymFloat
? ? c10::SymBool SymBool
? ? at::QScheme QScheme

Our confidently supported types are the ones in the table that have completed rows. For a limited set of use cases, we also implicitly support any literal type that is representable within 64 bits as StableIValues, as the default reinterpret_cast will succeed. You can work with StableIValue abstractions in your custom kernel for types such as c10::Device even if there is no standard defined representation of device in custom extensions. For example, a custom operator can take as argument a StableIValue device and directly pass it through to an aten operator with aoti_torch_call_dispatcher.