PyTorch Custom Operation
5 days ago
- #AOTInductor
- #PyTorch
- #Custom Operations
- PyTorch custom operations can be implemented in C++ and CUDA, including custom functions and classes.
- Custom functions are stateless and registered via TORCH_LIBRARY_IMPL, with dispatchers for CPU and CUDA.
- Custom classes use torch::CustomClassHolder for stateful operations and require registration with TORCH_LIBRARY.
- Shared libraries (e.g., libidentity_conv_ops.so) enable loading custom ops in both Python and C++.
- For torch.export compatibility, fake implementations of custom classes and ops must be registered.
- Models with custom ops can be exported and compiled with AOTInductor into a deployable package.
- Inference programs in Python or C++ load the compiled model and custom ops library for execution.