Model Export & Serving

Export models to ONNX or TorchScript by tracing the architecture and validating the resulting graph. This skill provides workflows to ensure the exported model is production-ready and numerically equivalent to the original.

Serve ONNX or TorchScript models using a FastAPI application deployed within containers. This approach provides a scalable API server capable of handling inference requests across CPU and GPU environments.

Apply dynamic or static quantization to reduce model size and increase inference speed. This skill includes validation and testing steps to verify numerical equivalence after applying quantization strategies.

Yes, the export pipeline validates ONNX and TorchScript graphs and tests numerical equivalence. This ensures the exported and quantized models maintain reliable inference outputs before production deployment.

Yes, the serving architecture supports deployment across both CPU and GPU environments. Containerized pipelines and FastAPI endpoints allow flexible execution regardless of the underlying hardware.

You need PyTorch, ONNX, ONNX Runtime, and Pytest installed. These dependencies support the export, quantization, numerical validation, and serving infrastructure required for the deployment pipeline.