tao-run-inference-service

You can deploy a TAO inference microservice on Kubernetes by using this Skill to automate container execution, handle image resolution, and construct job payloads for the cluster. It manages the deployment lifecycle automatically.

Yes, the TAO inference service deployment supports HuggingFace Hub models for inference. You can integrate these models directly when constructing job payloads and deploying the microservice containers.

TAO inference microservices can be deployed on local Docker, Brev, Slurm, or Kubernetes. The Skill resolves platform-specific requirements and integrates with platform-specific skills to manage execution.

After deploying the inference microservice, you can send an inference request to the running service by using the inference_request functionality. This handles interaction with the service and manages its lifecycle.

Yes, tao_sdk is a required dependency for deploying and managing TAO inference microservices. You must install it before using the Skill to automate container execution and service registry.