hf-cloud-sagemaker-production-defaults

What problem does it solve?

This Skill automates the creation of SageMaker endpoints with production-ready configurations, including autoscaling, CloudWatch alarms, and tagging, streamlining the deployment process and reducing the risk of configuration errors.

Core Features & Use Cases

• Automated Endpoint Creation: Deploy SageMaker endpoints with pre-configured autoscaling, alarms, and tagging, ensuring a standardized production setup.
• Real-time and Async Deployments: Supports both real-time and asynchronous inference endpoint configurations.
• Use Case: When you need to deploy a SageMaker endpoint that will be serving live predictions, this skill helps set up everything needed for a robust deployment with minimal configuration.

Quick Start

Deploy a real-time endpoint with the following command:

python scripts/deploy.py \
    --model-name <model-name> \
    --image-uri <image-uri> \
    --inference-ami-version <ami> \
    --role-arn <role-arn> \
    --instance-type <instance-type> \
    --region <region> \
    --env SM_VLLM_MODEL=<model-id> \
    --env SM_VLLM_HOST=0.0.0.0 \
    --env SM_VLLM_TRUST_REMOTE_CODE=true \
    --env SM_VLLM_MAX_MODEL_LEN=4096