skills.rest
guanxiaol
guanxiaolCOMMUNITY
Explore all skills by this author →
GitHub

managed-model-endpoints

Register and manage model endpoints for local and remote inference services.

Updated Aug 23, 2026
Search Tags:#Docker#Python#model management#model endpoint#inference service
One-click install
npx skills add https://github.com/guanxiaol/open-science --skill managed-model-endpoints
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: managed-model-endpoints
Source: https://github.com/guanxiaol/open-science/tree/main/runtime/skills/managed-model-endpoints
Command: npx skills add https://github.com/guanxiaol/open-science --skill managed-model-endpoints

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires Docker, Python, and includes scripts (resource) components.

What problem does it solve?

This Skill simplifies the process of registering and using model endpoints for local and remote inference services, eliminating the need for manual setup and management.

Core Features & Use Cases

  • Model Registration: Register local model servers or remote model APIs for on-demand inference.
  • Endpoint Management: Manage the lifecycle of registered endpoints, including starting, stopping, and connecting to them.
  • Inference Execution: Execute inference requests against registered endpoints using the compute_provider tool.
  • Use Case: For a user who wants to quickly set up a local model server for a deep learning model and connect it to their project for real-time inference.

Quick Start

Register a local model endpoint named 'local-model-service' with the following command:

port = host.model_endpoints.free_port()
host.model_endpoints.register(
    name="local-model-service",
    url=f"http://127.0.0.1:{port}",
    credential="NVIDIA_API_KEY",
    skill="<model-runbook-skill>",
    start=START_SCRIPT,
    stop="docker stop local-model-service",
    live="/v1/health/ready"
)

Frequently Asked Questions about managed-model-endpoints

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I manage local Docker model endpoints for inference services?▼

To manage local Docker model endpoints, you can register your model server containers with a start and stop script to handle on-demand inference and lifecycle management. This allows you to control endpoint availability without manual setup.

Can I register remote model APIs for on-demand inference using Python?▼

Yes, you can register remote model APIs for on-demand inference using Python. The system supports both local model server containers and remote APIs, allowing you to execute inference requests against registered endpoints through the compute_provider tool.

Do I need Docker and Python to set up model server containers?▼

Yes, you need both Docker and Python to set up and manage model server containers. Docker is required to run the local model server containers, while Python is used to execute the registration scripts and manage the endpoint lifecycle.

What is the best way to handle model endpoint lifecycle management and health checks?▼

The best way to handle model endpoint lifecycle management is to register the endpoint with specific start and stop commands alongside a live health check URL. This enables automatic monitoring of endpoint readiness and allows on-demand starting and stopping of the inference service.

How do I execute inference requests against a registered model endpoint?▼

You execute inference requests against a registered model endpoint by using the compute_provider tool. Once a local or remote endpoint is registered with its URL and credentials, the tool routes on-demand inference queries to the active model server.

Related Skills

Explore More Software Engineering Skills→
breakcafebreakcafe
1

setup

Automate NanoClaw initial deployment with environment checks and runtime selection.

Official
Advanced
randoneeringrandoneering

spcs-inference

Deploy registered Snowflake models to Snowpark Container Services for real-time inference.

Community
Advanced
stevengonsalvezstevengonsalvez

add-ollama-tool

Expose local Ollama models as tools via an MCP server.

Community
Advanced
tylerjrbuelltylerjrbuell
21

mcp-tool-integration

Connect Reactive Agents to MCP servers with automatic transport detection and Docker lifecycle management.

Community
Advanced