skills.rest
inference-sh
inference-shOFFICIAL
Explore all skills by this author →
WebsiteGitHub

building-inferencesh-apps

Build, test, and deploy Python or Node.js applications on the inference.sh platform.

723|101|Updated Jan 31, 2026
Search Tags:#node.js#Python#usage tracking#app deployment#inference.sh#belt cli#gpu apps
One-click install
npx skills add https://github.com/inference-sh/skills --skill building-inferencesh-apps
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: building-inferencesh-apps
Source: https://github.com/inference-sh/skills/tree/main/sdk/building-apps
Command: npx skills add https://github.com/inference-sh/skills --skill building-inferencesh-apps

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Developers building apps on inference.sh often scaffold projects incorrectly, misconfigure GPU resources, or lose usage tracking data due to subtle mistakes like extending the wrong base class. This Skill guides the full app lifecycle — scaffolding, local testing, deployment, and cloud verification — using the belt CLI so apps are structured and configured correctly from the start.

Core Features & Use Cases

  • CLI-Driven Scaffolding: Enforces belt app init for project creation, generating correct inf.yml, inference.py/inference.js, and dependency files for both Python and Node.js.
  • Mandatory Development Workflow: Walks through scaffold → implement → local test → deploy → cloud verification, including output_meta validation for usage-based pricing.
  • GPU & Resource Configuration: Covers VRAM/RAM settings, accelerate-based device detection, device_map for fast large-model loading, and CPU-only app setups.
  • Use Case: You want to wrap an external image-generation API as an inference.sh app. The Skill provides the API-wrapper template, secrets declaration, File handling, and ImageMeta usage tracking so the deployed app reports accurate per-request metadata.

Quick Start

Ask the AI to scaffold a new inference.sh app called my-app, implement a run function, test it locally, and deploy it with the belt CLI.

Frequently Asked Questions about building-inferencesh-apps

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create and deploy an app on inference.sh?▼

Run `belt app init my-app` to scaffold the project, implement inference.py or inference.js, then test locally with `belt app test` and deploy with `belt app deploy`. Always cd into the app directory first, since shell cwd does not persist between commands.

How do I configure GPU resources in inf.yml?▼

Set resources.gpu with count, vram in GB, and type (any, nvidia, amd, apple, or none). For CPU-only apps, set gpu count to 0 and type to none, with ram specified in GB.

Why is output_meta missing from my inference.sh app response?▼

The output class is likely extending BaseModel instead of BaseAppOutput. Output classes that include output_meta must extend BaseAppOutput, otherwise the metadata is silently dropped from the response.

Can inference.sh apps expose multiple functions?▼

Yes. Add public methods with type-hinted Pydantic models in Python, or export matching PascalName Input/Output Zod schemas in Node.js. Call a specific function by passing "function": "method_name" in the API request body.

Why does torch.cuda.is_available() fail in inference.sh containers?▼

It does not reliably detect GPUs in grid containers. Use accelerate's Accelerator().device for device detection, and use device_map for large models over 1B parameters to stream weights directly to GPU.

How do I access secrets and OAuth credentials in an inference.sh app?▼

Declare secrets and integrations in inf.yml, then read them via os.environ in Python or process.env in Node.js during setup(). OAuth credentials like Google service accounts arrive as JSON environment variables.

Related Skills

Explore More Software Engineering Skills→
drpedapatidrpedapati
1

Claude Code API

Wrap the Claude Code CLI as an HTTP API for chat and queries.

Community
Advanced
crazymsncrazymsn
22

modal

Run Python AI/ML workloads serverlessly on cloud GPUs with Modal.

Community
Advanced
claudcherejiclaudchereji

llm-architect

Designs, deploys, and optimizes LLM architectures for production environments.

Community
Advanced
zainhaszainhas
2

together-dedicated-containers

Deploy Dockerized inference workloads on Together AI's managed GPU infrastructure.

Community
Advanced