llm-provider-setup

Configure LLM provider connections, model profiles, and credentials via CLI with live verification.

1.2k|166|Updated Feb 7, 2026
One-click install
npx skills add https://github.com/vellum-ai/vellum-assistant --skill llm-provider-setup
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: llm-provider-setup
Source: https://github.com/vellum-ai/vellum-assistant/tree/main/skills/llm-provider-setup
Command: npx skills add https://github.com/vellum-ai/vellum-assistant --skill llm-provider-setup

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Adding a new LLM provider or model to a Vellum assistant involves credentials, connections, model id discovery, and profile activation — a chain where one wrong step (a guessed model id, a leaked API key, an unverified profile) breaks the chat model. This Skill provides the canonical, strictly sequential procedure that validates each step before the next.

Core Features & Use Cases

  • Managed-first routing: Checks whether the platform-managed Vellum route already serves the requested model, avoiding unnecessary API key collection entirely.
  • Secure credential collection: Prompts for API keys through a masked inline prompt stored in a vault, so secrets never enter the chat conversation.
  • Model discovery and verification: Lists valid model ids from the provider catalog, probes unlisted models with live calls, and mandates a real inference call before any profile is activated.
  • Use Case: A user wants to add their own Anthropic key and use Claude Opus as the chat model. The Skill reuses or collects the key securely, creates the connection, discovers the exact model id, creates a labeled profile, verifies it with a live call, and only then activates it.

Quick Start

Set up a new LLM provider connection for OpenAI with my API key, create a profile for GPT-5, verify it with a live call, and make it my active chat model.

Frequently Asked Questions about llm-provider-setup

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new LLM provider to a Vellum assistant?

Run the sequential CLI workflow: check managed availability with assistant inference providers list, collect or reuse a credential, create the connection, discover a valid model id, create the profile, then verify with a live call before activating it.

How do I securely store an API key without pasting it in chat?

Use assistant credentials prompt with the service and field flags, which renders a masked input inline and stores the key in a vault. The assistant only ever references the vault path, so the secret never enters the conversation.

Can I use a local or self-hosted model endpoint like LM Studio or vLLM?

Yes, create a connection with --provider openai-compatible, supply --base-url and at least one --model id. For keyless local endpoints use --auth none and omit the credential flag.

Why does profile creation fail with a model id error?

The model id was likely guessed rather than taken from the provider catalog. Run assistant inference models list --provider <provider> to get valid ids, or probe an unlisted model with a live send call and pass --allow-unlisted.

When should I use the managed Vellum route instead of my own API key?

Use the managed route whenever you are signed in to Vellum and the requested model is served by it. It requires no credential or connection, so you skip key collection entirely and create the profile with --provider vellum.