voice-assistant-pipeline

Builds voice AI assistants with wake word, VAD, STT, LLM, and TTS pipeline stages.

Updated Aug 21, 2026
One-click install
npx skills add https://github.com/TylerSimons1127/vibe --skill voice-assistant-pipeline-tylersimons1127
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: voice-assistant-pipeline
Source: https://github.com/TylerSimons1127/vibe/tree/main/skills/software-development/voice-assistant-pipeline
Command: npx skills add https://github.com/TylerSimons1127/vibe --skill voice-assistant-pipeline-tylersimons1127

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires numpy, sounddevice, openwakeword, faster-whisper, kokoro-onnx, piper-tts, scipy, requests, onnxruntime, pycaw, sherpa-onnx, huggingface_hub, and includes scripts (resource) and references (resource) components.

What problem does it solve? Building a JARVIS-style voice assistant requires wiring together wake word detection, voice activity detection, speech-to-text, LLM reasoning, and text-to-speech — each with its own latency pitfalls, echo problems, and microphone quirks. This Skill provides a verified architecture and working code patterns for assembling that full audio pipeline, including hard-won fixes for common failures like echo self-retriggering and silent microphones. ## Core Features & Use Cases - Full Pipeline Architecture: Wake word (openWakeWord) → VAD (Silero or RMS energy gate) → STT (faster-whisper) → LLM (OpenRouter or Hermes CLI) → TTS (Kokoro or Piper) → playback, with a defined state machine (SLEEP, LISTENING, PROCESSING, SPEAKING, FOLLOW_UP). - Engine Selection Guidance: 2026 comparisons of STT engines (faster-whisper, Nemotron Streaming, Parakeet TDT) and TTS engines (Kokoro, Piper, Chatterbox, Step Audio EditX) with latency, license, and quality trade-offs. - Diagnostics & Pitfall Fixes: A mic self-test script for the "never hears me" problem, echo cancellation via mic draining, persistent output streams to avoid PortAudio open/close latency, and Windows .bat/PYTHONPATH gotchas. - Use Case: Ask the agent to build a single-file jarvis.py that listens for "Hey Jarvis", transcribes your command with Whisper tiny, gets a reply from an LLM, and speaks it aloud through a British male voice — all running locally on CPU. ## Quick Start Ask the agent to build a minimal single-file voice assistant that wakes on "Hey Jarvis", transcribes speech with faster-whisper, answers via an LLM, and speaks the reply with Kokoro TTS.

Frequently Asked Questions about voice-assistant-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a voice assistant with wake word, STT, and TTS in Python?▼

Chain openWakeWord for wake detection, faster-whisper for speech-to-text, an LLM call via OpenRouter or the Hermes CLI, and Kokoro for text-to-speech. A single-file script with config constants, loader functions, and a main loop is the recommended minimal shape.

What is the best STT engine for a low-latency voice assistant?▼

faster-whisper tiny is the simplest option at roughly 0.3 seconds per utterance on CPU. For true streaming with 80-560ms chunk latency and no silence hallucination, use NVIDIA Nemotron Streaming EN 0.6B via sherpa-onnx.

Kokoro vs Piper TTS for a JARVIS voice assistant?▼

Kokoro 82M offers the bm_george British male voice and runs on CPU under 4GB RAM with Apache 2.0 licensing. Piper provides a free J.A.R.V.I.S.-style voice model on HuggingFace and also runs CPU-only, but uses a different synthesis API returning AudioChunk iterables.

Why does my voice assistant never hear me or transcribe silence?▼

The most common cause is the microphone not delivering audio to the OS, not a code bug. Run a per-device record test measuring signal std while speaking; the working device is often not the Windows default input.

How do I stop my voice assistant from re-triggering on its own TTS output?▼

Drain the microphone queue after every playback so the assistant's own voice cannot echo back into the wake word detector. A short sleep followed by a second drain catches trailing audio callback chunks.

Can a voice assistant run fully offline on CPU without a GPU?▼

Yes. openWakeWord, Silero VAD, faster-whisper tiny, and Kokoro or Piper TTS all run on CPU. The LLM stage needs either a cloud API like OpenRouter or a local Hermes CLI profile as the reasoning brain.