cef-inference

What problem does it solve?

Enable inline ML model inference inside CEF agent handlers, allowing models to run directly in the runtime without external API calls.

Core Features & Use Cases

• Inline inference via context.models.<alias>.infer(...) for non-streaming results and .stream(...) for streaming outputs.
• Supports the full 16-model registry (yolo, yoloXL, whisper, whisperTiny, whisperLarge, llm, llamaVision, qwenVision, qwenCoder, mistral7b, mistralSmall, embedding, emotionClassifier, sentimentAnalysis, plateDetector, plateOcr) with input/output schemas and practical handler examples.
• Streaming consumption enables per-chunk results and real-time feedback within agent workflows.
• Useful in building end-to-end agent pipelines: vision, ASR, NLP, and code-generation inside a single handler.

Quick Start

Create a CEF agent handler and call context.models.<alias>.infer or context.models.<alias>.stream to perform in-model inference.