skills.rest
oci-ai-incubations
oci-ai-incubationsOFFICIAL
Explore all skills by this author →
WebsiteGitHub

vss-benchmark

Benchmark VSS video summarization performance with VLM/LLM model selection and pack auto-selection.

3|2|Updated Sep 25, 2025
Search Tags:#performance#llm#benchmarking#deployments#vlm#video-summarization
One-click install
npx skills add https://github.com/oci-ai-incubations/ai-accelerator-starter-packs --skill vss-benchmark
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: vss-benchmark
Source: https://github.com/oci-ai-incubations/ai-accelerator-starter-packs/tree/main/.claude/skills/vss-benchmark
Command: npx skills add https://github.com/oci-ai-incubations/ai-accelerator-starter-packs --skill vss-benchmark

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill enables objective benchmarking of VSS video summarization performance by measuring how different VLM/LLM configurations perform on a deployed VSS stack, including pack selection, timing, and accuracy metrics.

Core Features & Use Cases

  • Automated pack auto-selection based on chosen VLM/LLM models and available capabilities.
  • End-to-end benchmarking workflow: configuration gathering, run orchestration, metric collection, and reporting.
  • Optional ground-truth evaluation to quantify summarization accuracy against a provided.csv of reference timelines.

Quick Start

Provide VLM model, LLM model, frontend URL, video source, and number of runs to initiate the benchmark.

Frequently Asked Questions about vss-benchmark

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark video summarization performance for different LLM and VLM configurations?▼

To benchmark video summarization performance, provide a VLM model, LLM model, frontend URL, video source, and number of runs. The skill orchestrates the benchmarking workflow across deployed VSS stacks and collects timing, accuracy, and resource usage metrics.

How does automated pack selection work for VSS benchmarking?▼

Automated pack selection evaluates your chosen VLM and LLM models against available capabilities, automatically assigning the appropriate POC, small, or medium pack for the benchmark run without requiring manual configuration.

Can I evaluate summarization accuracy against ground truth during a VSS benchmark?▼

Yes, you can optionally provide a CSV file of reference timelines to quantify summarization accuracy. The benchmark evaluates the generated summaries against this ground truth to produce accuracy metrics.

What metrics are collected when running CI/CD style evaluation campaigns for VLM deployments?▼

CI/CD style evaluation campaigns collect per-run metrics including timing, accuracy, and resource usage. The reporting covers configuration gathering, run orchestration, and results across the evaluated VLM and LLM model packs.

What information do I need to provide to start a VSS benchmark run?▼

To initiate a VSS benchmark run, you must supply the VLM model name, LLM model name, frontend URL, video source, and the desired number of runs for the evaluation campaign.

Related Skills

Explore More Data & Analytics Skills→
sayalinvidiasayalinvidia

vss-generate-video-report

Generate structured video analysis reports from VSS clips and incidents.

Community
Advanced
andisabandisab

gitops-argocd

Generate Argo CD Application and ApplicationSet manifests for multi-environment Kubernetes deployment.

Community
Advanced
oci-ai-incubationsoci-ai-incubations
3

vss-test-coverage

Codify end-to-end test flows for API, UI, and infrastructure in the VSS stack.

Official
Advanced
zhizhunbaozhizhunbao

resource-discovery

Scan online sources and benchmark AI resources to recommend tools and frameworks.

Community
Advanced