model-bank-metadata

Backfill knowledgeCutoff, family, and generation fields on model-bank TypeScript model cards.

74|11|Updated Jul 4, 2024
One-click install
npx skills add https://github.com/OpenSourceAGI/qwksearch-research-agent --skill model-bank-metadata-opensourceagi
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: model-bank-metadata
Source: https://github.com/OpenSourceAGI/qwksearch-research-agent/tree/main/apps/qwk-in-lobe/.agents/skills/model-bank-metadata
Command: npx skills add https://github.com/OpenSourceAGI/qwksearch-research-agent --skill model-bank-metadata-opensourceagi

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Maintaining accurate metadata (knowledgeCutoff, family, generation) across roughly 1900 AI model entries in 80 provider files is error-prone: vendors publish cutoffs inconsistently, third-party aggregators copy wrong values, and naming conventions hide family/version info. This Skill provides sourcing rules, derivation regexes, and idempotent codemods to populate these fields correctly at single-model or repo-wide scale. ## Core Features & Use Cases - Authoritative sourcing rules: Defines which sources are acceptable (vendor docs, official Hugging Face cards, tech reports) and which to reject (aggregator sites, AWS Bedrock cards, release-date inference), with per-vendor footguns documented. - Rule-based family/generation derivation: scripts/derive-family.ts maps normalized model ids to family and generation slugs, handling traps like date suffixes, size suffixes, and vendor spelling variants. - Idempotent codemods: scripts/apply-cutoffs.ts and scripts/apply-family.ts insert fields into prettier-formatted model entries without touching existing values, plus scripts/extract-model-ids.ts to enumerate chat-model ids. - Use Case: When a new Claude model PR lands, fill all three metadata fields inline citing the official system card; or run a repo-wide sweep with multi-agent research and adversarial verification to backfill cutoffs across all providers. ## Quick Start Ask the agent to backfill knowledgeCutoff and family metadata for the aiModels provider files using the model-bank-metadata skill's extract, derive, and apply scripts.

Frequently Asked Questions about model-bank-metadata

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I backfill knowledgeCutoff for AI models in a model database?

Extract chat-model ids with extract-model-ids.ts, research official vendor sources for each cutoff, then run apply-cutoffs.ts with the resulting JSON map. The codemod inserts knowledgeCutoff after the id line of matching chat entries and skips entries that already have the field.

What sources are acceptable for LLM knowledge cutoff dates?

Accept only vendor official docs, official Hugging Face org model cards, and official tech reports or launch posts. Reject third-party aggregator sites, AWS Bedrock model cards as sole source, and any inference from release dates, since these have proven unreliable.

How do I derive model family and generation from model ids?

Run derive-family.ts, which applies per-vendor regex rules to normalized ids and outputs family and generation slugs. It handles traps like date suffixes (claude-sonnet-4-20250514 is claude-4), size suffixes (llama-3-8b is llama-3), and spelling variants like qwen2p5.

Do distilled models inherit the teacher model's knowledge cutoff?

No, distills do not inherit cutoffs from their teacher or base model. Use the distill's own published cutoff value or leave the field empty, since an empty field is correct when no authoritative source exists.

Why did the apply codemod skip some model entries?

The codemods match ids verbatim after normalization, so invisible characters like trailing tabs in aggregator data can prevent matches. They also skip non-chat model types and entries that already contain the target field.

Do I need a database migration when adding metadata fields to model cards?

No migration is needed. Builtin models are merged from model-bank at read time, so new card fields like knowledgeCutoff, family, and generation flow to the client automatically once added to the data files.