eval-model-comparison

Benchmark OCR and LLM transcription accuracy by configuring a model matrix with multiple local LLMs, varied context windows, and decoding strategies. The evaluation produces per-field confusion summaries and a standardized comparison report against your gold references.

LLM form-filling accuracy evaluation measures how well models populate structured fields from source text. Hallucination checks identify fabricated data not present in the source, using field-level metrics to compare model outputs against typed gold references.

Yes, you can compare multiple local LLMs in one benchmark. The skill requires configuring a model matrix that includes varied context windows and decoding strategies, then produces a model-comparison report summarizing per-field performance across all configurations.

Model comparison reports for medical notes transcription are generated by building a benchmark with typed notes as gold references. The skill evaluates OCR, transcription, and LLM form-filling pipelines, producing per-field confusion summaries to inform model selection.

The best way to build a test harness for OCR and transcription model selection is configuring a model matrix with multiple local LLMs and varied decoding strategies. The skill evaluates field-level accuracy and hallucination rates to produce a standardized comparison report.

The model comparison approach works without external dependencies, as the skill requires no dependencies listed in its configuration. It supports benchmarking multiple local LLMs for OCR, transcription, and form-filling accuracy evaluation entirely.