eval-pe-output

To check AI-generated PE documents for hallucinations and citation accuracy, you can evaluate the memo against a structured source-of-truth. This verifies that financial figures trace back to source fields and confirms named entities actually exist in the source data.

Source-of-truth validation for private equity document evaluation is a deterministic process that measures AI output accuracy without LLM calls. It checks citation accuracy, hallucination rate, coverage, and consistency against structured JSON data to ensure reproducibility and defensibility.

To evaluate PE memo consistency using JSON sidecars and OpportunityMap files, provide the memo JSON path and source JSON path to the evaluation script. It verifies that memos derived from the same source agree on the headline number and measures source coverage.

You do not need an LLM to grade AI output for citation accuracy and consistency. This evaluation process uses deterministic scripts to compare documents against structured JSON sources, ensuring reproducible and defensible results without relying on language models.

You can use deterministic scripts to measure coverage in private equity memos. The evaluation checks the fraction of source opportunities addressed in the document by comparing the memo JSON directly against the OpportunityMap JSON from the diagnostic tool.

A limitation of using deterministic evaluation for AI-generated financial documents is the strict dependency on specific JSON sidecars and OpportunityMap files from the diagnostic tool. It cannot evaluate unstructured text or sources lacking this structured metadata.