Generate probing questions to stress-test answers across six evaluation angles.
npx skills add https://github.com/JGalego/Bag-of-Tricks --skill grill-jgalego
Please help me install this Agent Skill. Skill: grill Source: https://github.com/JGalego/Bag-of-Tricks/tree/main/grill Command: npx skills add https://github.com/JGalego/Bag-of-Tricks --skill grill-jgalego
This Skill scrutinizes answers for hidden assumptions, potential errors, and lack of support, helping you ensure the reliability and accuracy of information.
Analyze an answer with grill. Run the command 'python3 grill.py answer.txt'.
High-intent search queries and answers about installing and using this skill.
To stress-test LLM outputs for hidden assumptions, you can use an adversarial questioning framework to interrogate answers across six angles: assumptions, edge cases, contradictions, sources, overconfidence, and falsifiability. This process identifies unsupported claims and potential errors.
Cross-examining a draft answer for information reliability involves generating probing questions that challenge weak points and evaluating the response under scrutiny. This ensures the answer is well-supported and highlights areas of overconfidence or lack of falsifiability.
Yes, you can use this adversarial questioning framework with Anthropic, OpenAI, and Google generative AI models. The tool requires a compatible LLM provider and SDK to optionally run the generated questions against a model for analysis.
To analyze an answer file for contradictions and edge cases from the command line, run the Python script with your text file as an argument. The command 'python3 grill.py answer.txt' initiates the cross-examination process on the provided text.
You need to apply falsifiability checks to technical or business decision answers when information reliability is critical. This scrutiny identifies hidden assumptions, potential errors, and lack of support, ensuring your decisions are well-supported and robust against adversarial interrogation.
PostHogWrap LLM SDKs with PostHog instrumentation to emit generation and trace events.
olivierlesnickiForce re-evaluation of evidence and alternative interpretations before asserting claims.
nikhilvallishayeeDetect edge cases and challenge assumptions through three verification questions.
hhhi21gImplement runtime safety checks for LLM applications.
