Logs human feedback assessments on MLflow traces for agent quality review.
npx skills add https://github.com/FMurray/mlfts --skill feedback-fmurray
Please help me install this Agent Skill. Skill: feedback Source: https://github.com/FMurray/mlfts/tree/main/.claude/skills/feedback-trace Command: npx skills add https://github.com/FMurray/mlfts --skill feedback-fmurray
quality=good, thumbs_up=true) with optional rationale to any trace, stored as assessments with source_type='HUMAN'. - List Recent Traces: Display recent traces with ID, timestamp, status, and input preview to find the trace you want to review. - Searchable Tagging: Automatically sets a has_feedback=true tag so annotated traces can be filtered in MLflow searches. - Use Case: After a coding session, run feedback with quality=poor and a rationale like "hallucinated the API endpoint" to flag the trace for your evaluation dataset. ## Quick Start Ask the agent to log thumbs up feedback on the most recent MLflow trace with a short rationale.High-intent search queries and answers about installing and using this skill.
Run the trace_annotate.py script with the feedback subcommand, passing --name and --value, for example quality=good. Without a --trace-id it targets the most recent trace, and you can add --rationale for a free-text explanation.
Run the list subcommand of trace_annotate.py to show recent traces with their IDs, timestamps, status, and input previews. Use --max-results to control how many traces are displayed, then pass the chosen ID via --trace-id.
Feedback is stored as assessments on the trace's info.assessments array with source_type HUMAN and source_id claude_code_user. A has_feedback=true tag is also set so you can search traces with a filter string on that tag.
Yes, multiple assessments can be logged on one trace, such as both quality and relevance ratings. Each feedback call appends a new assessment rather than overwriting existing ones.
The error occurs when the current MLflow experiment contains no traces, excluding env_snapshot companion traces. Verify your MLFLOW_TRACKING_URI and experiment name are configured and that at least one session has been traced.
mlflowDisplays the current MLflow tracing configuration for Claude Code.
JingyiWngEvaluate MLflow GenAI agents with built-in and custom metrics.
whd4Capture git state, decisions, and remaining work for session resumption.
yanochka11Calibrate LLM judges against human labels using TPR/TNR analysis.