scoring-engine

Scores job listings against a candidate profile using embedding filters and LLM analysis.

Updated May 19, 2026
One-click install
npx skills add https://github.com/Jessitoii/career-os --skill scoring-engine-jessitoii
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: scoring-engine
Source: https://github.com/Jessitoii/career-os/tree/main/.agents/skills/scoring-engine
Command: npx skills add https://github.com/Jessitoii/career-os --skill scoring-engine-jessitoii

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires sentence-transformers, scikit-learn, pydantic, httpx.

What problem does it solve? Manually reviewing every scraped job listing is slow and inconsistent. This Skill implements a two-stage pipeline that filters irrelevant jobs locally with embeddings and then applies LLM-based deep scoring, so only high-match jobs reach the application or approval stage. ## Core Features & Use Cases - Two-Stage Filtering Pipeline: Stage 1 uses local Ollama embeddings (nomic-embed-text) with cosine similarity to discard low-relevance jobs; Stage 2 sends survivors to Groq Llama-3-70b for structured JSON scoring. - Structured Output Validation: A Pydantic schema enforces score (0-100), reasoning list, critical flags (visa sponsorship, language requirements, seniority), and a decision of auto_apply, ask_user, or reject. - Confidence Threshold Routing: Scores of 85+ route to AUTO_APPLY, 60-85 to AWAITING_APPROVAL via Telegram, and below 60 to DISCARDED, with results persisted to the job_listings table. - Use Case: After scraping 200 job postings, run the pipeline to discard 80% locally at zero API cost, then LLM-score the top 20% and automatically queue only the strongest matches for application. ## Quick Start Score the newly scraped job listings against my profile and route them by confidence threshold.

Frequently Asked Questions about scoring-engine

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I filter job listings with embedding similarity before LLM scoring?

Generate embeddings for the candidate profile and each job description using a local Ollama endpoint with nomic-embed-text, then compute cosine similarity. Listings below the threshold (default 0.65) are discarded, and only the top 10-20% proceed to LLM scoring.

How to get structured JSON output from an LLM scoring prompt?

Use a system prompt that instructs the model to return only JSON with score, reasoning, flags, and decision fields, then validate the response with a Pydantic model. Invalid JSON or schema violations raise an error instead of silently passing bad data.

What cosine similarity threshold should I use for job matching?

Start with 0.65 and tune based on funnel metrics. Raise the threshold if interview rates are low and too many weak matches pass; lower it if good jobs are being rejected, and check profile summary quality first.

Why are good jobs being filtered out by the embedding stage?

Over-filtering usually comes from a threshold set too high or a weak profile summary that poorly represents the candidate's skills. Lower EMBEDDING_THRESHOLD and improve the profile text used for embedding before changing the LLM stage.

When should I not use this scoring pipeline?

Do not use it for duplicate detection, which belongs to a separate duplicate-prevention concern, or for modifying how scores drive application state transitions, which belongs to session lifecycle logic. It only handles relevance scoring and threshold routing.