context-scrapers

Use Scrapy spiders to extract meetings and codes from Legistar and Municode. This Skill provides pre-built spiders, configurable pipelines for data storage, and Scrapy settings tailored to municipal sources, handling jurisdiction-specific variations and ensuring consistent ingestion.

Yes. Modify spider configuration and settings to target additional jurisdictions. The Skill includes verification scripts—verify_raw_scrapes.py and verify_municode_discovery.py—to validate that new jurisdictions ingest correctly and discover data reliably.

Run the included verification scripts to validate raw scrapes and discovery logic before full ingestion. These tools check data integrity, identify pipeline failures, and confirm that Legistar meetings and Municode codes are captured and stored consistently.

Web scraping against government sources requires validation. This Skill provides pipelines and verification scripts that check raw data consistency, detect schema drift, and confirm discovery logic across Legistar and Municode to catch ingestion issues early.

Yes. Scrapy settings control spider behavior, rate limiting, and storage paths per jurisdiction. This Skill bundles jurisdiction-specific configurations and pipelines, so you adjust settings rather than rewriting spiders when adding new cities or councils.

Legistar spiders extract meeting records, agendas, and votes; Municode spiders extract municipal code sections and ordinances. Both require different selectors and data schemas. This Skill provides separate spiders and pipelines tuned to each source's structure and update frequency.