scrape-and-load

You can scrape Wookieepedia data and load it into SQLite by running an end-to-end pipeline that extracts the data, converts it to a CDS-ready format, and persists it locally. The pipeline coordinates scraping, building CDS artifacts, and loading fixture data for testing.

A CDS-ready format is a transformed data structure compatible with CAP's Core Data Services. The scraping pipeline converts raw Wookieepedia data into this format to enable building CDS artifacts, running migration tests, and validating schema compatibility in SQLite.

Yes, scraped Star Wars data can be used for migration tests in SQLite. The pipeline loads fixture data from Wookieepedia into a local SQLite datastore and runs a full test suite to ensure data integrity and compatibility with the target schema.

The scraping pipeline implements a cache-first strategy to avoid redundant requests and stops on failure to prevent cascading errors. It reports summary metrics for each step, including scraping, building CDS artifacts, loading fixtures, and running tests.

No external dependencies are required to run the Wookieepedia scraping pipeline. The skill operates independently to scrape data, convert it to a CDS-ready format, and load it into SQLite for testing and migration validation.

The best way to validate scraped data against a target schema is to run a pipeline that builds CDS artifacts from the data and executes migration tests. This ensures data integrity and compatibility by loading fixtures into SQLite and running a full test suite.