ingest-source
CommunityIngest a new data source for investigative platforms.
Software Engineering#data ingestion#data schema#api endpoint#investigative tools#source characterization
Authortcole333
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Streamline the process of onboarding new data sources into investigative platforms, enhancing data integration capabilities.
Core Features & Use Cases
- Source Characterization: Automate characterization of data sources (format, size, content, provenance, overlap).
- Data Download & Access: Handle direct downloads and API data retrieval with endpoint discovery and documentation.
- Schema Analysis: Analyze and output data schema details like columns, data types, and row count.
- Query Wrapper Creation: Facilitate creation of query wrappers for custom data source queries.
- Initial Investigation: Perform initial searches to identify relevant data from new sources.
- Lead Generation: Generate leads based on significant findings in new data sources.
- Source Reporting: Update source reporting tools with new data sources.
- CLAUDE.md Integration: Include new data source information in the platform's inventory guide.
- Ingestion Logging: Document data ingestion details for future reference.
Quick Start
Use the ingest-source skill to onboard the new data source 'tensonaut/EPSTEIN_FILES_20K'.
Dependency Matrix
Required Modules
datasetsrequestspandassqlite3
Components
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: ingest-source Download link: https://github.com/tcole333/ithildin/archive/main.zip#ingest-source Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.