weaviate-data-ingestion

Batch uploading to Weaviate ingests thousands of objects from JSON, CSV, or Python lists with automatic progress tracking and error handling. The Skill processes multiple data sources simultaneously, vectorizes content, and inserts them into your local Weaviate collection efficiently.

Intelligent chunking automatically processes PDFs and other documents into semantically meaningful units, extracting metadata like page numbers and topics. The Skill then uploads these chunks directly to Weaviate, enabling RAG-ready knowledge bases without manual segmentation.

Data ingestion supports JSON, CSV, and text files, plus base64-encoded images for multi-modal collections. All formats undergo automatic vectorization and are uploaded with robust error handling and progress tracking.

Yes, data ingestion requires a locally running Weaviate instance, typically via Docker. Prerequisites include weaviate-local-setup, weaviate-connection, and weaviate-collection-manager configured before uploading data.

Multi-modal support enables uploading base64-encoded images into collections configured for visual search. Text and images ingest together with automatic vectorization, enabling rich content indexing in a single workflow.

Robust error handling catches failures during batch uploads and reports them with progress tracking. The Skill logs errors without halting the entire upload, allowing partial ingestion to complete while identifying problematic objects.