What problem does it solve?
When a data pipeline finishes, its result files often need to be persisted into a shared Dataspace storage space. Doing this manually is error-prone and breaks DAG automation. This Skill acts as the terminal node of a DAG, taking the upstream operator's file output and uploading it into a specified Dataspace directory automatically.
Core Features & Use Cases
- Terminal DAG Sink: Designed to be placed at the end of a pipeline, receiving a single file output from an upstream operator.
- Managed Staging & Upload: Copies the input file into a local managed directory, then uploads the whole directory to the target Dataspace datasource via its instance ID.
- Overwrite Control & Local Output: Optional overwrite flag protects existing files, and a local output reference is retained for downstream result tracking.
- Use Case: A research data pipeline produces a cleaned CSV; this operator uploads it to the team's Dataspace space under a given relative directory, keeping the original filename.
Quick Start
Ask the agent to run the dataspace_file_sink_stop operator with the upstream file output, your Dataspace datasource ID, and the target relative directory to upload the result file at the end of the pipeline.
