weighted_sampler

Perform weighted random sampling on JSONL files with configurable weight fields.

541|171|Updated May 3, 2018
One-click install
npx skills add https://github.com/cas-bigdatalab/piflow --skill weighted-sampler
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: weighted_sampler
Source: https://github.com/cas-bigdatalab/piflow/tree/main/workspace/skills/weighted_sampler
Command: npx skills add https://github.com/cas-bigdatalab/piflow --skill weighted-sampler

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill addresses the need for precise weighted sampling from JSONL files, offering a solution for when users require a subset of records with higher probability based on their weights.

Core Features & Use Cases

  • Weighted Sampling: Perform random sampling on JSONL records with customizable weights.
  • Sampling Modes: Supports both with-replacement and without-replacement sampling.
  • Use Case: Ideal for scenarios where you need to prioritize certain records based on their importance or priority score.

Quick Start

Utilize the weighted_sampler skill to randomly sample 1000 records from 'data.jsonl' with 'priority_score' as the weight field and save the results to 'sampled.jsonl'.

Frequently Asked Questions about weighted_sampler

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I perform weighted random sampling on a JSONL file?

Perform weighted random sampling on a JSONL file by specifying a numeric weight field and the desired sample size to extract prioritized records. The process utilizes Python's random library to read your JSONL data and output a subset based on your custom weights.

Can I sample JSONL records with or without replacement?

Yes, you can sample JSONL records with both with-replacement and without-replacement modes. This allows you to either allow the same record to be selected multiple times or ensure each record appears only once in your sampled output.

What is the best way to prioritize specific data records during sampling?

The best way to prioritize specific data records during sampling is to use a numeric weight field to define their importance. This weighted sampling approach ensures that records with higher priority scores have a greater probability of being selected in the final subset.

Does weighted sampling work for data prioritization and importance ranking tasks?

Yes, weighted sampling works effectively for data prioritization and importance ranking tasks. By assigning higher numeric weights to critical JSONL records, the sampling mechanism naturally prioritizes them, yielding a dataset that reflects your defined importance scores.

How do I specify the sample size when extracting records from a JSONL dataset?

You specify the sample size as a numeric parameter when initiating the weighted sampling process on your JSONL dataset. This defines the exact number of records to be extracted and saved to your output file based on the specified weights.