CrucibleDatasets.Loader.NoRobots (CrucibleDatasets v0.5.4)

View Source

NoRobots instruction-following dataset loader.

NoRobots contains human-written instruction-response pairs for training instruction-following models. The dataset includes high-quality examples across various categories like coding, summarization, creative writing, and more.

Source: https://huggingface.co/datasets/HuggingFaceH4/no_robots

Categories

The dataset covers these instruction categories:

  • Open QA: General question answering
  • Generation: Text generation tasks
  • Brainstorm: Creative ideation
  • Rewrite: Text transformation
  • Summarize: Summarization tasks
  • Classify: Classification tasks
  • Closed QA: Factual question answering
  • Extract: Information extraction
  • Chat: Conversational responses

Examples

# Load with defaults
{:ok, dataset} = NoRobots.load()

# Load with sample size
{:ok, dataset} = NoRobots.load(sample_size: 100)

# Load specific split
{:ok, dataset} = NoRobots.load(split: :train)

Summary

Functions

load(opts \\ [])

Load NoRobots dataset.

parse_jsonl(content)

Parse NoRobots JSONL format (if loading from file).

Functions

load(opts \\ [])

@spec load(keyword()) :: {:ok, CrucibleDatasets.Dataset.t()}

Load NoRobots dataset.

For demo purposes, generates synthetic data. In production, would fetch from HuggingFace.

Options

  • :split - Dataset split (:train, :test) default: :train
  • :sample_size - Limit items (default: 50)
  • :seed - Random seed for reproducibility

parse_jsonl(content)

@spec parse_jsonl(String.t()) :: [map()]

Parse NoRobots JSONL format (if loading from file).

Expected format: {"prompt": "...", "completion": "...", "category": "..."}