Data your model hasn't seen
and can't game

Expert-built, verified by code, and proven to transfer.
License our off-the-shelf data for eval, SFT or RL.

expert-verified and eval task

hours of physical AI data

STEM, Coding, Knowledge Work,
Enterprise Tool-Use data

STEM, Coding, Knowledge Work, Enterprise Tool-Use data

Trusted by Leading AI Teams

Proven to move models

One epoch of RL on our enterprise data lifts Qwen3.5-27B on agent benchmarks it never trained on.

Overall

Qwen3.5-27B base

After 72 GRPO steps

+3.4pp

partial credit

41.6%

AutomationBench

+8.9pp

pass@1

86.8%

τ³ Retail

+4.3pp

pass@1

46.0%

Toolathlon

Where gains are largest

Gains land where our environments match the work. Operations and customer-support environments make up most of the training data.

AutomationBench

+9.6pp

partial credit

39.6%

Operations

+7.6pp

partial credit

40.1%

Support

What the model learned

Consults the knowledge base before its first write.

It checks policy before acting, instead of acting first.

49.6%

81.5%

of held-out tasks

Qwen3.5-27B base vs. the same model after 72 GRPO steps on 1,540 enterprise tasks. Identical serving for both; paired 95% confidence intervals don’t fall below zero. Evals Sep 25 – Oct 5, 2026.

Off-the-shelf datasets, ready to license

Private, verified data with deterministic rewards, ready for eval, SFT or RL.

Terminal-Bench 4

Available

Private terminal tasks for agents, spanning coding, STEM, hardware and operations. Terminal-Bench 4.0 compliant, with a multi-container setup that keeps the agent isolated from a held-out verifier.

Volume

1,500 tasks

Domains

Coding, STEM, hardware, operations

Difficulty

Calibrated against Sol6.1 High. Between 1 to 5 passes every 8 rollouts

Related
benchmarks

Terminal-Bench 4.0

Enterprise Tool Use

Available

RL environments where the agent works as company staff across 11 enterprise domains, using 19–39 integrated tools such as CRM, ERP and ticketing. It has to find the business rules itself in a knowledge base, and the right answer is often to refuse or escalate.

Volume

2,500+ tasks

Domains

11 enterprise domains, such as HR service desk, brokerage support, and operations

Difficulty

Frontier models score 38% or lower (pass^2)

Related
benchmarks

τ², MCP-Atlas, BFCL

Frontier STEM

Available

PhD-authored problems across 10+ domains, weighted toward math and physics. Every answer is recomputed by standalone verification code. Each problem ships with a step-by-step derivation.

Volume

2,000 problems

Domains

Physics, maths, computer science, ML, biology, chemistry, electrical, civil, and mechanical engineering, data science, statistics, financial mathematics

Difficulty

Calibrated so GPT-5.4 (high) solves each problem in only 10–75% of 8 runs

Related
benchmarks

HLE, GPQA, AIME, AMO-Bench

SciCode

Available

Expert-authored multi-step scientific coding tasks. Each splits into 2–5 sub-steps with ground-truth Python and assertion-based tests, in SciCode-harness-compatible JSON.

Volume

2,000 tasks

Domains

Physics, materials science, biology, maths, chemistry, computer science, engineering, data science, ML, statistics, biochemistry, optics

Difficulty

Every task ships with an 8-attempt pass record. Frontier slice vs. GPT-5.4 (high): all tasks solved 1–5 of 8, median 3 of 8

Related
benchmarks

SciCode

Enterprise Workflows

Available

The agent works business requests end to end inside a simulated company: 20+ apps rebuilt from real vendor APIs, plus 1,000+ documents, tickets, emails and chats. The fact that decides each case sits in a different place than the request points to.

Volume

500

Domains

Enterprise IT operations (5 business functions on the roadmap)

Difficulty

Opus 5.5 and GPT-6.1: ~40% on one try, 10–20% on all four tries

Related
benchmarks

τ³, AutomationBench, Toolathlon

 Terminal-Bench-Science

Available

Private terminal tasks built to Terminal-Bench Science's format and taxonomy. The agent works a lab, clinic or engineering team's records to produce a certified result to the field's standard, and must find the faults hidden in the data that routine checks miss. Graded on the final output by a separate, offline verifier.

Volume

200 tasks

Domains

Life sciences

Physical sciences

Earth sciences

Mathematical sciences

Difficulty

Calibrated against Meta's Muse Spark 1.3 Max: every task is solved in at most 1 of 4 rollouts

Related
benchmarks

Terminal-Bench Science, Terminal-Bench 4.0

Real Office Knowledge Work

Available

Agents step into a live client engagement, a project world of 400+ files, and produce the next deliverable: a recalculating workbook, a client-ready deck, a signed memo (.xlsx, .pptx, .docx). Each task is 4–8 hours of analyst work, and every task ships with an expert deliverable, reviewed and signed off by a domain expert, and scored on the same grader.

Volume

200+ tasks

Domains

Strategy, finance, pricing, operations and legal work across energy, real estate and banking

Difficulty

Expert deliverables score 0.99; gpt-6-astra averages 0.46 on the same grader

Related
benchmarks

GDPval, APEX-Agents

Agentic Coding

Available

Never-published terminal tasks built from real production incidents. The agent inherits a running system, often a multi-container stack of up to six services, and has to investigate, root-cause, fix and repair the data the bug already damaged, all from the terminal. Delivered in Terminal-Bench 4 format and graded on final state. The same tasks power Toloka Arena.

Volume

1,300 tasks

Domains

Software engineering, DevOps and operations, security, scientific computing, machine learning, hardware and graphics

Difficulty

Opus 5.5 never fully solves 44% of tasks in 3 tries (Qwen 3.8 Max 58%, GPT-6 Luna 81%); 8% of tasks beat all 15 models

Related
benchmarks

Terminal-Bench 2.1

Screen-Use

Available

1:1 UI-fidelity replicas of real mobile and web apps, each a self-contained RL package. Agents act on real backend state and are graded on the final state.

Volume

16 apps (13 mobile, 3 browser)

Domains

13 mobile apps ranging from food and dining to shopping, social, travel and maps, music streaming, tax and finance

Difficulty

Mobile: 650 tasks in 3 tiers. Standard 286, Medium 273, Hard 91

Best model pass rate: Standard 85%, Medium 63%, Hard 36%, all tasks 69%
Models: Claude Sonnet 5, Gemini 3.5 Flash, Gemini 3.5 Flash-Lite. 3 runs per task

Related
benchmarks

WebArena

Terminal-Bench 4

Available

Private terminal tasks for agents, spanning coding, STEM, hardware and operations. Terminal-Bench 4.0 compliant, with a multi-container setup that keeps the agent isolated from a held-out verifier.

Volume

1,500 tasks

Domains

Coding, STEM, hardware, operations

Difficulty

Calibrated against Sol6.1 High. Between 1 to 5 passes every 8 rollouts

Related
benchmarks

Terminal-Bench 4.0

 Terminal-Bench-Science

Available

Private terminal tasks built to Terminal-Bench Science's format and taxonomy. The agent works a lab, clinic or engineering team's records to produce a certified result to the field's standard, and must find the faults hidden in the data that routine checks miss. Graded on the final output by a separate, offline verifier.

Volume

200 tasks

Domains

Life sciences

Physical sciences

Earth sciences

Mathematical sciences

Difficulty

Calibrated against Meta's Muse Spark 1.3 Max: every task is solved in at most 1 of 4 rollouts

Related
benchmarks

Terminal-Bench Science, Terminal-Bench 4.0

Enterprise Tool Use

Available

RL environments where the agent works as company staff across 11 enterprise domains, using 19–39 integrated tools such as CRM, ERP and ticketing. It has to find the business rules itself in a knowledge base, and the right answer is often to refuse or escalate.

Volume

2,500+ tasks

Domains

11 enterprise domains, such as HR service desk, brokerage support, and operations

Difficulty

Frontier models score 38% or lower (pass^2)

Related
benchmarks

τ², MCP-Atlas, BFCL

Real Office Knowledge Work

Available

Agents step into a live client engagement, a project world of 400+ files, and produce the next deliverable: a recalculating workbook, a client-ready deck, a signed memo (.xlsx, .pptx, .docx). Each task is 4–8 hours of analyst work, and every task ships with an expert deliverable, reviewed and signed off by a domain expert, and scored on the same grader.

Volume

200+ tasks

Domains

Strategy, finance, pricing, operations and legal work across energy, real estate and banking

Difficulty

Expert deliverables score 0.99; gpt-6-astra averages 0.46 on the same grader

Related
benchmarks

GDPval, APEX-Agents

Frontier STEM

Available

PhD-authored problems across 10+ domains, weighted toward math and physics. Every answer is recomputed by standalone verification code. Each problem ships with a step-by-step derivation.

Volume

2,000 problems

Domains

Physics, maths, computer science, ML, biology, chemistry, electrical, civil, and mechanical engineering, data science, statistics, financial mathematics

Difficulty

Calibrated so GPT-5.4 (high) solves each problem in only 10–75% of 8 runs

Related
benchmarks

HLE, GPQA, AIME, AMO-Bench

Agentic Coding

Available

Never-published terminal tasks built from real production incidents. The agent inherits a running system, often a multi-container stack of up to six services, and has to investigate, root-cause, fix and repair the data the bug already damaged, all from the terminal. Delivered in Terminal-Bench 4 format and graded on final state. The same tasks power Toloka Arena.

Volume

1,300 tasks

Domains

Software engineering, DevOps and operations, security, scientific computing, machine learning, hardware and graphics

Difficulty

Opus 5.5 never fully solves 44% of tasks in 3 tries (Qwen 3.8 Max 58%, GPT-6 Luna 81%); 8% of tasks beat all 15 models

Related
benchmarks

Terminal-Bench 2.1

SciCode

Available

Expert-authored multi-step scientific coding tasks. Each splits into 2–5 sub-steps with ground-truth Python and assertion-based tests, in SciCode-harness-compatible JSON.

Volume

2,000 tasks

Domains

Physics, materials science, biology, maths, chemistry, computer science, engineering, data science, ML, statistics, biochemistry, optics

Difficulty

Every task ships with an 8-attempt pass record. Frontier slice vs. GPT-5.4 (high): all tasks solved 1–5 of 8, median 3 of 8

Related
benchmarks

SciCode

Screen-Use

Available

1:1 UI-fidelity replicas of real mobile and web apps, each a self-contained RL package. Agents act on real backend state and are graded on the final state.

Volume

16 apps (13 mobile, 3 browser)

Domains

13 mobile apps ranging from food and dining to shopping, social, travel and maps, music streaming, tax and finance

Difficulty

Mobile: 650 tasks in 3 tiers. Standard 286, Medium 273, Hard 91.

Best model pass rate: Standard 85%, Medium 63%, Hard 36%, all tasks 69%.Models: Claude Sonnet 5, Gemini 3.5 Flash, Gemini 3.5 Flash-Lite. 3 runs per task.

Related
benchmarks

WebArena

Enterprise Workflows

Available

The agent works business requests end to end inside a simulated company: 20+ apps rebuilt from real vendor APIs, plus 1,000+ documents, tickets, emails and chats. The fact that decides each case sits in a different place than the request points to.

Volume

500

Domains

Enterprise IT operations (5 business functions on the roadmap)

Difficulty

Opus 5.5 and GPT-6.1: ~40% on one try, 10–20% on all four tries

Related
benchmarks

τ³, AutomationBench, Toolathlon

Physical AI

Egocentric video

Available

First-person video of household and workplace tasks, with optional aligned narration or a head-and-wrist rig with 100 Hz IMU and 6-DoF pose

Volume

17,800 hours, plus 2,100 narrated hours

UMI manipulation

Available

Head and two hand-held gripper cameras, 1080p30, 100 Hz IMU, 6-DoF pose, millimeter gripper aperture

Volume

1,200+ hours

Teleoperation

Available

VR-driven bimanual and single-arm robots, 25 Hz state and actions, wrist and head RGB-D

Volume

10,000+ hours

Three ways to get started

01 — Ready to ship

Samples in 24 hours.

Fully produced, validated datasets. Request samples today, align on format, and license the full set with no ramp time.

02 — Pipeline-ready

Production live in 48 hours.

We align on volume, format and timeline at kickoff, then run our production pipeline to your spec at scale.

03 — Expand or customize

Tailored to your harness.

Grow any dataset in volume, adapt it to your domain, or turn it into a custom RL environment.

FAQs

What are Toloka's off-the-shelf datasets?

Ready-to-license, expert-authored datasets for evaluating and post-training frontier models. They're private, verified and available now, and each one can be extended.

What kinds of datasets does Toloka offer?

Agentic RL environments (enterprise tool use, enterprise knowledge, mobile use), long-horizon knowledge work, coding, STEM reasoning and scientific coding. The catalog also covers visual reasoning and physical AI data: egocentric and exocentric video, UMI manipulation and teleoperation.

Who creates the datasets?

Domain experts from Toloka's network, including PhD researchers, engineers and industry professionals.

How do you know the data is correct?

Every task has a deterministic check: verification code for STEM, unit tests for coding, and final-state checks plus task-specific rubrics for enterprise tasks, with 35,000+ criteria in the enterprise set alone.

Does training on it actually improve models?

Yes. One epoch of RL on our enterprise data lifted Qwen3.5-27B on external agent benchmarks it never trained on, including +7.6pp on AutomationBench Support and +8.9pp on τ³ retail. Contact us for more details.

How do you keep it uncontaminated?

The data has never been published and Arena has no public test split.

What can we use the datasets for?

Evaluation, SFT and RL with verifiable rewards. Agentic environments ship with deterministic graders that you can set looser for training and stricter for evaluation. Coding tasks run on Harbor, enterprise and mobile RL gyms on Toloka Forge, WorkBench on Inspect AI, and SciCode in harness-compatible JSON. Physical AI data supports robotics pre-training, imitation learning, vision-language-action policy training and world models.

What format is the data delivered in?

Formats vary by dataset: harness-ready JSON for STEM and coding tasks, Docker-packaged RL environments for enterprise and mobile gyms, MCAP plus JSON for UMI, and MP4 plus JSON for video.

Can we review samples before licensing?

Yes. Request samples and we'll send them within 24 hours. Larger trials are available for selected labs.

Can we use the data for commercial training?

Yes. Contact us for licensing terms and pricing.

Can you extend a dataset or build a custom one?

Yes. We can scale volume, adapt a dataset to your domain or harness, or build something new. Pipeline-ready datasets go live within 48 hours.

Trusted by Leading AI Teams

Request dataset samples

Get samples, specs and delivery timelines within 24 hours.

Available now