Enterprise Tool Use

How well agents complete real enterprise workflows across 11 industries, using simulated CRM, ERP and ticketing tools, graded on whether the task actually got done and how it got there.

11

Industries

26

Models evaluated

[date]

Prices recorded

Agentic intelligence index

Composite pass^5 score across Tool use evaluations (higher is better). 

Error bars show 95% confidence intervals. We test models on private, non-contaminated tasks.

0 of 0 models
0 of 0 models

Performance vs. cost

Models over time

Tokens used

Total cost

Performance vs. cost

Score (pass^5) vs. average inference cost per task

↖ Better: higher score, lower cost

010203040506070800.20.30.51.02.03.05.010.015.0Cost (USD per task)Composite pass^5 score (%)

Performance vs. cost

Models over time

Tokens used

Total cost

Performance vs. cost

Score (pass^5) vs. average inference cost per task

↖ Better: higher score, lower cost

010203040506070800.20.30.51.02.03.05.010.015.0Cost (USD per task)Composite pass^5 score (%)

RL Gyms and evaluation data

License tasks and environments for training and evaluating your models. Request samples to see the format and content.

Methodology

Private tasks, rebuilt in full

Test tasks are never published. We rebuild each set rather than patch it. The current Enterprise Tool Use set went live on September 29.

Tested across repeated runs

On Enterprise Tool Use, we report pass^5 across independent runs. Every score includes a confidence interval.

Tested across repeated runs

On Enterprise Tool Use, we report pass^5 across independent runs. Every score includes a confidence interval.

Every failure traced

We trace every run and classify failures, including incorrect results, wrong tool calls, policy violations and turn limits.

Every failure traced

We trace every run and classify failures, including incorrect results, wrong tool calls, policy violations and turn limits.

We publish regressions

When a new model performs worse, we publish that result too. The leaderboard records drops in performance as well as gains.

We publish regressions

When a new model performs worse, we publish that result too. The leaderboard records drops in performance as well as gains.

RL Gyms and evaluation data

License tasks and environments for training and evaluating your models. Request samples to see the format and content.