← Blog

/

Arena

Arena

Claude AI Family Guide

Claude models explained: Opus, Sonnet, Haiku, and Fable Guide

Toloka Arena is live. See how your model ranks.

See how frontier models perform on tasks they’ve never trained on

See how frontier models perform on tasks they’ve never trained on

Toloka Arena ranks Claude, GPT, Gemini, and other models by pass^5 reliability and cost per task on private, non-contaminated agentic benchmarks.

Updated September 30, 2026


Version numbers alone do not tell you where a model sits in the Claude model family, and after September 2026 they tell you less than ever. Anthropic released Opus 5.5 on September 22 and Sonnet 5.5 on September 28. Opus 5.5 performs at the level of Fable 5.1 on most work while costing 40% less to run than Opus 5, which means the Mythos-class tier above Opus is no longer automatically the strongest choice. Haiku, meanwhile, is still on version 4.5 until Haiku 5.5 ships.

The live standings say the same thing. On Toloka Arena, which scores models by pass^5, the probability that all five independent runs of a task succeed, Opus 5.5 leads the entire leaderboard at 77.3, ahead of Fable 5.1 at 74.0 and GPT-6 Astra at 70.2. Sonnet 5.5 sits fourth at 68.7, above the Opus 5 it replaces in most workflows, at a fifth of Astra’s token price.

For teams putting Claude into production, those differences translate into practical choices about capability, latency, cost, and deployment. Model lifecycle matters too: Anthropic deprecates individual model versions on published schedules, while Amazon Bedrock and Google Cloud set their own retirement timelines. Choosing a model is therefore not simply a matter of taking the newest release, but of matching its strengths and operating constraints to the job.

This guide traces how Claude’s tiers emerged, explains the differences that matter when choosing and deploying one, compares Claude with GPT, Gemini, and leading open-weight alternatives on live reliability data, and looks at where Anthropic may take the family next.

What is Claude AI?

Claude is Anthropic’s family of large language models and the name of its AI assistant and developer platform. First released publicly in March 2023, Claude is used for language, reasoning, coding, analysis, and agentic tasks, with different models offering different balances of capability, speed, and cost.

The models belong to the broader class of foundation models, general-purpose systems trained to support many downstream tasks. Claude is available through claude.ai and the Claude API, as well as on Amazon Bedrock, Google Cloud, and Microsoft Azure.

Anthropic introduced Haiku, Sonnet, and Opus in 2024 in ascending order of capability, with speed and cost as the other major trade-offs. Fable later extended the lineup above Opus as the generally available model in the newer Mythos class.

A distinctive part of Anthropic’s approach is Constitutional AI. The company publishes the constitution used in Claude’s training; its current version was released in January 2026. Anthropic says its principles help generate synthetic conversations, responses, and preference rankings.

The approach grew out of Anthropic’s 2022 Constitutional AI research. In its original formulation, a model critiques and revises responses according to written principles, then generates AI preference data for reinforcement learning. Compared with conventional RLHF, this replaces some human preference labeling with AI feedback while keeping the principles human-written.

How the Claude model family evolved

Claude did not begin with Haiku, Sonnet, and Opus. At its public launch in March 2023, Anthropic offered Claude and the faster, lower-cost Claude Instant through its API. Later that year, Claude 2 expanded the model’s output and reasoning capabilities, while Claude 2.1 increased the context window to 200,000 tokens and added features including system prompts and early tool use.

The first major structural change came with Claude 3 in March 2024. Anthropic introduced Haiku, Sonnet, and Opus together and explicitly ordered them by increasing capability. The literary names also became persistent tier labels across generations: Sonnet, for example, remained Sonnet as the version number moved on.

Three months later, Claude 3.5 Sonnet outperformed the older Claude 3 Opus across many of Anthropic’s published evaluations while retaining the speed and price of the Sonnet tier. The 3.5 generation also brought major gains in coding and vision, with Claude 3.5 Haiku following later in 2024.

The next shift was toward longer, more autonomous work. Claude 3.7 Sonnet introduced extended thinking in February 2025 alongside the research preview of Claude Code. Claude 4 followed in May with Sonnet 4 and Opus 4, putting coding, tool use, and long-running agent workflows at the center of the launch. The 4.5 generation then brought Sonnet 4.5, Haiku 4.5, and Opus 4.5.

The cadence accelerated again in 2026: Opus 4.6 and Sonnet 4.6 arrived in February, followed by Opus 4.7 in April and Opus 4.8 in May. By then, Anthropic reported that Claude authored more than 80 percent of the code merged into its own codebase. That is an internal figure rather than a general benchmark, but it shows how far Claude’s role had moved from answering prompts toward carrying out sustained work.

The family itself expanded in April 2026 with the limited-access Mythos Preview. Fable 5 and Mythos 5 followed in June, establishing a Mythos-class tier above Opus; Sonnet 5 arrived at the end of June and Opus 5 in July. Anthropic then refreshed the frontier tier with Fable 5.1 and Mythos 5.1 on September 1.

September 2026 brought the change that matters most for model selection today. Anthropic opened a new 5.5 generation with Opus first rather than Sonnet, releasing Opus 5.5 on September 22 and Sonnet 5.5 on September 28, with Haiku 5.5 promised in the following weeks. Opus 5.5 was the company’s first release after CEO Dario Amodei’s September call to pace the frontier, and it shipped with the safeguards Anthropic had built for Fable 5.1. For the wider chronology behind this shift, see our history of LLMs.

Claude AI models: Haiku, Sonnet, Opus, and Fable

With the lineage established, the useful distinction among current Claude models is how they spend inference budget. Haiku 4.5 is optimized for speed, while Sonnet 5.5 and Opus 5.5 use adaptive thinking and let developers adjust effort to trade capability against latency and token use. Fable 5.1 keeps adaptive thinking permanently enabled.

Context length also separates the generations: Haiku 4.5 has a 200,000-token window, while Sonnet 5.5, Opus 5.5, and Fable 5.1 support 1 million tokens, with up to 128K output tokens per request on the frontier models.

List prices are a starting point rather than a cost model, because the models differ in how many tokens they spend to finish the same task. The comparisons below pair published rates with observed cost per task where it is available.

Claude Haiku: The fastest, lowest-cost tier

Haiku 4.5 is the operational outlier in the current family. It has the shortest context window and does not use the adaptive-thinking system found in the 5-series models. Developers can instead enable extended thinking explicitly when a task needs it. For workloads built around tight feedback loops, that gives Haiku a different role from simply being a less capable Claude.

Anthropic points to chat assistants, customer-service agents, pair programming, rapid prototyping, and multi-agent systems as examples. At launch, Haiku 4.5 delivered coding performance similar to Sonnet 4 at one-third the cost and more than twice the speed, and Anthropic showed Sonnet 4.5 acting as a planner that delegated subtasks to several Haiku instances in parallel.

That makes Haiku particularly useful as a worker model when a system needs many fast calls rather than one expensive, deeply reasoned response. Haiku 5.5 is expected to close part of the capability gap while keeping that role; Anthropic has said it is built for high-volume and cost-sensitive applications.

Claude Sonnet: The balanced production tier

Sonnet 5.5 is the clearest illustration of how quickly the middle tier is moving. Anthropic reports it scoring 70.6% on Terminal-Bench 4.0, an agentic coding evaluation, against 10.3% for Sonnet 5, and positions it as a faster, lower-cost complement to Opus 5.5 rather than a scaled-down version of it. It keeps Sonnet 5’s pricing at $2 per million input tokens and $10 per million output tokens while generating output more than 30% faster and costing up to 30% less per task in Anthropic’s testing.

The same jump shows up on independent testing. Sonnet 5 scores 25.4 on the Toloka Arena composite pass^5 index, near the bottom of the frontier field, because pass^5 punishes a model that solves a task most of the time but not every time. Sonnet 5.5 scores 68.7, ahead of Opus 5 at 66.2 and GPT-6 Sol at 63.2. The generational gain is therefore mostly a consistency gain, which is exactly the property single-run benchmarks hide.

Anthropic describes Sonnet 5.5 as strongest at well-scoped everyday tasks, bug fixing, and producing polished documents, slides, and spreadsheets. It is also the first Sonnet to complete Pokémon Red from screenshots alone, a long-horizon test of sustained work and image understanding, and the first Sonnet to ship with the cyber safeguards and higher-risk fallback behavior previously reserved for Anthropic’s most capable models.

Defaults matter in practice: effort is set to Medium in Claude Code and the Claude apps, and High on the Claude Platform, which is also the API default.

Claude Opus: The high-capability core tier

Opus 5.5 changed the shape of the lineup. Anthropic says it performs at the level of Fable 5.1 on most work while costing 40% less to run than Opus 5, and priced it at $4 per million input tokens and $20 per million output tokens, down from Opus 5’s $5 and $25. It leads Anthropic’s own benchmarks in agentic coding, computer use, and knowledge work, and the company calls it the strongest performer to date on its automated behavioral audit, the alignment suite it runs across thousands of simulated scenarios.

On Toloka Arena it is the top-ranked model overall, with a composite pass^5 of 77.3 against 74.0 for Fable 5.1 and 70.2 for GPT-6 Astra. Because Fable 5.1 lists at $10 and $50 per million tokens, Opus 5.5 currently offers the highest measured reliability in the field at roughly 40% of the frontier tier’s token price, which is an unusual position for a model one tier below the top.

Opus 5.5 also makes test-time computation part of model selection. Developers can adjust its effort level rather than changing models whenever a task needs more or less reasoning, effectively moving along a capability, latency, and token-use curve within Opus itself. This kind of controllable inference is part of the broader shift toward reasoning in large language models.

A second change is communication rather than capability. Anthropic says Opus 5.5 puts important information up front, uses less jargon, and follows writing rules more reliably, addressing common feedback about Opus 5 in long working sessions. Because Opus 5.5 is comparable to Mythos 5.1 in biology and cybersecurity, it deploys with Fable-class safeguards, and vetted organizations can apply for expanded access through Anthropic’s Life Sciences and Cyber Verification programs.

Claude Fable: The Mythos-class frontier model

Fable 5.1, released September 1, 2026, is the current Mythos-class model and was the most capable publicly available Claude model at launch. It combines always-on adaptive thinking with a large output budget, and Anthropic reports that its advantage over other Claude models increases with task length and complexity.

In Stripe’s early testing of the previous Fable 5, as reported by Anthropic, the model completed a codebase-wide migration across a 50-million-line Ruby codebase in a day, work Stripe estimated would otherwise have taken a team more than two months.

Opus 5.5 complicates the choice. On most work the two now perform at a similar level, and on Arena’s multi-turn tool-use tasks Opus 5.5 is ahead. Fable remains the right default where maximum capability on very long, open-ended runs justifies the higher rate, but it is no longer the automatic answer to “which Claude is strongest.”

Choosing Fable also has an operational consequence that raw capability tables do not show: its safeguards can change which model actually answers a request. Fable runs additional classifiers for sensitive domains, and some requests are routed to a less capable model instead. Anthropic substantially reduced false-positive biology fallbacks in an August 2026 update, but dual-use areas including virology, toxicology, and molecular design still fall back to a lower tier. Sonnet 5.5 and Opus 5.5 now behave the same way for higher-risk cyber requests, so the requested model and the served model should both be logged.

The practical difference between Fable and Mythos is the safeguard layer. They use the same underlying model, but Mythos omits some of Fable’s classifiers and is available only to approved organizations through trusted-access programs such as Project Glasswing.

Access briefly became part of the model’s story too. A US export-control directive led Anthropic to suspend Fable 5 and Mythos 5 on June 12, 2026. The controls were lifted later that month, and access was restored on July 1.

Claude Opus vs Sonnet vs Haiku vs Fable

The differences among current Claude models become clearest when the same workload constraints are compared side by side. Haiku minimizes latency and cost; Sonnet trades some of that efficiency for substantially more headroom; Opus now combines the highest measured reliability with mid-frontier pricing; and Fable holds the ceiling for long-running, capability-sensitive tasks.

Current specifications also split the family by reasoning controls: Haiku uses manually enabled extended thinking, Sonnet and Opus use adaptive thinking, and Fable keeps adaptive thinking on by default.

Model

Position

Best for

Relative cost

Arena pass^5

Haiku 4.5

Lightweight

Fast, high-volume and routine tasks

Lowest

Not listed

Sonnet 5.5

Balanced production tier

General development, agents, most production workloads

Medium

68.7

Opus 5.5

High-capability core tier

Complex reasoning, coding, and agentic work

High

77.3

Fable 5.1

Mythos-class frontier

Long-running, capability-sensitive tasks

Highest

74.0

Arena pass^5 figures are the composite tool-use scores as of September 29, 2026. Haiku 4.5 is not currently listed on the leaderboard.

The table gets you to a shortlist. From there, Anthropic recommends evaluating those models on representative prompts, data, and edge cases, then weighing the resulting quality against latency and cost. Single-run benchmark scores hide a dimension that matters in production: whether a model succeeds every time or only on average. Toloka Arena scores models by pass^5, the probability that all five independent trials of the same task succeed, which is exactly the consistency question single-run numbers miss.

A practical strategy is therefore to start at the cheapest tier that plausibly satisfies the workload and move upward only when evaluation exposes a capability gap. The 5.5 generation makes that cheaper than it used to be: Sonnet 5.5 covers more of the middle at unchanged prices, and Opus 5.5 raises the ceiling of the non-Mythos tiers while costing less to run than Opus 5.

How Claude models rank on Toloka Arena

Toloka Arena evaluates frontier models on private, non-contaminated agentic benchmarks: multi-turn customer-service scenarios with live databases, tool calls, and strict business rules, inspired by the τ-bench methodology. Because the tasks are never in any model’s training data, the standings show how these families behave on unseen work rather than on memorized tests. The primary metric is pass^5, the probability that all five independent trials of the same task succeed.

The snapshot below is the composite tool-use index with published API list prices for context.

Model

Composite pass^5

API price (in / out, per 1M)

Claude Opus 5.5

77.3

$4 / $20

Claude Fable 5.1

74.0

$10 / $50

GPT-6 Astra

70.2

$10 / $50

Claude Sonnet 5.5

68.7

$2 / $10

Claude Opus 5

66.2

$5 / $25

GPT-5.6 Sol

64.7

$4 / $20

Gemini 3.8 Flash

63.5

$0.75 / $3.75

GPT-6 Sol

63.2

$2 / $10

Gemini 3.7 Flash

62.7

$0.75 / $3.75

Grok 4.6

62.1

$2 / $6

GPT-6 Luna

39.1

$0.10 / $0.50

Claude Sonnet 5

25.4

$2 / $10

Source: Toloka Arena composite pass^5 index. Prices are published list rates per million tokens and exclude caching discounts.

Three things stand out. First, the Claude tier order no longer matches the capability order: Opus 5.5 sits above Fable 5.1 at 40% of its token price. Second, the generational gap inside a tier can be larger than the gap between tiers, as Sonnet 5 at 25.4 and Sonnet 5.5 at 68.7 show. Third, a newer release is not automatically a more reliable one, which is visible in the gap between Grok 4.6 at 62.1 and Grok 4.7 at 37.5.

Standings also differ by domain. Arena offers separate boards for airline, banking HR, manufacturing, logistics, hotel, restaurant, and short-term-rental scenarios, so the domain closest to your own workload is more informative than the composite score.

Evaluate your model on private benchmarks

Test your model on Toloka Arena’s non-contaminated agentic tasks, or license the RL and evaluation datasets behind the leaderboard.

Test your model →

Claude vs other AI model families

The snapshot below reflects public model lineups at the end of September 2026. Benchmark positions change quickly, so our LLM leaderboard overview is the better place for current rankings, and Toloka Arena tracks how Claude, GPT, Gemini, and Grok models perform on private agentic benchmarks, scored by pass^5 reliability and cost per task. The comparison here focuses mainly on differences in how these families are packaged, deployed, and controlled.

Claude vs ChatGPT and GPT models

For teams deploying AI systems, the closer comparison is Claude versus the GPT model family rather than Claude versus ChatGPT, which is the consumer-facing app.

OpenAI replaced its GPT-5.6 lineup with GPT-6 over September 2026. GPT-6 Astra, released September 3, is the flagship at $10 per million input tokens and $50 per million output tokens, and OpenAI describes it as state of the art on computer use, browsing, software engineering, and professional work; it is also the company’s first model to reach the Critical cybersecurity threshold under its Preparedness Framework, which is why advanced cyber capability is restricted to vetted organizations. GPT-6 Sol and GPT-6 Luna followed on September 22 at $2 / $10 and $0.10 / $0.50, roughly half their GPT-5.6 equivalents.

That makes the Claude comparison unusually direct, because the two families now price at similar points. Astra and Fable 5.1 share the same $10 / $50 list rate. GPT-6 Sol and Sonnet 5.5 share $2 / $10. Opus 5.5 sits between them at $4 / $20 with no exact OpenAI counterpart.

On Arena’s pass^5 index the Claude models lead at each of those price points as of September 29: Opus 5.5 at 77.3 and Fable 5.1 at 74.0 are ahead of Astra at 70.2, and Sonnet 5.5 at 68.7 is ahead of GPT-6 Sol at 63.2 at identical token prices. Anthropic’s own figures point the same way, with Sonnet 5.5 at High effort matching GPT-6 Sol’s best CursorBench score for about a fifth of the cost per task.

Benchmark leadership still moves by task, and the vendors’ own evaluations disagree with each other. OpenAI reports Astra reaching 57.9% on its computer-use evaluation against 55.8% for Fable 5.1 at roughly 63% lower estimated API cost per task, while Anthropic reports Opus 5.5 leading on agentic coding and knowledge work. Running both families on your own workload, or on an independent non-contaminated benchmark, resolves more than reading either launch post.

Claude vs Gemini

Google’s Gemini lineup combines general-purpose models with a broader set of specialized variants for different modalities and workloads. Its current API includes Flash and Flash-Lite variants alongside Pro, plus separate models for image generation, real-time interaction, translation, and other multimodal tasks.

One quirk matters when shortlisting: Gemini tier names do not reliably indicate capability order. Gemini 3.8 Flash scores 63.5 on Arena, well ahead of Gemini 3.1 Pro at 10.0, so the Flash label does not mean a weaker model in the way the Haiku label does inside the Claude family. Google’s breadth still matters most when audio, image, or real-time interaction is central to the application rather than an added capability around a primarily text-and-code model.

Claude vs Grok

xAI’s Grok lineup is narrower and more concentrated around coding, agentic tasks, and knowledge work, and xAI does not divide its models as explicitly into persistent capability tiers. On Arena, Grok 4.6 reaches 62.1 while the newer Grok 4.7 records 37.5, a reminder that version order and reliability order are separate questions in every family, not only Claude’s.

Claude vs open-weight models

The bigger architectural choice appears when Claude is compared with open-weight families such as Llama, DeepSeek, Qwen, and GLM. Many models in these families make their weights available for teams to deploy themselves, modify where the license permits, and integrate into infrastructure they control.

The exact freedoms depend on the model’s license, which is why “open-weight” is often more precise than “open-source.” The reliability gap is still visible: on Arena the strongest open-weight entries trail the frontier closed models by a wide margin on multi-turn tool use.

That changes the engineering trade-off. With Claude, model serving and vendor-side safeguards remain with Anthropic; self-hosted models give teams more control over infrastructure, fine-tuning, and data flow, while shifting more responsibility for deployment and maintenance to the team running them. Our guide to open-source vs closed-source LLMs goes deeper into that distinction.

Anthropic models: What comes next for Claude?

The immediate roadmap is public. Anthropic has said Haiku 5.5 will complete the 5.5 family in the coming weeks, built for high-volume and cost-sensitive applications, with the same improvements to performance, efficiency, and safety that Opus 5.5 and Sonnet 5.5 received.

Beyond that, the pattern to watch is the relationship between the Opus and Mythos tiers. Opus 5.5 reaching Fable 5.1’s level at a lower price suggests Anthropic can push frontier capability down the price curve faster than it extends the ceiling, which would make the Mythos class a narrower, longer-horizon option rather than the default answer for hard work.

The model is also becoming only one layer of the product. Claude Science, for example, combines Claude with research tools, compute, and auditable artifacts rather than treating the model as a standalone chat interface. Anthropic has taken similar domain-specific approaches in software development and cybersecurity.

Access may also become more differentiated as capability rises. Fable and Mythos already show that the same underlying model can be offered under different safeguard and access regimes, and the verification programs now attached to Opus 5.5 and Sonnet 5.5 extend that pattern down the lineup. Pacing commitments add a further variable: Opus 5.5 was the first release after Anthropic publicly argued for slowing capability gains to match progress on alignment.

How human data supports model training and evaluation

Constitutional AI automates part of the preference-labeling process, but Anthropic still relies on people to design evaluations, investigate failures, and red-team its models. Its system cards combine automated testing with human and expert review, especially for behavior that is hard to capture with a single metric.

Human input matters during training and fine-tuning when the target behavior cannot be captured by a simple rule or automated score. Domain experts can provide examples, preference judgments, and annotations for specialist tasks, while gaps in that data can later show up as uneven model performance.

Automated evaluators can make testing easier to scale, but teams still use human judgments to check what those scores actually capture. With agents, that often means evaluating intermediate steps such as plans, tool calls, and error recovery rather than looking only at the final answer, which is the same reason a reliability metric like pass^5 separates models that a single-run score groups together.

Human data is therefore most valuable where the target is hard to measure automatically: specialist knowledge, subjective quality, unusual failure modes, and safety boundaries. Toloka provides data for LLM training and fine-tuning, expert model evaluation, data for coding models, and AI safety testing for those parts of the development process.


Train and evaluate your AI with expert human data

Toloka Platform delivers high-quality data for LLM training, RLHF, and model evaluation.

Get started free →

Frequently asked questions

What is the Claude model family?

Claude is Anthropic’s family of general-purpose AI models. The names Haiku, Sonnet, Opus, and Fable identify different tiers, while numbers such as 4.5, 5, or 5.5 identify generations within those tiers. As of September 2026 the current models are Haiku 4.5, Sonnet 5.5, Opus 5.5, and Fable 5.1, with Haiku 5.5 announced for the coming weeks.

Is Opus 5.5 better than Fable 5.1?

On most work they are close, and on some measures Opus 5.5 is ahead. Anthropic says Opus 5.5 performs at Fable 5.1’s level for most tasks while costing 40% less to run than Opus 5, and on Toloka Arena’s multi-turn tool-use tasks Opus 5.5 leads the whole leaderboard with a composite pass^5 of 77.3 against 74.0 for Fable 5.1. Fable still holds the Mythos-class position for very long, open-ended runs, but it is no longer the automatic choice for demanding work.



Which Claude model is best for coding?

There is no single coding tier for every workload. Anthropic reports Sonnet 5.5 scoring 70.6% on Terminal-Bench 4.0 against 10.3% for Sonnet 5, which makes it strong on well-scoped coding work at $2 / $10 per million tokens, while Opus 5.5 leads Anthropic’s agentic coding and knowledge-work benchmarks and remains clearly stronger on complex, open-ended problems. Fable 5.1 is aimed at ambitious, long-running projects that may run for hours or days.

How does Claude compare with GPT-6 Astra?

GPT-6 Astra, released September 3, 2026, is OpenAI’s flagship at $10 / $50 per million tokens, the same list price as Fable 5.1. On Toloka Arena’s composite pass^5 index as of September 29, Astra scores 70.2, behind Opus 5.5 at 77.3 and Fable 5.1 at 74.0 and ahead of Sonnet 5.5 at 68.7. OpenAI reports Astra ahead of Fable 5.1 on its own computer-use evaluation at a lower estimated cost per task, so the answer depends on the workload and is worth testing directly.



Which Claude model is best for agentic tool use?

It depends on how much reliability the workflow needs. Toloka Arena evaluates Claude models alongside GPT, Gemini, and Grok on private, multi-turn tool-use tasks with strict business rules, scoring each model by pass^5, the probability that all five independent runs of a task succeed. Opus 5.5 currently leads at 77.3 and Sonnet 5.5 reaches 68.7 at a quarter of the price, so the cheaper tier is a real option for many agent workloads. Standings differ across Arena’s domains, from airline and manufacturing scenarios to banking HR and logistics, so check the domain closest to your own workload rather than the composite score alone.

What is the difference between Fable and Mythos?

Fable 5.1 and Mythos 5.1 use the same underlying model. Fable adds safeguards that make Mythos-level capability suitable for general availability; Mythos removes some of those restrictions and is available only through trusted-access programs such as Project Glasswing. Since Opus 5.5, comparable safeguards and fallback behavior have also applied lower down the lineup.

From model choice to system design

The lesson from Claude’s expanding family is that model selection increasingly happens at the task level. A single project may need fast, inexpensive inference in one part of the workflow, deeper reasoning in another, and a frontier model only for a small set of difficult cases. The practical question is therefore how to build the right model architecture for the project.

The 5.5 generation sharpens that point. When a mid-tier model closes most of the gap to the previous flagship at a fifth of the frontier price, the cost of routing badly grows, and so does the value of knowing which model actually holds up across repeated runs rather than winning a single attempt.

Those routing decisions are only as good as the evaluation data behind them. Teams need representative prompts, edge cases, and human judgments that reflect real failure modes, and ideally independent evaluation on tasks models have never trained on. Otherwise, the model that looks best in testing may not be the one that performs best in production.


Related reading

Toloka Arena: independent evaluation of agentic intelligence

Base LLM vs instruction-tuned LLM

GPT models 2026 explained



Subscribe to our newsletter

Product updates, case studies and the latest news from our team, straight to your inbox