A practical session on human data for agent development.
Free, 90 min, live Q&A
Overview
Agentic AI systems can plan, use tools, and act across multi-step workflows, but reliability in production remains an open problem. Current limitations in scaling trustworthy agents aren't just a model issue. They're a data problem.
This session covers where and why human data matters across the agent development lifecycle: from pre-training and trajectory optimization to production monitoring and real-time expert escalation. Expect concrete use cases, a live demo, and time for questions.
What we’ll cover
The data gap
Why training data built for standard LLMs is insufficient for agents operating across multiple steps and modalities.
Human feedback for planning and optimization
Verifying tool selection, reasoning chains, and intermediate steps; adapting preference ranking to sequential task evaluation.
Production reliability
How human annotators catch failure modes that automated metrics miss, and how Tendem via MCP makes verified expert judgment a callable layer inside live agent workflows.
Live demo
Toloka Arena, RL Gym walkthroughs, and self-service presets you can apply to your own projects.
Format
60-minute presentation followed by a 30-minute live panel discussion and audience Q&A. Aimed at data scientists, ML engineers, and AI developers.
Free to attend. Reserve your spot for April 30. A recording will be available to registered attendees.
Hosts
Toloka, Technical Solutions Engineer
Program Director, Toloka
Evidently AI, Co-founder and CTO
Research Lead, LLM, RL & Agents, Nebius
Registration form
© 2025 Toloka AI BV