Let’s talk →
Intelligence · Research brief

The Autonomy Paradox Why less autonomy may deliver greater business value.

A new study of real deployments finds the agents creating value today are narrow, controlled and human-checked. Here’s what it says, and what it means for anyone trying to put AI to work.

By Evan Forsberg, Chief AI & Strategy Officer6 min readOn: Measuring Agents in Production, Pan, Arabzadeh et al. (ICML 2026)Read the paper ↓

The skinny

The AI agents actually working in businesses today are much simpler, more controlled and less autonomous than the industry suggests. And that’s not necessarily a bad thing.

Researchers studied 20 real-world deployments in depth and surveyed practitioners about 86 systems in production or pilot.

The biggest findings, in plain English

  • 68%

    Agents aren’t running wild.

    Most take fewer than 10 steps before a human is involved. Companies deliberately limit what AI can do on its own.

  • 70%

    Companies aren’t building their own models.

    They use existing models like GPT and Claude, give them instructions, and engineer systems around them, rather than training their own.

  • 74%

    Humans are still checking the work.

    People remain central to validating AI output. Fully hands-off operation isn’t the norm.

  • 80%

    Productivity is the main motivation.

    Organizations mostly want existing work done faster with less manual effort, not entirely new autonomous businesses.

  • 66%

    Speed isn’t always critical.

    Many agents take minutes or longer. If a task takes a person three hours, five minutes is still a massive improvement.

Figures from pages 1–7 of the paper. Percentages come from different subsets of respondents, not one uniform sample.

The most important finding

Reliability is the biggest challenge, not raw intelligence.

Companies are deliberately trading some autonomy for predictability. Instead of telling an agent “go manage our procurement process,” they break the work into steps and decide exactly where a person signs off.

The real engineering is in connecting those steps, handling exceptions, securing data and making sure the output is consistently right.

85% of the 20 case studies used custom-built agents rather than third-party frameworks. The broader survey showed more framework use, so that preference wasn’t universal.

A procurement agent, in practice

5 steps

  1. 1AI collects supplier informationAI
  2. 2AI compares prices and termsAI
  3. 3AI generates a recommendationAI
  4. 4A human reviews and approves itHuman
  5. 5The system executes the approved actionSystem

What this means for AI services

Three business implications

  1. 01

    Deployment beats model access.

    Everyone can reach powerful models. The value comes from wiring them into business processes, proprietary data, decision rights and existing systems.

  2. 02

    More autonomy isn’t more value.

    A well-engineered agent that reliably removes four hours of work a day can be worth far more than a sophisticated multi-agent system that occasionally makes serious mistakes.

  3. 03

    Human oversight is architecture, not a limitation.

    The paper suggests people in the loop may be a permanent feature of effective AI systems, not something organizations eventually remove.

75%

One warning: 75% of the detailed cases had no formal evaluation benchmarks, a real maturity gap. Security was also often handled by restricting what agents could access or change, rather than through comprehensive safeguards.

My biggest takeaway

Building an AI agent and transforming a business are two very different things.

You can get tremendous, even asymmetric, value from an agent working inside a narrowly defined workflow.

But making AI work reliably across an enterprise takes more than one agent. It takes architecture, integration, testing, security, human oversight and an understanding of how the organization actually operates.

Caveat: this is an observational study of a self-selected sample that includes pilots. It shows what practitioners were doing, not proof that simpler agents always beat more autonomous ones.

47 pages, one sentence

The companies getting agents into production aren’t giving AI more freedom. They’re getting better at designing the systems, controls and workflows that make it useful.

That’s the gap between AI capability and AI deployment maturity.

Read the full paper

Measuring Agents in Production

Proceedings of the 43rd International Conference on Machine Learning (ICML), Seoul, 2026 · 47 pages

Read the paper on arXiv ↗

Authors

Melissa Z. Pan¹*, Negar Arabzadeh¹*, Riccardo Cogo², Yuxuan Zhu³, Alexander Xiong¹, Lakshya A Agrawal¹, Huanzhi Mao¹, Emma Shen¹, Sid Pallerla¹, Liana Patel⁴, Shu Liu¹, Tianneng Shi¹, Xiaoyuan Liu¹, Jared Quincy Davis⁴, Emmanuele Lacavalla², Alessandro Basile², Shuyi Yang², Paul Castro⁵, Daniel Kang³, Koushik Sen¹, Dawn Song¹, Joseph E. Gonzalez¹, Ion Stoica¹, Matei Zaharia¹*, Marquita Ellis⁵*

¹ UC Berkeley² Intesa Sanpaolo³ UIUC⁴ Stanford University⁵ IBM Research* Project co-leads

All findings and figures on this page are from the authors’ work. Commentary and interpretation are Avanta Labs’ own.

Have a workflow that could run this way?

We’ll help you find where an agent pays off, design the checkpoints, and get it into production.

Let’s talk →Not sure where to start?
Source: Pan, M. Z., Arabzadeh, N., et al. Measuring Agents in Production. Proceedings of the 43rd International Conference on Machine Learning, PMLR 306, 2026. arXiv:2512.04123. Researchers from UC Berkeley, Intesa Sanpaolo, UIUC, Stanford University and IBM Research.