AI agents promise to revolutionize the enterprise. They already write code, resolve tickets and orchestrate workflows across applications and systems. But as CIOs race to deploy them, they’re discovering an uncomfortable truth: It’s remarkably difficult to string AI tools together and create a multi-agent framework that does more than cut costs.
CIOs are under pressure to show quantifiable value from agentic AI investments. But there’s a catch, said Anushree Verma, senior director analyst at Gartner: Many organizations continue to rely on conventional metrics like headcount reduction, time savings and productivity gains to measure investments. These approaches cannot capture the “unique cost and value dynamics of AI-agent-powered workflows,” she said.
Gartner predicts that 40% of enterprise applications will feature task-specific AI agents by the end of 2026, yet organizations will pull the plug on 40% of agentic AI projects by 2027. “CIOs must move away from opaque vendor-supplied measures and [focus on] financial gains that matter most to business leaders,” Verma cautioned.
Matt Kropp, managing director and chief technology officer at BCG X, an AI integration arm of Boston Consulting Group, echoed these concerns. “There is a lot of hype, and there are a lot of emerging vendor solutions that describe themselves as agent-based AI. For now, most deliver very little ROI.”
AI orchestration is essential for reducing fragmentation, observability gaps, and security risks, Kropp said.
Beyond chatbots
Stitching together autonomous agents to streamline complex processes is an appealing proposition. By 2028, 15% of all daily work decisions will take place autonomously, Gartner predicts. Yet there’s significant confusion about what constitutes agentic AI — and many vendors magnify the problem by falsely claiming that chatbots and other AI tools deliver agentic capabilities.
Agentic AI is built on autonomous systems that can perceive their environment, make decisions and take actions to achieve specific goals — with little or no human intervention. Ideally, it’s a way to rewire workflows. Yet agents — and groups of agents that create multi-agent frameworks — don’t just redistribute tasks; they reshuffle traditional roles and create new functions.
Rethinking roles and accountability
Used effectively, greater organizational intelligence follows. “By continuously sensing, deciding and adapting processes across agents, multi-agent systems enable organizations to dynamically respond to demand shifts, disruptions and opportunities in near real time,” explained Jacob Wilson, AI factory leader at PwC. “There’s a shift from people moving faster to the business moving faster and better.”
Multi-agent systems require more than new tools and technology, however. As DevOps teams and other groups switch from writing code to prompting agents to write code for them, thinking must change, too, Kropp said. He described this as a fundamental departure from previous waves of automation. “It requires that you stop thinking in code and, instead, think in terms of features and capabilities.”
Owning outcomes
CIOs must reset their thinking in other ways. It’s critical to accept that “humans no longer need to touch every line of code to stay in control,” PwC’s Wilson said. In an agent-driven software development lifecycle, people must define processes, workflows and critical metrics — and then let agents “handle the heavy lifting.” SDLC doesn’t disappear; it evolves into a more adaptive and automated system with humans validating outcomes and intervening when signals indicate drift, he said. The same logic applies to ITSM, which has traditionally involved human-centric ticketing. In a multi-agent system, service handoffs take place in milliseconds between machines, and there is no “ticket” to serve as a unit of record. Instead, incident resolution shifts to near real time, with machine-to-machine handoffs across monitoring, diagnosis, remediation, and validation. In this model, a digital “unit of record” captures intent, context, actions, outcomes and human interventions.
From efficiency metrics to business value
With this information on hand, an organization can evolve from basic tactical metrics like time savings or cost savings to strategic metrics such as enhanced capital velocity, time-to-value and idea-to-impact. For example, this might translate to cutting mean time to resolution from hours to minutes, or increasing the number of production experiments teams can run during a specific period. Developers, DevOps teams and others begin to focus on ways to unlock business value. “Is the objective to code in 30% less time or ship new and innovative features — and enhance a product?” Wilson asked.
Governance in an agent-driven enterprise
Not surprisingly, strong governance is essential. In an agent-centric enterprise, the human role shifts to orchestration and audit — with agents aiding in observability, analysis and quality control, Wilson pointed out. Feedback loops continuously improve behavior based on performance, drift signals and business outcomes, rather than teams executing line-by-line code reviews. The result, Wilson said, is an auditable accountability stream that advances the organizational focus from managing requests to governing autonomous operations.
Yet, even with all the right controls in place, it’s important to build in safeguards. This includes a built-in kill switch if an agentic system goes haywire. An organization must designate human owners who set objectives, guardrails and risk thresholds, Wilson said. At the same time, CIOs must establish “hard guardrails that trigger escalation, throttling or shutdown when agents exceed confidence, financial or compliance thresholds.”
Orchestrating success
As organizations build out multi-agent AI frameworks, Gartner’s Verma advised CIOs to thoroughly vet vendors and understand their vision and definitions of agentic AI. It’s also important to account for internal organizational factors such as scalability, agent sprawl, and technical debt. Success, she said, depends on learning to translate complex, multifaceted business processes into highly orchestrated agent workflows.
Verma also urged CIOs to keep an eye on emerging standards and connectivity tools that reduce vendor lock-in and make it easier to swap tools without rebuilding workflows. Model Context Protocol, for example, simplifies connections between LLMs and enterprise systems. Gartner predicts it will drive 50% of AI integrations by 2028, up from 2% today. Stripe’s Agentic Commerce Protocol, Google’s Agent Payments Protocol and various agent-to-agent interaction standards have also emerged.
Best practice organizations focus on four core characteristics — orchestration, observability, security and governance, Wilson said. Together, they enable an AgentOps function that keeps CIOs in control by setting standards, managing risk and enforcing responsible AI. At the same time, developers and business units remain in charge of key outcomes.
“Leadership in the AI race will not be achieved by those chasing better models but by those who close the value gap,” Verma concluded.
Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.