AI fuels a new wave of technical debt
For technology leaders, the management issue is not whether AI creates debt, but how quickly that debt becomes a portfolio constraint. The articleโs key implication is that AI should be treated as a business operating-model change with explicit owners, not as a collection of pilots. That means CIOs and business executives need shared decision rights for funding, scale-up, and retirement of weak use cases before AI sprawl turns hidden weaknesses into board-level risk.
The practical challenge is prioritisation under uncertainty. Leaders will need to compare productivity tools, agentic workflows, and control-heavy use cases with different value tests, rather than applying one generic ROI model. That raises an uncomfortable trade-off: slowing some deployments to fix data, APIs, permissions, and observability may delay visible wins, but ignoring those dependencies can make later scaling far more expensive. The right next question is which initiatives can prove measurable business value now and which should be paused until the enabling architecture and governance are ready.
AI also changes the risk conversation. Because agents can act at machine speed and create opaque interactions, traditional monitoring and approval processes may no longer be enough. CIOs, CFOs, risk leaders, and internal audit should align on debt-adjusted ROI, ownership of agent creation, and controls for data sovereignty, IP leakage, and compliance. A useful test is whether every material AI initiative has a named business owner, a retirement path, and monitoring that can explain cost, failure, and exposure in board terms.
At the margins
At first glance, AI-specific debt resembles other types of technical debt. It slows teams down, inflates budgets and short-circuits transformation. But AI dials up the challenges: aging code, undocumented systems and siloed data expand from an IT headache to a full-blown business problem. Because AI reshapes workflows across units and departments, CIOs must examine it through a broader lens of change management and opportunity costs. The consequences of this debt compound quickly. “It isn’t clear who owns, pays and supports AI initiatives,” said Carlos Casanova, a principal analyst at Forrester. This makes it difficult to pin down the source of a problem — or identify the right outcome. What’s more, unlike an on-premises server or infrastructure in the cloud, AI debt is often invisible — until a project goes astray, a security gap appears or a budget overrun surfaces. AI debt often hides behind early success, Schelfaut said. Chatbots assist workers, pilot projects show promise and initial rollouts deliver progress. Initiatives gain momentum, and business leaders gain confidence. Then, suddenly, as the organization attempts to scale an initiative, things go astray. “Suddenly, you can’t get systems to talk to one another, and you can’t accomplish what you had set out to do,” he said. Part of the problem is how CIOs frame the issue. Many view AI debt as an IT maintenance problem rather than a business challenge, Schelfaut said. As a result, they focus on the cost of maintaining legacy systems but overlook the barriers they impose. AI flips this logic. “Technical debt is less about what outdated systems are costing you to maintain than what they aren’t allowing you to do,” he said. Escaping this myopia starts with an understanding of what technical debt actually costs, Schelfaut said. He identified the following four distinct dimensions:-
The direct cost of running and maintaining systems and infrastructure.
-
The interest cost associated with inefficiencies that extend over time.
-
Liability costs related to security, compliance and resilience risks.
-
The opportunity costs that make it impossible for an organization to build out AI.
New rules, new tools
Things aren’t going to get any easier in the months and years ahead. According to the IBM Institute for Business Value survey, 69% of executives believe that unaddressed technical debt will render some AI initiatives financially untenable. “CIOs and CFOs need to be talking about debt-adjusted ROI now,” Lyteson said. Agentic AI raises the stakes because it introduces new risks — and exposure points. Permissions and controls designed for humans often break down when agents operate at machine speed. And because these agents communicate with each other in ways that are difficult to predict and monitor, compute and token costs can spiral, driving the need for AgentOps alongside FinOps. As agents proliferate, traditional monitoring tools fall short. New metrics and monitoring tools must deliver visibility into AI agent behavior, interactions and the infrastructure, data and models they consume. Without this visibility, CIOs can’t explain costs, risks or failures to the board, Casanova said. They also can’t intervene before issues trigger compliance, security or operational failures. The fix isn’t more technology; it’s better visibility into AI and the workflows it touches. Lyteson said a crucial starting point is to reexamine the way projects unfold — and who is responsible for them. IBM uses “AI fusion teams” that span IT and business functions. These groups “define the outcomes we want to achieve through AI, run rapid experiments to gauge how they impact workflow and engage employees to see exactly how their work changes,” he said. As IBM spins up AI projects, it measures their value against three criteria — using each as a tool to spot technical debt. Productivity tools must demonstrate time savings. Agentic workflows are held to a different standard: measurable gains in revenue growth, operational efficiency or per-unit workflow costs. Compliance and security initiatives must show a clear reduction in risk.Balancing the books
The idea isn’t to eradicate technical debt before deploying AI, Schelfaut said. It’s to identify barriers to progress and engineer essential fixes. This requires abandoning the mindset that new AI solutions can sit directly atop existing infrastructure and function within point-to-point interfaces. The good news? AI itself is a good tool for identifying issues — documenting legacy systems, rewriting fragile code and identifying what architecture needs to change. A strong governance framework is the glue that holds everything together, Casanova said. As AI tools multiply across IT and business units, organizations must fully understand hidden infrastructure costs, data sovereignty, access permissions and controls, AI sprawl and IP leakage. “If someone creates an agent, perhaps it should go into a repository for vetting before it’s deployed,” he said. In the end, CIOs must recognize that AI technical debt isn’t a problem to solve — it’s a condition to manage. Throwing technology at the challenge won’t pay down the debt. “It’s about more than transformation,” Lyteson concluded. “It is about continuous improvement. You need a framework that is good enough to start and flexible enough to refine, so you can iterate on what’s working and weed out what is not.”Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.