Amid rapid-fire AI model releases, daily deprecations, geopolitical uncertainty, token costs, compliance risks and uneven vendor service guarantees, CIOs are confronting a new and sometimes disturbing reality: AI model churn has become an operational problem, not just a technical one.
Validating AI models — and ensuring that they work with existing systems — consumes significant time and resources.
“Model deprecation is rarely just a technology event,” said Scott Likens, U.S. and global chief AI engineering officer at consulting firm PwC. “It can trigger a chain reaction across governance, risk, compliance, model validation, security and operational teams.”
The challenge is more than replacing one model with another. As AI is intercalated into business applications and processes, organizations need architectures and governance frameworks that absorb constant model changes without disrupting the business.
This dependency “extends far beyond the API,” Likens explained. Today, a resilient architecture and strong governance framework are as important as raw model performance.
As a result, CIOs are turning to a multi-model strategy that relies on abstraction and orchestration layers to guard against IT and business disruption. They’re also using validation methods, model registries, prompt and version controls, lifecycle governance and cost-aware routing to better match models with use cases.
Model citizens
Organizations have near-zero control over how providers such as Anthropic, OpenAI and Google update, retire or replace their models. In many cases, AI providers retire versions within weeks. Yet many enterprises — particularly those in regulated industries, including banks, biopharma, aviation and healthcare — require months to fully validate changes.
Organizations in unregulated industries may be vulnerable for different reasons. Without strict guardrails and mandated reviews — and with AI scattered across functions like customer service, sales, support, marketing and operations — things can quickly go off the rails. When an AI provider deprecates a model, validation methods can break, workflows can stall, and security risks can rise.
“If you haven’t sufficiently tested a model, it may do something that it is not supposed to do,” said Ashwin Bhave, senior partner in the technology practice at Boston Consulting Group. And while it can be tempting to use version pinning and contract clauses to guarantee transition windows, these tools can backfire because models can produce different results from one minute to the next — and version pinning can throttle innovation.
Related:How ADP’s chief AI officer keeps a tight rein on AI without holding it back
Model churn also carries financial consequences. Token consumption, inference charges and repeated large-scale testing during revalidation can also cause costs to spiral. In the coming months, the situation could worsen if frontier providers raise their prices. At present, “a lot of the costs are being propped up by subsidies that aren’t likely to persist,” Bhave said.
There’s also a question of whether a model will exist when it’s needed. For example, on June 12, Anthropic pulled access to its leading-edge models, Fable 5 and Mythos 5, to comply with U.S. government export controls. The ensuing disruption extended across regions and companies until the suspension was lifted on June 30.
Further complicating matters, service guarantees and SLAs widely used in the tech industry are often lacking in the AI space, where uptime commitments, disaster recovery and data-loss protections frequently lag.
A model outage doesn’t just break an app; it can take down an entire workflow, business process or tool chain, noted Kai Waehner, an independent consultant based in Germany. And while self-hosting may seem like a viable option, this can introduce a separate set of problems, including finding talent to manage the systems.
Building for constant AI change
These forces are reshaping the way organizations approach AI model management. What used to be a decision handled by engineers — selecting vendors and software — now lands squarely on the CIO’s desk.
As CIOs delve into the intricacies of AI model management, they are discovering that it isn’t IT as usual. While it generally makes sense — and saves dollars — to standardize hardware and software, AI is different: A multi-model, multi-vendor framework typically reduces risk while allowing an organization to assign the most efficient and cost-effective models to specific projects and tasks.
A one-size-fits-all governance framework also fails. Instead, it’s necessary to build orchestration and abstraction layers that reside outside specific AI models. This makes it possible to sync governance, security and AI functions across models and users — and make changes dynamically. There’s no need to update a company’s controls every time a model updates or an enterprise replaces it, Waehner pointed out.
Free of the constraints imposed by a single model or an overly rigid framework, an enterprise can adapt to change and plug in the right AI model for the right task. This includes using lower-cost, open-source models such as DeepSeek and Mistral without constantly adapting the entire IT environment. As Likens explained: “Every organization wants access to innovation, but not every workload requires the newest model on day one.”
A flexible AI framework doesn’t eliminate the need for model management, but it does dial down complexity. When abstraction and orchestration layers separate business logic, workflows, governance, prompts, data definitions and routing, deprecation ceases to be a tangible threat. It’s possible to swap models on the fly.
“A well-designed abstraction layer can significantly reduce operational dependency on any single model provider,” Likens said.
With a solid AI foundation, a CIO can plug in other tools that further enhance visibility — and flexibility. This includes a model registry that displays approved products, owners, use cases, versions, cost profiles and migration plans. Combined with using regression-style testing, a CIO or chief AI officer can evaluate a new or updated model and make necessary tweaks before introducing it to the entire organization.
“Abstraction isn’t a magic shield,” Likens said. “It can simplify model replacement, but it doesn’t eliminate the need to understand how a new model performs, how its outputs differ or whether it introduces new risk considerations.”
How CIOs balance stability and innovation
CIOs recognize that AI isn’t a sprint; it’s a sustained journey. It’s critical to balance the stability that version pinning and guaranteed transition windows provide with the benefits of more advanced models.
“You don’t design for a five-year stability window. You design for maximum performance with the ability to upgrade as it makes sense,” Bhave said.
Organizations that get the equation right, Bhave argues, sidestep constant validation and integration headaches. They’re able to make changes rapidly and unlock greater innovation and ROI.
“This new world is a real opportunity for organizations that have historically struggled with technology to leapfrog into the lead,” he concluded.
Are you dealing with AI model churn in your organization? How are you balancing stability with access to the latest innovations? Email [email protected] and let us know.
Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.