Why companies are shifting toward private AI models
For technology leaders, the key issue is not whether private AI is โbetterโ, but which work should be controlled closely enough to justify the extra cost, governance and operating discipline. That reframes AI from a model-selection exercise into a portfolio choice: public models for experimentation and scale, private models for differentiated, data-intensive or regulated use cases.
The management implication is that AI roadmaps need explicit decision rules. CIOs should define where sensitive data can flow, which workflows require tighter control, and what level of model ownership is warranted by customer impact, compliance exposure or margin potential. Without that clarity, organisations risk funding private AI as a status symbol rather than a value engine.
This also changes how success is measured. Private models bring additional overhead in data readiness, lineage, observability, vendor abstraction and ongoing tuning. Leaders should ask whether they can track token cost, service cost, model performance and business outcome together. If those metrics are not visible, it becomes difficult to know whether a private deployment is improving decision quality or simply shifting spend around.
A practical next question for boards and executive teams is: what is our operating model for hybrid AI? That includes who approves model use cases, who owns the data layer, how fast the organisation can swap providers, and which skills must be retained in-house. The real trade-off is not speed versus control; it is where control creates durable business advantage and where it only adds friction.
Private AI models will drive 70% of revenue created by AI within five years, according to a recent blog post from Forrester CEO George Colony.
“The revenue will definitely show up in private models because that’s where enterprises will need to differentiate and monetize their own data, not because private models are inherently better,” Ha Hoang, CIO of Commvault, a cyber resilience company, told InformationWeek.
Five years is a long time in the fast-paced world of AI โ new models, new capabilities, new predictions spring forth every day. Getting to a future where private models drive a significant chunk of revenue is going to require CIOs to understand the roles public and private models play in their organizations and adjust their AI roadmaps accordingly.
Why enterprises are building private AI models
The big, public AI models are at the frontier of the industry for a reason. “We lean into public models for what they’re really good at, which is speed, innovation and access to cutting-edge capability that allows us to experiment, to move fast and really bring new experiences to our users and to our customers,” Hoang said. But public models are not the answer to leveraging enterprises’ valuable proprietary data. Companies cannot risk exposing sensitive internal information. Instead, they can lean into private models to differentiate and create unique value with that internal data. Hoang said at Commvault, the question animating the company’s AI plans is, “How do we deliver new features and functionality โฆ through our product to our customer using our private model with our proprietary data, our workflows?” Colony underscored that mindset in his blog, arguing that customers should be the goal of AI innovation. “The real AI game will be winning, serving and retaining customers. And that will be the sweet spot of the private-model business model,” he wrote.How hybrid AI architectures are changing enterprise IT
Playing that game is going to require a significant shift. Enterprises will need to consider where their AI strategies and dollars are going. How much money has been spent on public models? Where should they be building and investing in private models? “What you’re really seeing isn’t a shift away from public models, per se, but a shift toward capturing that last mile where private models tend to sit closer to โฆ the proprietary data, the workflow and the outcome,” Hoang said. Enterprises will still make use of public models to reach that last mile. They will build their private models on top of public foundations. “We use public models today, and we use RAG to feed that with fresh data that can help differentiate some of the decisions that we’re making. And we’ve started doing some fine-tuning of the models around our agentic deployment,” said Shannon Bell, executive vice president, chief digital officer and CIO of OpenText. As more enterprises focus on using techniques like retrieval-augmented generation (RAG) to build their private models, this could shift the role the public models play in the AI ecosystem. “I do think that the public models will shift upstream and become more like foundational infrastructure,” Hoang said. “The competition there is really about who has the best base intelligence at the lowest cost.” This shift toward private and hybrid AI strategies is also changing decisions about where models should run and how. As a part of this transition, CIOs will also be thinking about where to run AI models: locally or on the cloud. “We’re already seeing a shift to moving some of that load, especially for the easier tasks that really can be run locally, off the cloud. That gives them more bandwidth to focus on the higher-value tasks, the ones that require more complexity to run,” said Sebastien Jean, CTO of Phison US. CIOs, CTOs and other technology executives are hard-pressed to keep up with these changes and guide their organizations toward the still-undetermined future of enterprise AI. Michael Facemire, CTO of Forrester, described spending his days focused on making sure everything is up and running in his organization and dedicating his after-work hours to keeping abreast of the constant changes in AI. “It is a pace that I’ve not seen in my career,” he said. These leaders cannot realistically keep up this pace of change on their own. They need support to understand where AI is going and how they can keep their enterprises from falling behind, according to Jean. “You want a small pathfinding team that’s exploring various solutions that are available to see if it makes sense to move the organization in that way,” he said.What CIOs should consider before building private AI models
The future of enterprise AI isn’t a binary question of public versus private. It is more likely a hybrid approach defined by specific use cases. “We believe strongly that there will be an evolution of a hybrid agentic model and a hybrid cloud architecture, where there’s protected and private data sets,” OpenText’s Bell said. “And between the two of those, you have the ability to run agentic.” She added that she expects adoption of small language models to increase in the coming years, particularly in regulated industries. Cost and energy constraints will also define the future of enterprise AI, Facemire noted. “We live in a cost-constrained world. We can’t just have big LLMs that do everything for everybody,” he said. As compute constraints continue, Facemire anticipates that large public models will increasingly lean into the use cases that benefit their bottom lines. CIOs will be challenged to determine what that means for their own strategies. “They’ll have to understand the individual models and what they were tuned to do best,” and then figure which workload they’re going to give it, he said.What CIOs should consider before adopting private AI
What does all of this mean for CIOs who are building out their AI roadmaps today? “The biggest mistake right now is treating private and public as a technology decision because I really don’t think that it is. It’s a value and operating model decision,” Hoang said. CIOs debating how to leverage public and private models need to think about:-
Data readiness and governance. When AI projects stall and fail to scale, it often comes down to the data. However CIOs want to leverage public and private models, they need a strong foundation of data readiness and governance. “[Strengthen] the data layer, the data lineage, metadata data governance and so on, and [ensure] that you have flexibility around your workloads, your data sets, so that as the market continues to evolve, you’re in control of your AI strategy,” Bell said.
-
The right use cases. Not every use case will necessitate feeding sensitive data to a private model. CIOs will have to decide when public models are sufficient and when private models justify the cost. “Private models can certainly require more investment, more governance and more discipline,” Bell said. “It’s important to note that this is why they should be used where there’s clear business value and control requirements that justify them.”
-
Vendor flexibility. CIOs want to live in a world where they can take advantage of emerging capabilities in AI, and that means building flexibility into their strategies. “[Put] an abstraction layer in place so that you’re not locked into one vendor or forced to rebuild everything when the landscape changes,” Hoang said.
-
The talent pipeline. As enterprises rapidly put AI models to work, they risk the accumulation of tech debt. Without the human talent to grapple with that tech debt, enterprises may find themselves facing major bugs and security issues, according to Jean. “People that are too quick to fire their staff or not hire junior staff to replace the senior people that will retire will end up in a situation where they can’t get the help that they need at a price that they would like to pay,” he said.
-
Costs and outcomes. CIOs are already under pressure to prove AI delivers measurable value. Going forward, they need a way to track token costs, management overhead and actual business outcomes. Without stronger observability into costs and performance, CIOs may struggle to determine whether private AI deployments are actually delivering value, Facemire warned. “Make sure that you are not upside down in a service delivery model,” Facemire said. “If you wait too long and you don’t build an observability layer in your own private models today, you might find that out and find it out far too late.”
Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.