AI governance lags behind deployment. CIOs need to fix that

The era of CIOs deploying AI is over. Today, AI capabilities arrive in rapid succession. Some come from frontier and open-weight model providers; others are baked into enterprise software tools and platforms.

This pervasiveness can change how people and systems work, and introduce testing, validation and security challenges. It paves the way for shadow AI and introduces a greater risk of agents autonomously performing undesirable actions such as altering code or leaking intellectual property.

“CIOs may find employees using tools that sit outside the organization’s formal governance process, both intentionally and without sufficient awareness,” said Jennifer Kosar, AI assurance leader at PwC U.S. “A complete view of AI use is much harder than keeping a list of projects you’ve approved.”

Managing this new frontier of AI is nothing short of daunting. Part of the hardship for CIOs is the pace of change, which is shifting the enterprise-vendor relationship. Rather than waiting for enterprise applications to catch up, CIOs need to act now, according to Lauren Kornutick, senior director analyst for analytics and AI at Gartner.

“Organizations are deploying AI faster than the ability to govern it,” she said.

While they can’t turn back the clock, CIOs can lay the groundwork for a best-practice approach that focuses on continuous discovery, a control plane, evaluation loops and vendor conversations that prioritize risk over features.

When AI bypasses IT

AI manageability is won or lost in the gap between rapid AI updates and how quickly an enterprise governance system can spot an issue. Yet, the gap is fraught with digital landmines. Part of the problem is that enterprise software vendors are increasingly bundling AI suites into products.

While it might be possible to switch tools on and off, it is next to impossible to determine how a chatbot or AI agent will interact with other AI systems. A routine update could change what data the system sees and what it can do. Neil Ward-Dutton, a research vice president at IDC, argued that while SaaS vendors provide release notes, upgrade paths and deferral options, the documentation isn’t perfect.

Embedded AI, however, is the lesser of two worries compared with ease of access. Employees running assistants and departments installing AI tools and agents might fall into the authorized category. Yet, CIOs and security teams might not have had an opportunity to determine whether the AI plays nice with other systems and agents — or whether a change in a frontier model or the software triggers undesirable behavior.

And then there’s shadow AI, where employees are using AI tools without IT’s knowledge. All told, 69% of organizations suspect or have evidence that employees are using prohibited public generative AI, according to a Gartner survey of more than 300 cybersecurity leaders. The consultancy predicts that by 2030 more than 40% of enterprises will suffer security or compliance events linked to unauthorized shadow AI.

“Shadow AI, on its own, is not necessarily a problem,” Kornutick said. The danger doesn’t lie so much in the tool as in what it can reach. An improperly configured system can inadvertently sweep up private data or intellectual property — and generate inaccurate, biased or incorrect results.

Personal agentic systems raise the stakes further. Environments such as Claude Cowork, Gemini Spark, Microsoft Scout and ChatGPT’s agent mode push past the boundaries of conventional chatbots. Enterprise development platforms place limits on agents and enforce identity, access keys and tool permissions through agent and model gateways. The concern grows when a business user with no engineering expertise uses a personal account.

Within this scenario, an agent with no tool for the job might simply write its own, according to Ward-Dutton. He said CIOs should keep consumer-centric agentic tools locked down in a tightly controlled sandbox — something resembling a micro VM.

Gaining AI visibility

To combat growing AI complexity in the enterprise, building a governance framework that delivers visibility into vendor updates and changes is critical.

“A potential blind spot is assuming that because the underlying system has already been through your risk and security processes, a new AI capability is covered by that same assessment,” Kosar said. “If your technology or related risk and governance functions have not evolved to address novel AI risks, that may not be the case.”

CIOs must know what a system is doing, which data tools and agents can access it, and what decisions or actions it can take. They also need visibility into how it interacts with other systems it touches.

“You may not be implementing a new system, but you may still be introducing a new capability that needs to be evaluated differently,” Kosar said. “Data risk is tied to both what the model has access to and how its outputs are subsequently used.”

The starting point is continuous discovery. Kosar advised clients to assemble a complete list of internally approved AI tools and applications, along with the capabilities that already reside inside major software platforms. It pays to establish a dialog with vendors — and to the extent possible, map out updates.

“An AI inventory can’t just be a survey you conduct once a year asking people what they’re using,” she said. “The technology — and use of it — is changing too quickly for that.”

A point-in-time audit won’t suffice; evaluation has to be continuous. Most major platforms — from the likes of Google, AWS, Microsoft and IBM — offer built-in tools that can provide ongoing oversight. Once an agent is running, these platforms track telemetry and use observability tools to watch agent behavior in the runtime environment, Ward-Dutton said. Along with user feedback, they can spot issues and establish an improvement loop.

Kornutick said she believes CIOs must rethink the fundamentals. Traditional protections work on binary rules; AI doesn’t. These systems need a dedicated infrastructure layer — a control plane — that governs runtime security, tool authorization, traffic routing and prompt-level filtering. The last piece is a kill switch.

“You want to be able to block actions from the control plane,” she said.

Vendors are another consideration. It’s wise to present software suppliers with a defined set of risks and ask how their tools actively manage these issues — and how their products interact with other vendors, Ward-Dutton said. The responses can prove illuminating. “Some vendors do this really well; others don’t have much to say,” he added.

Finally, there’s accountability. When an unapproved feature triggers a compliance issue, who answers for it remains unsettled — though AI clearly complicates matters. Responsibility, might land on both the vendor and the customer, Kosar said. “This is a yet-to-be-defined space,” he explained. “But if history is an indicator, there could be shared responsibility.”

AI governance can’t wait

The end goal isn’t to slow innovation or create burdensome restrictions, Kosar said. AI governance works best as a management layer that abstracts controls from specific models and apps. Then, when a vendor introduces changes or a frontier provider serves up a new model, nothing winds up broken.

In the end, timing matters as much as technology and strategy. Governance can’t wait until an organization scales agents across the enterprise — and beyond. By then, governance decisions are already set. It’s essential to treat governance as part of the underlying AI strategy, Ward-Dutton said. Without it, “all you have is words on a bit of paper.”