Google retools Kubernetes for AI agents instead of building a new stack

Google retools Kubernetes for AI agents instead of building a new stack
The most important architectural signal here is not simply that Google wants Kubernetes to host AI agents, but that agent platforms may force a shift from “one general-purpose cluster fits all” toward workload-shaped clusters with tighter runtime assumptions. For platform teams, that raises a design question: should agent execution live in the same Kubernetes estate as conventional services, or in separately tuned clusters with different scheduling, autoscaling, security and API-rate expectations? The answer will affect tenancy models, cost controls and operational blast radius.

The security implications are equally significant. If agent-generated code is untrusted by default, sandboxing cannot be treated as a feature bolted onto a standard container platform. Teams need to think through isolation boundaries, egress controls, credential handling, artifact provenance and session lifecycle management before exposing agents to internal systems. In practice, the challenge is less about running an agent pod and more about safely mediating what that pod can generate, execute, call and retain.

There is also a control-plane economics issue beneath the article’s performance discussion. High pod churn and long idle periods suggest that naive per-session provisioning will punish both cluster efficiency and control-plane stability. IT operators evaluating agent runtimes should therefore look beyond raw node capacity and test API-server pressure, startup latency, sandbox reuse strategies, queueing models and observability overhead under bursty agent workloads.

If this model succeeds, Kubernetes remains the common substrate, but the operational contract changes: platform engineering becomes less about standardizing one cluster pattern and more about offering specialized execution tiers for different classes of AI-driven work.


 

 

Agentic workloads are pushing Kubernetes into territory its creators never planned for. Agents generate untrusted code on the fly, sit idle for long stretches and churn through pods at a pace that strains the orchestrator’s control plane.

Cloud providers are now rethinking how general-purpose a cluster should be. Google LLC’s answer is to stop treating every cluster as an infinitely flexible engine and tune each one into a runtime optimized for its workload, according to Jago Macleod (pictured, right), director of engineering for Kubernetes and Google Kubernetes Engine at Google Cloud.

“There isn’t enough memory. There aren’t enough machines. So we have to get more efficient. And so one of the ways that we do that is to focus the Kubernetes cluster on the workload it’s running, and optimize it for that specific workload,” Macleod said.

Macleod and Jaana Dogan (pictured, left), principal engineer at Google Cloud, spoke with theCUBE’s Rebecca Knight at KubeCon + CloudNativeCon NA, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how Google is customizing Kubernetes for agentic workloads without fragmenting the open-source ecosystem. (* Disclosure below.)

Agentic workloads strain the Kubernetes control plane

Kubernetes was built for microservices, then stretched to handle training jobs in which a single pod claims an entire node, Macleod explained. Agents are a different beast again, and some teams have resorted to building custom control planes just to cope.

“I think the agentic workloads are so different in the thrash and the churn that they represent is really not what we built Kubernetes for at all,” Macleod said. “And then agentic applications are almost the polar opposite, which is, you know, in the naive implementation, you just have as many pods as you can possibly fit, and so then the API machinery is running into limits.”

Platform teams have spent a decade making Kubernetes their compute fabric, and customers told Google they did not want agents to fragment it, Dogan explained. Google’s response is Agent Substrate, which runs AI agents in sandboxes directly on Kubernetes. The company donated the project to the Cloud Native Computing Foundation and recently released version 1.0. Because agents write their own code, that code is untrusted by default and needs sandboxing and network guardrails, according to Dogan. Customers also flagged a costly inefficiency.

“For example, our customers came to us, ‘Hey, majority of the time our agents are idle, but we have to allocate a sandbox all through the lifetime of a session,’” Dogan said. That means wasting a lot of CPU cycles, she explained.

Coding agents are also changing how engineers work, producing more prototypes and smaller, more focused teams, Dogan noted. As agentic workloads multiply, human judgment grows more valuable rather than less, according to Macleod.

“It’s not really, you know, fire off a prompt and forget it and walk away,” Macleod said. “It’s just a different level in the system where you’re applying your judgment and expertise. And that has never been more important.”

(* Disclosure: Google Cloud sponsored this segment of theCUBE. Neither Google Cloud nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

 

Google retools Kubernetes for AI agents instead of building a new stack

Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.

Leave a Reply