Why observability has become essential to the CIO’s job

Observability, once largely a DevOps tool for understanding and troubleshooting applications and infrastructure, has become a CIO management tool as technology stacks grow more distributed and interconnected.

Observability’s value now extends beyond understanding how systems are working. It can reveal what technology is being used, how well it is performing and whether the investment is paying off. AI has added another layer of complexity to observability, as companies deploy new applications and agents across those technology environments.

Fernando Castanheira has firsthand experience with the expanded use of observability. As CIO at Riverbed Technology, an observability vendor, he uses the technology inside his own IT organization and works with the company’s customer-facing teams.

“What observability really does is provide an operational truth across networks, applications, cloud, endpoints and that end digital experience,” Castanheira said. “You need it to connect technology performance to employee productivity.”

Castanheira and Eric Johnson, CIO at PagerDuty, spoke to InformationWeek about how they use observability, how it has changed over the course of their careers and how AI changes the way CIOs anticipate and react to problems.

Observability helps cut through the noise

Critical business services generate a flood of performance data. In that sea of information, what matters, and what doesn’t?

“It’s a battle that I’ve had my entire career,” Johnson said. “How do you continually find ways to reduce the noise and find what’s most important, and at the same time not miss anything?”

Observability helps make sense of that noise, and it has gotten better over time, Johnson said. Looking back, he recalled how difficult it used to be to sift through the information delivered by observability tools. “I remember years and years ago, you would have tools that would just throw you a warning if something came out of a tolerance, but you got lots of false positives,” he said. “It was always hard to figure out where the real problem was or where the real opportunity was.”

Within the past several years, more advanced machine learning capabilities have improved observability. More recently, AI has made it even better. It excels at sifting through large data sets, identifying trends and automating tasks.

“AI helps correlate and contextualize telemetry,” Castanheira said. “It consolidates alerts and separates meaningful signals from noise.”

More technology, same observability fundamentals

Even as enterprise technology environments grow more complex, the goals of observability and the core metrics have not changed significantly.

“The way I’m measuring it still gets back to identifying a problem, resolving the problem quickly and — if you can actually avoid the problem — where there’s preventive things,” Johnson explained.

Observability tools collect telemetry data across networks, applications and devices. That data tells a story: what normal looks like and what indicates that an incident needs to be triggered. Core metrics revolve around errors, latency, CPU and memory usage.

At Riverbed, Castanheira has used observability to reduce software spending. “We had X [number] of licenses; we reduced those licenses by 40%,” he said. “The way we did that is by using our own tool to measure actual usage.”

On the hardware side, observability can inform asset refresh cycles. If metrics on laptop performance and memory indicate that a device meets an employee’s needs, the refresh cycle could be extended beyond three years, according to Castanheira.

“You can imagine the impact on the P&L when you have 100,000 of these assets on the books,” he said.

AI is making observability both easier and more complex

Infusing AI into observability platforms makes it easier to identify patterns and even automate responses, but AI applications and agents also add new systems and activities to monitor, Johnson noted.

“It’s yet another critical business service that we need to make sure is running the way it’s supposed to run,” he said.

AI agents present another observability challenge because their actions can be difficult to predict and trace.

“Now, you’re going to have all these agents, millions of them, potentially creating havoc in the environment. You need to measure what those agents are doing,” Castanheira said.

Understanding how AI is behaving begins with the data that fuels it. Johnson and his team are using observability tools that provide telemetry on data quality. “We use a lot of tooling around observability for data and then triggering an incident if data is not looking the way it’s supposed to,” he explained.

Go-to-market teams at PagerDuty were using agentic solutions, and through observability, Johnson was able to see issues with errors and latency. “Through our observability, we could see that and very quickly take corrective action to optimize the way MCP [model context protocol] actually works, update the data source,” he said.

Eventually, AI may be able to identify and fix incidents surfaced through observability without human intervention, but we aren’t quite there yet, Johnson said, cautioning against removing the human in the loop. “I’m finding as a practitioner with AI that there’s a lot of nuances, and sometimes AI doesn’t always get it just right,” he said.

The future of observability is more proactive than reactive

Historically, observability has helped IT teams respond to problems by identifying what isn’t working and why. AI could push observability toward identifying potential problems before they occur.

“You may be about to have a problem and maybe trigger some activity or automation to avoid the problem,” Johnson said. “That’s the Holy Grail.”

He said he envisions AI agents acting as his “observability chief of staff.”

“They’re going to be running and managing and operating and making sure that every morning I come in, I’ll get my summary of all the proactive work that was done the night before,” he explained.

Castanheira also said he expects AI agents to reduce the need to dig through multiple observability dashboards. Instead, they could provide context on what requires attention and recommend specific actions to address those issues.

IT infrastructure is only going to get more complicated as enterprises continue to layer AI on top of legacy, SaaS and mobile technology. That complexity will make maintaining visibility across the technology environment more difficult.

“You’re going to have this very sophisticated mapping of your infrastructure, your services running, maybe even outcomes you expect, all the way down to each agent on what they’re supposed to be doing,” Johnson said. “If this doesn’t happen, I don’t know how we continue to operate and manage all this infrastructure.”