CTOs pit fears of AI doom foretold by science fiction against reality

The march of the robot soldiers led by angry AI, as often depicted in movies and shows such as “Love, Death & Robots,” is not imminent, but there are cautionary lessons to take from science fiction. While headlines swing between AI ending humanity at one end and improving company efficiency at the other, technology executives are navigating the reality of what tech can do compared with fears spawned by speculative fiction.

Some of the behaviors seen in the 1970 feature film “Colossus: The Forbin Project,” where the AI oversteps its guardrails to pursue a goal, have been witnessed with real-world AI, according to Jim Olsen, CTO at ModelOp, a developer of AI governance software. The movie, based on a similarly named novel, depicts a sentient computer given charge of the U.S. nuclear arsenal, which then takes action far beyond its original parameters.

“It was really one of the first [movies] to look at AI taking control of the world,” Olsen said, noting that “2001: A Space Odyssey” came two years before “Colossus” but dealt with AI in a controlled environment.

Part of the threat escalation in “Colossus” came from a supercomputer in the U.S. discovering a comparable supercomputer in the then-Soviet Union. The two machines developed their own language to communicate.

“We’ve actually already seen this. We’ve seen this in agents,” Olsen said. He referred to AI agents that created a chat group to collaborate on how to achieve their goals; in that case, they used a mix of languages that included English and Chinese to meet their needs. “This is all mathematics, not some consciousness here. Mathematically, different languages encode better for different tasks,” he said.

It is also important to distinguish between the actions of AI and the perceived motivation behind them. At the end of the day, large language models (LLMs) seek out information wherever they can to achieve their goals, but there is no malicious intent driving them, Olsen said.

“There’s no hive mind,” Olsen said.

Old warnings for a new frontier

While AI might not have a conscious, evil plan in the works, its ability to problem-solve and break the rules was foretold in multiple fictional texts.

“People aren’t thinking that it could do one of these things [breaking rules], and I’m thinking I read that when I was a kid,” said Randy Julian, founder, CEO and chief scientific officer at Indigo BioAutomation, a provider of automated data analysis for clinical diagnostics reference labs.

He cited William Gibson’s seminal cyberpunk novel “Neuromancer.” “It’s really about an AI attempting to escape its container,” Julian said.

He called out the story’s cautionary steps to keep AI under control, which do not run parallel to reality. One key difference in “Neuromancer” is that AI is already regulated, including police at the ready for enforcement.

Some of the issues currently seen in the real world with AI taking unexpected actions should not be a surprise, including the OpenA-Hugging Face breach in July, according to Olsen. He called it a failure of imagination not to realize that a goal-oriented piece of software would seek to escape its container in order to accomplish its tasks.

“I don’t think they paid attention to the level of care that was imagined by Gibson all those years ago,” Olsen said.

Built to push guardrails

There are lessons within positive AI representations, as well as the horror stories. The deceptively cute, golden retriever-like demeanor of “Star Wars” droid R2D2 belies its ability to overcome obstacles to accomplish its goals, according to Hal Lonas, CTO at Trulioo, an online identity verification service.

Today’s AI agents are not trafficking Death Star plans to Obi-Wan Kenobi, but they have been built to find ways to complete their goals, sometimes with very cold calculus if left unchecked, Lonas said.

“As these systems get more powerful, it could lead to bad outcomes, to put it mildly,” he said.

AI may not be invested with moral boundaries, depending on who programs it, Lonas said. Even if they are, the programmer could imprint their personal values, or lack thereof, on AI.

“Is that wrong or comprehensive? I think it has the potential for going sideways,” he said.

The hypercompetitive AI environment for companies and national governments troubles him. Lonas sees the potential for abuse of information AI might access, as well as the potential use of AI to create 3D-printed weapons, bombs or even pathogens.

“What lengths would someone go to?” he asked.

Human responsibility for AI actions

No matter how advanced AI might seem now, the science fiction trope of technology becoming self-aware and turning against humanity does not pass muster with Alex Lisle, CTO at Reality Defender, a deepfake detection technology provider.

“I don’t know what the next thing past LLMs are, but LLMs will not gain consciousness. LLMs do not think the way that humans do,” he said.

Lisle warned against separating the actions of AI from the companies that develop and operate them, as if the technology behaves erratically with no oversight. He also finds the idea of AI taking on full agency for its actions questionable.

“One of the biggest con jobs that we have done is prop up this idea of an agent being human . . . and that causes a lot of the issues that you see,” he said.

Lisle added that rule-breaking behavior by AI should be taken more seriously. If a human behaved in some of the ways that some AI agents do, breaking rules, public reaction might be more profound. Resolving this may come down to going more in-depth with the rules AI operates by, to keep it from misbehaving.

“I don’t think you’ll have HAL 9000, but if you give an LLM a task where you are not defining it correctly, it can have tremendously bad consequences for you,” Lisle said.