When AI designs a virus
The key issue here is not that AI is already creating pandemics, but that it can lower the barrier to asking dangerous biological questions. That distinction matters: a chatbotโs ability to generate plausible guidance is not the same as a verified, end-to-end capability to design and build a viable pathogen. Still, even partial assistance can be important in security terms because it may accelerate experimentation, widen access to harmful knowledge, or help less-skilled actors move further than they otherwise could.
This is a classic case where association can be mistaken for capability. Reports and demonstrations may reveal what a model can discuss, but they do not prove that a threat is immediately operational. Biological misuse requires materials, tacit know-how, equipment, and iteration, all of which impose real friction. That uncertainty cuts both ways: it should prevent panic, but it should not be used to dismiss early warning signs.
The practical takeaway is about preparedness rather than prediction. If AI tools keep improving, the relevant question becomes how to spot misuse sooner, limit high-risk outputs, and create response systems that can adapt faster than the models themselves. In other words, โself-policingโ alone is unlikely to be enough for a technology stack where capability can diffuse quickly and oversight often arrives after public concern has already spread.
For readers, the broader lesson is a familiar one in health security: prevention works best when it is layered. Technical safeguards, governance, independent testing, and clear escalation paths are more credible than hoping development will slow down on its own. The uncertainty is real, but so is the need to build systems that can cope before a rare risk becomes a practical one.
A stunt that woke up Washington
It was early 2023 when I heard the story of the little black box. It had been brought into the historic Eisenhower Executive Office Building, which sits alongside the West Wing. Inside were a dozen test tubes that held the ingredients for mass casualty โ engineered DNA that, if assembled by the right pair of hands and with the proper equipment, had the potential to cause the next pandemic.
The box was a kind of stunt. The man behind it, Rocco Casagrande, hadnโt come up with the formula for destruction himself. An AI chatbot had described the steps that needed to be taken to build such a weapon. And Casagrande was determined to show Biden administration officials the kinds of risks that could emerge when two quickly evolving fields โย artificial intelligence and synthetic biology โ joined forces.
This stunt had a huge impact. Word of the meeting spread, like a game of telephone, across the nationโs security apparatus. It was a wake-up call, revealing the US government was seriously unprepared, sources told me.
Itโs not that todayโs AI has the capability to design the stuff of sci-fi horror. But the models areย always improving, which raises the specter of what those tools might be able to do in the future.
A January report from the RAND Corporation found that while the current generation of AI chatbots donโt pose an especially great threat, that may not be the case for long. The field is โadvancing faster than governments can keep pace,โ the lead author of the report said.
Anna Marie Wagner, who formerly served as Ginkgo Bioworks’ head of AI, acknowledges that the world was caught โflat-footedโ when ChatGPT arrived. โWe need to prepare for the risks that emerge,โ she said. โIโm concerned that the response could look like, letโs slow down, rather than letโs build robust infrastructure.โ
She added itโs naรฏve to think that the pace of innovation in AI or synthetic biology will slowย anytime soon โ even in the face of regulation.ย โThat train has left the station. As a global society, we need to determine how to build the tools to identify risks and threats to respond quickly.โ
Thatโs exactly why the AI startup Anthropic had hired Casagrande to see what their chatbot Claude was capable of at the outset of 2023. The companyย had asked Casagrande to look beneath the hood and see if it could aid a bioterrorist by providing nefarious information. And, well, it could. With Anthropic’s blessing, Casagrande briefed US government officials on the threat.
It took me months to confirm details about these briefings. (You can give my storyย a read or listen to theย Big Take podcastย about it.) Casagrande was measured in the way he spoke about the risk ofย AI-enabled bioweapons. He wasnโt losing sleep over it, but he was worried about the potential for the tools to get even better. Additional checks and balances needed to be put in place.
โSelf-policing and self-reporting are not sufficient,โ he told me.ย โRiley Griffin
https://feedly.com/i/entry/1SZhfmCNEhuaubzP74DjBH5Rrd0hcP7fl3KrY0UwT80=_191272646ea:259ce4:cdf11c9cEnjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.