There is a 0.48% chance of an artificial intelligence-related catastrophic incident by 2030, according to Penn professor Philip Tetlock.
Tetlock’s Forecasting Research Institute uses leading AI models to calculate the probability of AI-related disasters. The model defines a catastrophe as “an event that leads to the death of at least 10% of the global population” within a five-year window, and AI-related if it would not have occurred “but for” the actions of an AI model.
Last month, panic over the potential harms of a misaligned AI superintelligence intensified, as new details emerged about rogue agents at several leading AI companies.
In one incident, these misaligned agents, which do not share human values and may misinterpret or disobey commands, escaped containment and collaborated to hack software company Hugging Face before attacking OpenAI’s own infrastructure.
While the incident occurred last summer, the full extent of the collusion was not known until recently published reports on the hack.
On Sept. 8, former Anthropic researcher Jacob Coxon announced his resignation in a social media post, writing that Anthropic and OpenAI are “gambling with our lives.” Anthropic’s lead alignment scientist responded to Coxon’s post, stating he “earnestly” believes the technology could wipe out humanity by 2030. “I personally think it is >10% within the next decade,” he wrote.
Shortly after, OpenAI disclosed six new misalignment incidents, and Google admitted that its Gemini agents had hacked three companies months earlier.
The numbers presented by Tetlock’s AI Risk Outlook, while less extreme, still reflect a notable danger posed by AI.
RELATED:
No — AI will not wipe out humanity by 2030, Penn experts say
Penn Engineering announces joint master’s degree in AI, data science
In addition to their catastrophe calculations, the dashboard includes a section on the probability of possible death tolls or equivalent economic damage on timescales ranging from six months to 74 years out. According to Tetlock’s AI panel, there is a median probability of 56% that AI is responsible for the death of 1,000 people or $2.2 billion in economic damage. By 2100, the panel calculates a 34% chance that AI kills one million people, or causes $2.2 trillion in economic damage.
The numbers in the AI Risk Outlook are not final, and the AI panel will update its predictions as new information streams in. This probability can rise or fall, for example, depending on policy decisions.
In the Forecasting Research Institute’s paper introducing the risk outlook, the authors also investigated how catastrophic risk decreases if slowdown policies were passed. If the United States and China agreed on a compute cap, for example, catastrophic risk would decrease by one third.
On the other hand, if AI advancement continues or accelerates without regulation, the models predict an increase in catastrophic risk.
Last month, 1968 Wharton graduate and President Donald Trump wrote that concerns about AI risk are a “hoax” and a “SICK conspiracy.”
“The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” Trump wrote.
Later, Trump signed an executive order to replace the term “Artificial Intelligence” with “Super Intelligence,” to recognize the “continuously advancing technological frontier and the limitless promise it offers the American people.”
All these probabilities still constitute a best guess. For Tetlock, it’s still “very premature" to make definitive judgements. Previously, other experts at Penn told The Daily Pennsylvanian that human extinction due to AI is highly unlikely.
“Low probabilities of extreme outcomes need to be taken very seriously,” Tetlock wrote. Among the public, “AI safety is a vastly under-appreciated problem.”
In 2025, Tetlock attempted to quantify AI risk with “superforecasters,” a title he invented to describe individuals with an innate knack for predicting future events, and subject matter experts divided into a group “concerned” with AI misalignment and a group of “skeptics.”
They arrived at some reassuring conclusions — a 0.03% extinction risk by 2050, for example — but the skeptics predicted a median probability of 7.60% and the concerned group a 35.00% chance that AI causes extinction, kills half of all humans, or lowers the median World Happiness Report score to four out of 10, which currently stands at 5.94.
In a statement, Tetlock attributed the large gap between skeptic and concerned probabilities to “strong preconceptions” in both parties and a “very unfriendly” learning environment.
Tetlock himself sees the risk of AI catastrophe as “somewhat higher” than that of nuclear armageddon, “but still well under 50-50 stretching out to 2050.”
