Andrea Miotti is the founder and CEO of ControlAI, a non-profit dedicated to mitigating the risks from powerful AI systems. Working in the US, UK, Canada, and Germany, ControlAI calls for prohibiting the development of superintelligent AI, as AI experts assess it poses an extinction risk. Andrea’s published work on the extinction risk of superintelligent AI includes The Compendium, which explains the extinction risks from AI, and A Narrow Path, covering proposals intended for action by policymakers. His expert commentary and op-eds have appeared in outlets including TIME, The Guardian, BBC, and more.
Where do you see the most exciting research/debates happening in your field?
I wouldn’t exactly call it exciting, rather urgent, but the biggest policy debate in my field is around the AI industry’s race to develop superintelligence. Top AI scientists, Nobel Prize winners, and even the CEOs of leading AI companies themselves warn that superintelligence poses an extinction risk to humanity, yet the race to develop it continues. My organization, ControlAI, is advocating for a prohibition on superintelligence. We work to inform governments and the public about this issue. Most people don’t know that AI companies’ end goal is not chatbots but superintelligent AI. If they don’t know about the threat, they cannot act.
In January 2026, ControlAI supported two debates in the UK House of Lords in which members discussed this extinction threat and the pursuit of an international moratorium on superintelligent AI. Over 100 cross-party UK parliamentarians back our campaign recognizing the extinction risk from AI and identifying superintelligence as a global and national security threat. In Canada, we have briefed more than 100 lawmakers in less than a year about the risk from superintelligence, and we just launched our campaign with over 30 MPs and Senators calling for an international agreement prohibiting superintelligence. There have now been several rounds of hearings in the Canadian House of Commons and Senate covering the risks from AI. I was honoured to give expert testimony before the Canadian House of Commons, warning about the extinction risk posed by superintelligence.
How has the way you understand the world changed over time, and what (or who) prompted the most significant shifts in your thinking?
My worldview shifted significantly in 2019, while I was researching how government industrial policy shapes technological innovation, and the role of general purpose technologies across human history. That’s when I realized AI would be the next general purpose technology, and that sufficiently advanced AI would be capable of automating all intellectual labor humans can do. While GPT-2 had just been released that year, early examples of AI systems were already being used to automate discovery of new chemical compounds. Extrapolating that trend, I saw that once AIs surpass humans in key domains, they would accelerate AI development itself in a feedback loop that, by default, spirals outside human control. When I realized that, it became obvious to me that superintelligent AI would pose an extinction risk for humanity. Once I understood the enormity of the threat, I was startled to learn that only a very small group of people, averse to engaging governments and the public, were working on this major issue. Governments and lawmakers cannot address the threat from superintelligence unless they know what’s being developed, and what the risks are. Since then, I have committed myself to solving this problem. This is why I founded ControlAI: to help the public and lawmakers understand this threat, and help them act to keep humanity in control.
Can you explain what is meant by the term “AI superintelligence”? Are there different perspectives and debates around the understanding of this terminology?
Superintelligent AI is AI that vastly exceeds human capability across most cognitive tasks and can act beyond human oversight and control. It would be capable of outmanoeuvring and overpowering human institutions, including our national security apparatuses, which makes its development a major national and global security threat. This is why leading figures in the AI field warn it poses an extinction risk to humanity on par with nuclear war.
There are active debates around how to define superintelligence in legislation. ControlAI’s draft UK Artificial Superintelligence Security Bill defines superintelligent AI as “an AI system which can cause serious damage to the security of the United Kingdom because of its capabilities to neutralise, displace, circumvent, subvert, or render ineffective relevant human authorities in the exercise of their functions.” In the Artificial Intelligence Risk Evaluation Act, US Senators Josh Hawley and Richard Blumenthal define superintelligence as an AI system that can operate independently for extended periods, matching or surpassing human thinking and decision-making across most domains. Crucially, they add that it has the capability to “circumvent human control or oversight”, posing unprecedented threats to humanity.
Can you explain the line of reasoning for why superintelligence poses an existential threat to humanity (X-risk)?
No one, not even the engineers developing cutting-edge AI systems, fundamentally understands how these systems work. The CEO of Anthropic has said we maybe understand 3% of what’s going on inside them. This is because AI systems today are ‘grown’ from huge amounts of training data and computational power, rather than written line by line like traditional software. The result is AI systems that are neither predictable nor controllable, and this poses massive security risks as the race to superintelligence produces ever more powerful and autonomous AIs.
AI companies admit that no one knows how to control superintelligence, which is why leading scientists warn it poses an extinction risk to humanity. Even without malicious intent, superintelligent AI could wipe out humanity as collateral damage while pursuing its own goals, just as an anthill is collateral damage for contractors building a dam.
Are more recent shifts in the international order a contributing factor to the X-Risk?
We are facing an ongoing race between a handful of large tech companies to develop superintelligence. Geopolitical tensions certainly make international coordination harder, but they don’t change the underlying problem. Even if we had the most peaceful geopolitical period in history, the fundamental security threat posed by superintelligent AI would still be there as long as anyone were working to develop it.
In any geopolitical climate, the solution remains the same: an international agreement prohibiting superintelligence development globally. It is in no country’s interest for any actor to develop superintelligent AI, AI that would be capable of autonomously undermining the national security of any country. The main bottleneck is that most governments still have not heard about and reckoned with this risk. ControlAI is addressing that bottleneck, informing policymakers and the public about the threat from superintelligence and what they can do to address it.
Academic and industry figures have taken issue with the X-risk thesis, suggesting that the “take-off” dynamics are over-exaggerated. What is your view regarding this criticism?
We need to look at the exponential trajectory we are on. The 2025 METR Graph measuring AI’s ability to complete long tasks demonstrates the exponential growth in AI’s capabilities, and the leading AI companies are working to fully automate AI development itself through “recursive self-improvement”: AIs developing other AIs at machine speed with less and less human involvement. Once AIs can improve themselves autonomously, the feedback loop becomes extremely fast, putting us on an accelerated path to superintelligence.
Leading AI scientists, including ‘godfathers of AI’ Geoffrey Hinton and Yoshua Bengio, are on the record confirming the extinction risk from AI. Most worryingly, even the CEOs of the major AI companies have repeatedly acknowledged that the technology they’re building could lead to human extinction. This has been an open secret in the AI field for over a decade: it finally reached the public in 2023 with the Center for AI Safety’s Statement on AI Risk, but much more needs to be done to spread awareness of this.
What can national governments do to mitigate potential AI risks, especially for countries which do not host leading AI models?
Countries can take a stand by prohibiting the development of superintelligence on their soil, and championing an international agreement that bans it globally. This is a problem that requires a coalition of countries working together to prevent the threat. Countries that recognize the national security implications of superintelligence should be discussing an international prohibition with allies and partners at the highest level.
What does a global coordinated governance model look like and what immediate actions are available to policymakers?
When it comes to enforcing a prohibition on superintelligence, the specific instrument is secondary: what matters is a coalition of countries that treats the development of superintelligent AI the same way they treat nuclear proliferation, and works together to deter it and prevent it from happening. What governments can do right now is to publicly recognize superintelligence as a national and global security threat which poses an extinction risk to humanity. They should begin closely monitoring the development of superintelligence and start negotiating an international agreement to prohibit its development globally. ControlAI is advocating for a lean agreement among states that prevents the development of superintelligence internationally.
What is the most important advice you could give to early career scholars researching AI?
Speak truth to power. Until recently, most AI researchers were unwilling to speak publicly about the extinction risk from superintelligence: they would claim to be very concerned privately to others in the field, but avoid speaking clearly to the public and policymakers. I’m a strong believer in public debate. Early-career researchers should be willing to name hard, uncomfortable truths and to engage actively with the democratic process when it comes to risks posed by their field.
Further Reading on E-International Relations
