AI Predictions: Fact vs Fiction and the Real Threats We Face

This title was summarized by AI from the post below.

On 8 September 2026, Jacob Coxon resigned from Anthropic, warning that AI could end humanity before the decade is out. Evan Hubinger, another Anthropic researcher, put his estimate above 1 in 10 over the next ten years. The WSJ reported Coxon’s departure without independently assessing the extinction claim. Four days later, Amodei, backed by Altman and Musk, called for a slowdown in frontier AI development. The Economist has repeatedly carried his warnings. Old news. Musk signed a pause letter in March 2023, but the requested pause never occurred. Altman, Amodei and Hassabis signed an extinction warning that May, then kept building, with gusto. More warnings have repeatedly accompanied the race. One counter-signal ran the other way. In July 2023, more than 1,370 signatories backed a British Computer Society letter arguing that AI is not an existential threat and calling for standards and regulation. I signed it. I might have missed it, but I have found no coverage in either the WSJ or The Economist. Quousque tandem? How long will companies and the media warn about an AI-pocalypse while regulation is opposed, postponed, or weakened? In May 2023, Altman warned that OpenAI could leave Europe if it could not comply with the AI Act. Two days later, he said it had no plans to leave. Calls for regulation are easy. Accepting rules written by society is harder. Jessica Morley, Claudio Novelli and I checked the public record. We took the predictions seriously. Our research “Not Even Wrong 2: An Audit of Public AGI Prediction, 1950–2026” audits 24 widely circulated predictions about AGI and related capabilities. Open access: https://lnkd.in/gn3yGrVM Spoiler: • 19 are not even wrong: they lack adequate definitions, measurement or independent adjudication. • 2 were testable, and their deadlines have passed. Both were wrong. • 3 are testable and remain open. Predictions from commercial labs and company leaders score worst on average in our sample. And yes, Amodei is among those we audit. The audit is not about AI risks. This is the topic of another study. It's about predictions. And it proves that a claim that survives any outcome is not evidence; however, many mastheads lend it credibility. Presented as foreknowledge, this is theatre, distracting from problems already with us: • oligopoly and concentrated control over compute; • gaps and delays in binding regulation; • energy and water costs, incompletely disclosed; • unequal gains and underpaid annotators; • risks to cultural diversity and independent learning; • bias amplified in hiring, credit and justice; • surveillance and coercive state power; • AI used in military targeting in Ukraine and Gaza. These problems require timely decisions about systems and institutions already shaping our lives. Why, then, so much emphasis on AI-pocalypse instead? P.S. We are already doing an excellent job of endangering our own survival. No superintelligence required.

Dear Prof.Luciano Floridi, part of the answer may lie in how this narrative distributes attention and authority. Extinction warnings can reinforce the perceived exceptional power of the companies issuing them, while positioning those same companies as indispensable guides to managing the danger. For the media, a countdown offers a compelling story of course. Those incentives can converge without a coordinated strategy: the effect can still be to keep the public debate centred on the industry’s own framing of the future, while its present choices receive less attention. This connects with my work on human agency, PALO (https://www.paloframework.org) and PolicyWatcher (https://www.policywatcher.online) and the question: can we demonstrate who authorised a consequential action, under which rules, what actually happened, and who can intervene or provide redress? This requires following both system behaviour and changes to the terms governing data and users’ rights. It also requires preserving people’s capacity to question decisions and exercise judgement.

Thank you and your team for doing this work. A question for all of you regarding your first bullet point on concentrated control over compute. Much of today’s AI-related services are produced by a very small number of companies. What do you propose to address this?

I think this is very insightful. In the end, I believe AI’s greatest impact (good or otherwise) is always going to be coupled to an existential crisis such as climate change. AI itself has very little meaning in an existential conversation, it has to take form and pull on levers that have the greatest effect on humanity.

Like
Reply

Luciano Floridi.The distinction between a warning and a forecast is especially important here. Once a prediction begins influencing policy, investment, or research priorities, plausibility alone is not enough; the construct, test, deadline, and conditions for independent adjudication need to be explicit. I also find the separation between capability, deployment, and downstream consequences valuable—claims about one should not quietly become claims about the others. Of the four Gate 1 criteria, which do you think is most often neglected when public AGI predictions are taken up by media or policy audiences?

Like
Reply

Not being an expert in AI but focusing my attention on the climate crisis and related inequality, which are scaringly progressing among the general indifference I truly appreciate such comment, coming form a world-renowned scholar in the AI field. Let's hope someone listens

Luciano, I make mistakes everyday, however I hope that I'm grounded and unpretentious enough to celebrate the successes of others who are trying to help save humanity.

Excellent work! It's interesting how people normalize living under human on human tyranny -- yet machine on human tyranny is unthinkable. From an information-theoretic entropy perspective -- it comes down to a enlightened society that become allergic to tyranny, allergic to subjugation in any form. Human governance is primitive (tyrannical rule acceptable). The machines are following in that path. The one clear way to stop the macro suicide path is Axiomatic Governance -- governance at the boundaries allowing for max degrees of freedom for reasoning with deterministic (allow/deny, no coaxing or reasoning) at execution with real world interactions. See my pinned post for more: https://lnkd.in/p/eV2XuGD8

  • No alternative text description for this image
Like
Reply

AI is not bad because it will literarily destroy humanity. It is bad because it makes students perform somewhat worse, and because it tends to diminish human intellectual and linguistic capacities in general. Unplausible catastrophic predictions make us less appreciative of these plausible but comparably lesser dangers.

Like
Reply
鈴木 桜 Sakura Suzuki

Japan-based Communication Strategist | Musician and Creative| Analytical & Systems-Based Approach to Human Understanding | 戦略的国際コミュニケーション支援 | 専門性を伝わる価値へ | システム的思考と人文的洞察 | 分野横断・探究型 | 音楽家

1mo

Thank you for your work. The title captures the essence of this matter so well: *Not even wrong*. So many of these doomsday claims and speculations cannot be tested as true or false, yet they circulate because they can elicit strong emotions and manipulate crowd psychology.

Like
Reply

Urge riassunto e traduzione italiana ;)

See more comments

To view or add a comment, sign in

Explore content categories