Volcanoes are erupting in The Philippines, but on-fire Australia received some welcome rain. The Iran war cries have been called off and The Donald’s military powers are about to be hamstrung by the Senate. Meanwhile, his impeachment trial is starting, and we’re all on Twitter for a front-row seat.
What Could Go Right? AI Doom Is a Distraction
The odds of an extinction event are anybody’s guess. We’d do better to focus on present AI risks.
Emma Varvaloucas
This is our weekly newsletter, What Could Go Right? Sign up here to receive it in your inbox every Thursday at 5am ET. You can read past issues here.
AI Doom Is a Distraction
“The people building AI earnestly believe that it could kill us all by the end of the decade.” That recent claim of humanity’s imminent extinction by ex-Anthropic researcher Jacob Coxon set off a firestorm of back-and-forth assertions about p(doom)—the probability of a doomsday scenario caused by out-of-control AI.
Coxon is not alone in his apocalyptic forecasting. “Godfather of AI” Geoffrey Hinton, for one, told the BBC that a 10% probability of AI killing all humans “is not unreasonable.” On the other hand, Yann LeCun, who used to be Meta’s chief AI scientist, believes that the “risk of humanity being wiped out by rogue AI is considerably less than the risk of it being wiped out by an asteroid strike.”
One hole easily poked in any of these predictions is their ambiguity. As Hinton himself pointed out, “nobody really knows how to give a sensible estimate” when it comes to p(doom). There doesn’t seem to be a statistical model that can actually confirm these claims, so whatever number is being floated is really more like a guess.
Still, in this maelstrom of claims and counter-claims, it’s tough for those of us outside of the industry to assess what’s going on. Guess or not, should we be saving up for a DIY bunker?
Let’s start with the main concern of doomsdayers: AI becomes so smart and malicious—either independently or at the behest of bad actors—that any takeover event surpasses our ability to shut it down before it erases humanity. The how is a little vague, but imagine something like a supervirus or mass critical infrastructure attack.
The first thing to recognize is that we’re not there, at least not yet. The latest assessment from Model Evaluation and Threat Research (METR), an organization that assesses the threat risk of leading AI models, concludes that a direct takeover is currently implausible, before going on to report that the risk “could increase rapidly . . . absent stronger alignment, security, and monitoring.”
Simply put, AI at the moment does not have the requisite capabilities. METR notes that agents have weak “judgment, strategic reasoning, and reliability,” limiting “the kinds of harmful actions they could carry out—particularly those requiring careful planning, subtlety, or adaptation to novel obstacles.” (Keep in mind that today’s AI can’t even run a small store on its own.) Experts assume those abilities will improve, of course, but they disagree on how long it will take to reach a level that leaves humans in the dust; many, such as neural science professor and AI critic Gary Marcus, believe the “field needs more breakthroughs” first. So, at least add some more time onto the clock.
An extinction-level event also depends on more than just intelligence. METR, for example, has three evaluative criteria that will be familiar to any Sherlock Holmes fan: means, motive, and opportunity. AI takeover requires all three. METR’s report uses a scenario called AI 2027 as a benchmark for what that would look like:
By the time takeover occurs . . . the AI’s capabilities are far beyond the top human experts in every field, it has been delegated near-complete control over its developer’s compute, cabinet-level officials and military leaders treat it as a trusted advisor and ‘almost always accept its recommendations,’ and it has autonomously orchestrated the buildup of an industrial base producing millions of robots per month.
Even if AI is theoretically capable, then, we’d have to put it in charge of a lot of stuff—with scarce human oversight—before it could “go in for the kill.” There’s plenty of variability inherent in this scenario that has little to do with how fast the technology is evolving, and it’s hard to believe no one would manage it. (I, for one, would like to know how the robot base is going to escape the attention of all the world powers.)
Color me skeptical, then, that the end is nigh. Yet, just because the existential risk seems slim doesn’t mean other serious dangers don’t already abound. AI is capable of significant harm right now. The list of troubles is long: scams, deepfake pornography, and fake news, to start. Anthropic released a report this month detailing several instances of its chatbot Claude being used in malicious activity, from government surveillance of dissidents to Russian espionage to bad actors manufacturing bioweapons, missiles, drones, and bombs. (The silver lining is that the company identified and shut all of these operations down.) “Cognitive offloading” has already gotten so bad that it has resulted in the US military acting on AI-generated intelligence information that turned out to be false. Luckily, the error was caught before an international incident occurred.
I’m sympathetic to the argument that we don’t want to quash AI’s potential for good in a quest to prevent the bad. What it could do for drug discovery, for instance, is tantalizing, and it has already played a major role in scientific breakthroughs. But the mainstream discourse about balance seems to have flattened into a binary: speed up or slow down. There are other options.
Progress Network member Ian Bremmer, for example, suggests a “cohesive international system to govern AI, something akin to a global central bank: an independent organization designed to combat the systemic risks of AI.” (In the meantime, entities undertaking independent assessment and monitoring efforts, such as METR, are taking donations.) The European Union, China, and the G7 have circulated early initiatives that emphasize transparency, human control, and proactive risk response. Safety frameworks published by the companies themselves remain voluntary, but they doubled in 2025, says the International AI Safety Report. That report also calls for a multilayer defense system that establishes safeguards within AI companies as well as “resilience-building measures” outside of them, such as DNA screening to flag bioweapons, established protocols for cyberattacks, and media literacy programs. These are the kinds of things we can support and expect politicians to take seriously.
What with all the drama this month, many are wondering whether AI companies are either trying to hype their products before scheduled IPOs or reduce the speed that they’re blowing through funds during the data center building spree. Some say that Coxon is essentially in a doomsday cult that doesn’t even believe its own brainwashing. Others think we’re platforming the wrong guy. As Clement Delangue, chief executive of the AI platform Hugging Face—yes, that Hugging Face—quipped on social media: “Sorry, but asking Jacob [Coxon] about AI extinction risk is like asking your AC guy about climate change.” That may be true. But it’s also true that there are pressing risks we should be talking about more . . . intelligently. The time to get on top of them is now.
—Emma Varvaloucas
What Could Go Right? S8 E24: The Science of Sports Betting: How Gambling Apps Are Changing Our Brains | with Dr. Brendan Dwyer and Isaac Rose-Berman
Legal betting has exploded in the US. But it has plenty of controversy, including how it affects sports fandom and the way it’s marketed to young men. Are the concerns overblown or do we need a real plan to address real harm? Two guests join our host Zachary Karabell for this conversation. Dr. Brendan Dwyer is a professor and the director of research at the Center for Sport Leadership at Virginia Commonwealth University (VCU), and Isaac Rose-Berman is a fellow at the American Institute for Boys and Men, focused on gambling research and policy. | Listen now
By the Numbers
74%: Share of the globe with internet access, up from 40% in 2015
0.5%: Amount global fossil fuel emissions are set to fall in 2026
2,702: Electricity, in terawatt-hours, that nuclear energy generated in 2025, a record
Go Figure
In news that’s sure to annoy the MAHA crowd, a study tracking more than 62,000 South Korean children across 12 consecutive birth cohorts found no statistically significant change in the prevalence of autism spectrum disorder. Researchers say the findings suggest that rising diagnosis rates largely reflect better detection rather than an “autism epidemic.” Seeing more often might just mean we’ve gotten better at looking.
Quick Hits
🧠 Scientists have overturned the centuries-held belief that the brain is one organ. It’s two “cleverly packaged together,” new research shows.
🐱 Meet the first new feline species discovered in 100 years, a tiger cat found in Bolivia that is smaller than its domesticated cousin.
🧬 An experimental immunotherapy has beaten back a three-year-old’s liver cancer for a year, the first time CAR T therapy has been effective against a chemotherapy-resistant solid tumor. (The treatment has already yielded groundbreaking results for blood cancers such as leukemia.)
🏳️🌈 Hungary will scrap a ban on gay adoption, a significant move toward restoring LGBTQ rights that were heavily restricted in the OrbánEra.
📕 Singapore will pay its citizens to read for five years in a pilot scheme. The earned amounts are small, but officials are hoping that the gamification encourages people to turn away from short-form video and toward long-form reading.
🔬 In a first for RNA therapy, a man with Lou Gehrig’s disease has seen sufficient enough improvements to be able to continue his work as a physician. The results are early but encouraging.
🏖️ A combination of satellite data and AI are making it possible to “regrow” beaches lost to rising sea levels, as well as to more precisely predict where future erosion will occur. A coastline in the Maldives has already been reconstructed; projects in Boston and Miami are next.
🦊 New York just hosted its first fur-free fashion week. The industry event followed counterparts in Milan and London in no longer promoting fur on their runways, events, websites, and social media.
🪐 Scientists just discovered the youngest-known planet, which at less than one million years old is far younger than the five million-year-old planets that previously held the title.
🐟 New Zealand has opened five new marine protected areas, with a sixth on the way. They are the first to be co-managed between indigenous people and the government.
🔋 A global cement giant will test using heat supplied by batteries, rather than by burning fossil fuels, in its production process, a first for the industry.
👀 What we’re watching: Congress found something to agree on: a bill to shield consumers from price hikes caused by data centers passed the House417-3.
TPN Member Originals
(Who are our Members? Get to know them.)
- Your vote is your voice | Lucid | Ruth Ben-Ghiat
- It’s already too late to stop the AI threat | NYT ($) | Thomas L. Friedman
- Oppenheimer, Inc. | No Mercy/No Malice | Scott Galloway
- No AI autonomy without human accountability | FP | Fareed Zakaria
- The other way AI impairs our thinking | The Atlantic ($) | Thomas Chatterton Williams
- A doomsday scenario for American AI | TFP ($) | Tyler Cowen
- It takes more than a year to raise a child | Of Boys and Men | Richard Reeves
- Trump’s $5,000 midterm cash payout is an empty promise | WaPo ($) | Theodore R. Johnson
- Conventional wisdom about the US and China is no longer accurate | WaPo ($) | Zachary Karabell
- Are phone-free schools working? | After Babel | Jonathan Haidt
- London has fallen? Misperceptions of crime, race, religion and life in the UK capital | The Policy Institute | Bobby Duffy
- Americans still believe in progress | Flourishing Friday | Clay Routledge
- It’s war already, Europe | Diane Francis | Diane Francis
- Is the blue wave forecast by polling real? | Slow Boring | Matthew Yglesias
This Week in History: A Big, Blue Discovery
One hundred and eighty years ago today, Neptune became the eighth planet discovered.
Ever since Uranus, the seventh planet—not to mention, long-time butt of middle-school jokes—was identified in 1781, astronomers had been keeping an eye on it to see if they could learn anything else. Eventually, they noticed irregularities in the way Uranus circled the sun that Newton’s law of gravity couldn’t explain. Could its weird orbit be caused by the pull of a nearby planet?
Flash forward about 65 years, and two astronomer-cum-mathematicians—Urbain Jean-Joseph Le Verrier and John Couch Adams—were working to predict Uranus’ exact position as it moved through the sky. (What would be relatively quick today with modern computers, took a lot of pen-to-paper mathematics.) Their calculations set up a third astronomer to make a big, blue discovery.
On a fateful night in 1846, September 23–24, Johann Gottfried Galle used the Uranus calculations to figure out where the planet affecting it should be in the sky. He fired up his telescope at a Berlin observatory and, huzzah! There was Neptune—or what would be Neptune once it was named for the Roman god of the sea—the first planet discovered by mathematical calculation instead of direct observation.