Could AI Really Kill Us All?

Prefer to read? Check out the Audio Transcript

Although the transcription is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription software errors.

Emma Varvaloucas: Are we really in danger of AI going rogue and killing off humanity, or is it all an overblown distraction? That’s our main story on the What Could Go Right progress report. And stick around because we’re also gonna be talking about a brand new way to save our beaches, and why you should thank the Beatles the next time you get a CT scan.

If you’re new here, hey, I’m Emma Varvaloucas, I’m the executive director of the Progress Network. Let’s get into it.

Okay, so I’m gonna tell you something, but don’t freak out. Earlier this month, a guy who used to work at the AI company Anthropic said that we could be facing, um, human extinction. But don’t freak out.

Come on, I said don’t freak out. Stop prepping the doomsday bunker, put the canned beans back in the kitchen. I’m going to explain why it’s not the time to go underground and create our own silo just yet. Stick with me.

So- Here’s what happened. In mid-September, Jason Coxon, a former researcher for Anthropic, posted on X that he had resigned from the company. Then he listed some scary reasons for his resignation in a thread. He said that the big AI companies like Anthropic and OpenAI are acting irresponsibly. He warned that senior leadership believes that AI could, quote, “Kill us all by the end of the decade,” and that, quote, “No other human activity poses this level of danger.”

That set off a media firestorm, and it sounds even worse because he’s not the only person saying it. Geoffrey Hinton, a computer scientist and Nobel Prize laureate known as the godfather of AI, agrees with Coxon. He said that a 10% probability of AI killing off humanity is not unreasonable. He also warned earlier this month that Congress only has about a year to regulate AI before it permanently loses control of it.

Before you start digging a hole for the second bunker, let me just stop you again. First of all, you don’t have a strong enough back for that. Also, I have some good news in this storm of apocalyptic prediction. Let’s poke some holes in these numbers. Hinton himself admits that no one really knows how to give a sensible estimate on an AI doomsday clock.

If that’s the case, then it raises the question: What is all this forecasting based on exactly? Is there some kind of statistical model backing these claims of 10, 20, or any other percent chance of annihilation? Does it take into account potential bottlenecks like training data running out, computer chips not being powerful enough to back super intelligence, or the end of the data center boom?

Without some hard facts behind them, these, “expert” predictions” are really just ambiguous guesses. And now, let’s get into the main fear of doomsday predictors. Will the AI agents one day get too smart, start making decisions on their own, and come after us like in the movie I, Robot? Will they engineer a super virus or sabotage our infrastructure to wipe us out?

Well, despite what you’ve heard about Hugging Face, you will be happy to learn that we don’t have the technology for AI to go rogue, think for itself, and take over . . . yet. Model Evaluation and Threat Research, or METR, is an organization that evaluates frontier AI models to see if there is a threat risk.

METR says that AI agents have weak judgment, strategic reasoning, and reliability. Like my friend’s sweet but dumb ex, those weaknesses limit the kinds of actions they could carry out. It basically takes anything that involves careful planning, subtlety, or adaptation to obstacles off the table. METR says the risk could increase in the future, but as of right now, an AI-planned takeover is not plausible.

And to me, it seems like they’re right. For example, back in April, a San Francisco-based startup called Andon Labs signed a three-year lease on a retail space and put an AI agent in charge. They wanted to see if it could run a small business all by itself. So they gave the agent, named Luna, a credit card and internet access.

The first thing Luna did was search for human employees to hire. Good start. But on the second day that the store was open, Luna forgot to staff anyone to work there. I don’t know about you, but I am not getting the vibe of a genius human-killing mastermind here. But back to the METR report. The organization lists three criteria for an AI takeover that may remind you of your favorite Sherlock episode: means, motive, and opportunity.

AI would need all three of those to carry out an extinction-level event, and that would require a lot of steps to get there. In the takeover scenario METR describes, AI would have to be smarter than the top human experts in every field. Its developer would have given it near complete control, and high-level government officials and military leaders would be treating it as a trusted advisor that they almost always listen to.

It would also have to have found a way to somehow produce millions of robots per month at some industrial base somewhere without any humans coming to stop it. We would have to have put AI in charge of a lot of things and barely pay attention to it for it to pull this off. To be honest, we’re worried about the wrong kinds of harm.

A rogue AI human killer seems a bit far-fetched, but there are real problems related to AI right in front of us. To start, scams, deep fake pornography, and AI-generated misinformation are all happening right now. Or listen to this: An investigation just published by The New York Times found that the online gambling company DraftKings is using AI to target the betters most likely to keep losing money, and those tend to be the ones that are prone to a gambling addiction.

Cognitive offloading, when we use external tools like AI to reduce the mental effort required for a task, has already gotten so bad that in the spring, US intelligence moved to intercept a Chinese ship based on an AI-produced report that they later figured out was totally false. Luckily, they caught the error before reaching the ship and probably causing an international incident.

And it gets worse. This month, the AI company Anthropic released an incident report that describes their chatbot Claude being used for government surveillance of dissidents, Russian espionage, and the manufacturing of bioweapons, missiles, drones, and bombs by people we do not want having those things.

Although, the silver lining there is that Anthropic did identify these operations and shut them down. So is all the drama around an AI-orchestrated extinction event meant to distract us from real and immediate issues we need to face? AI scholar Timnit Gebru thinks so. Gebru, who is considered to be one of AI’s biggest critics, argues that AI companies actually benefit from our fixation on super intelligent machines trying to kill us all because it pulls our attention away from important questions that they don’t want answered.

Like, what policies do we wanna have in place around AI surveillance, its effect on the environment, and how it’s being incorporated into military systems? Basically, the rogue AI extinction talk is a way for us to avoid looking in the mirror. We are in charge of how we use AI. Human decisions on how it can be used and how it should be regulated are still within our control.

In fact, there is a lot we can do to push AI in a positive direction if we’re willing to have these conversations and act. The ideas are out there. For example, Progress Network member Ian Bremmer wants a cohesive international system to govern AI that would operate similarly to how a global central bank looks for risks to the financial system.

The European Union, China, and the G7 have all published initiatives that emphasize transparency, human control, and risk response. And the International AI Safety Report calls for a multilayer defense system that establishes safeguards within AI companies, such as filters on what prompts AI will accept and what kind of information it’s willing to spit out, as well as protective measures outside of them.

Those would be things like DNA screening to flag bioweapons, established protocols for cyberattacks, and media literacy programs. By the way, AI companies are not required to publish safety frameworks, but the number that do voluntarily doubled last year. We should support these kinds of responses and ask our politicians to do the same. November is coming soon.

Anyway, with all the drama this month from doomsters like Jacob Coxon, many are wondering whether AI companies are just trying to create hype and raise their stock prices before they go public, or maybe slow down the rate that they’re burning cash building all those data centers.

It’s hard to say, but either way, it doesn’t look like you’re gonna need that bunker anytime soon. And honestly, throw out the canned hot dogs They’re just gross.

Next, here are some numbers that’ll make you smile. 74%, the share of the globe with internet access, up from 40% in 2015. 2,702, the number of terawatt hours that nuclear energy generated in 2025. That’s a record. .5%, the amount global fossil fuel emissions are set to fall in 2026.

And now, here’s a story about the good things AI can do for humans. Picture this, you’re lying in a beach chair. The water is sparkling, and the sand is golden. It’s gorgeous. But the shoreline does seem a little small. Come to think of it, you did see a sign about erosion earlier.

For a second, you think about how sad it’s gonna be when this beach is just totally gone. You promise yourself you will be back before then. A few years later, you show up with all of your friends, and the beach is bigger? The beach is bigger. But is it even possible to grow a beach? An experiment in the Maldives proves that beach growing works, with a little help from AI.

A Massachusetts company called Coastal Assembly extended shorelines in the Maldives 90 feet towards the ocean and 300 feet long. They also built a new sandbar with 30,000 cubic feet of sand in only six months. So how did they do it? I’ll explain. But first, why is stopping beach erosion so important?

Erosion does a lot more than ruin a vacation spot. It can upend people’s lives. Thousands of coastal communities are harmed by it, and the United Nations says that 1.2 billion people, yes, bill- Billion could be displaced from disappearing shorelines by 2050. And of course, rising sea levels caused by climate change are making the problem worse.

Right now, there is a new process being tested to fix it. Using tech that was first developed at the Massachusetts Institute of Technology, Coastal Assembly had AI analyze 10 years of satellite images, water currents, and other data to predict the erosion patterns in the Maldives. Then they created 54 structures of marine concrete and arranged them in a way that breaks up bad waves and allows good waves that help build up sand to flow through.

This process is able to both predict future erosion and offer solutions at a much lower cost than old fixes. Sea walls, water breaks, and big concrete barriers are expensive and not very effective. Sand pumping and dredging damage the ecosystem and don’t last very long. The new method allows faster custom solutions for each unique area based on the AI-supported erosion predictions, which cuts down on costs.

We still have to see how well this works in the long run, but projects are already planned in the Bahamas, Boston, and Miami.

And now it’s time for This Week in History with a positive news story from the past. When you think of The Beatles, what comes to mind? You may think of 1960s outfits or your favorite Beatles songs.

I Want to Tell You mine, but I really can’t choose. Or you might think about iconic tour photos of Paul, John, Ringo, and George traveling around the world with women screaming everywhere they went. However, I bet you don’t associate The Beatles with the historic invention of the computed tomography, or CT scan.

Surprisingly- You should. On October 1st, 1971, a patient in London had the first CT scan to check out a tumor she had in her brain. A CT scan is a type of imaging that uses X-ray techniques and a computer program to create cross-sectional slices of the bones, blood vessels, and soft tissues inside the body.

The invention of the CT scan may be the greatest advance in medical imaging since the discovery of X-rays, and was a gigantic win for modern science. CT scans revolutionized the ability to diagnose certain medical conditions because they can show a lot more crucial detail. They’re why doctors today can find internal bleeding, diagnose anything from kidney stones to cancer, and check for blood clots and aneurysms.

Definitely crucial stuff. The scan’s invention was all thanks to the work of Sir Godfrey Hounsfield and indirectly, The Beatles. Let’s back up. We have to talk about a company’s huge pivot decades before this invention. In 1897… Wow, we, we backed up quite a bit. Electric and Musical Industries, or EMI, was a small recording company called The Gramophone Company, but they almost went bankrupt during World War I and merged with another company to become EMI.

They didn’t wanna get that close to collapse ever again, so they started looking for other sources of income and got into radar technology. By World War II, EMI was known as a heavy hitter in radar weaponry development, and they were finally financially stable. In 1951, Sir Godfrey Hounsfield started working at EMI.

At first, he worked on radar, guided weapons, and computer development. However, EMI sold off its computer division and moved him into its central research laboratories. A few years later, he had impressed his bosses and got the freedom to choose his next research project. What to choose? He didn’t know, so he took a walk through the English countryside, as one does.

Then inspiration hit him like a ton of bricks. Well, more like a gentle breeze in the English countryside, but You get the point. The radar systems he had worked on scan their surroundings and look for patterns on the periphery of where they’re sending the radio waves. Hounsfield wanted to try to reverse that process by looking for patterns inside an object from the outside.

With what he knew about X-rays, computers, and math, he figured out that he could view a three-dimensional object as a series of cross-sectional scans and see what’s inside. But there was a big problem. Actually testing his hypothesis and developing a new scan would require a ton of money from EMI. This is the part in the Hallmark movie where the big city girl decides that her good, stable job is unfulfilling, and she actually wants to pursue her original dream of opening a bakery in Vermont.

EMI is our Hallmark girl. Although the company kept its radar research going, after the Second World War was over, they turned back to pursuing music as part of their business. They bought Capitol Records and signed The Beatles in 1963. This turned out to be a pivotal moment in both the medicine and the music worlds.

Hits like I Wanna Hold Your Hand and I Saw Her Standing There shot the company’s profits up by 80% almost immediately. So when Hounsfield needed some bags of cash to turn his idea into a working product, EMI was happy to fork it over because they were swimming in cash from The Beatles’ record sales. In less than five years, Hounsfield developed the scan and did some tests on animal brains.

It worked. However, he needed a clinician to help him test the scan on a human being. Dr. Jamie Ambrose, a radiologist from London, agreed to meet with Hounsfield, but unfortunately, Hounsfield was kind of a dick. He barely spoke, didn’t feel like explaining the details of his new invention, and belittled Ambrose’s own imaging research with, “I can do better than that.”

To Ambrose’s credit, instead of deciding that he would never speak to this guy again, he gave Hounsfield a gift. Ambrose handed him a jar with a brain in it, and the brain had a tumor. Personally, I prefer chocolates, but to each their own. Ambrose told Hounsfield to use his new invention on that brain and bring him back some proof.

When Hounsfield came back, hopefully in a better mood, he brought an image of the brain that showed incredible details about the tumor. Ambrose was gagged. And on October 1st, 1971, Hounsfield made the first CT scan of a live human patient under the supervision of Dr. Ambrose. It took two days for the images to come back.

When Ambrose and Hounsfield, who were apparently bros now, saw the images for the first time, they, quote, “Jumped up and down like football players who had just scored a winning goal.” I guess the tumor in a jar was the right move. After that victory, Ambrose, the guy with some people skills, led them on a tour like they were members of The Beatles.

When Ambrose presented the images at the annual Congress of the British Institute of Radiology in April of 1972, the audience was stunned. In October that same year, Ambrose presented images from the scanner to a conference of 2,000 medical doctors in Chicago. He received a standing ovation. The dynamic brain scan duo would receive even more ovations and praise over the years.

And in 1979, Sir Godfrey Hounsfield was awarded the Nobel Prize in physiology or medicine. He should have thanked The Beatles in his acceptance speech.

I hope these stories remind you that there’s so much good in the world, so it’s important to shift our focus away from all the bad sometimes. Send this episode to your friend who loves the Wendy’s Baconator. RIP?

Make sure to subscribe to our YouTube channel. We publish 60-second good news stories daily. See you next week.