People keep saying AI could kill us all. How exactly?

“Artificial intelligence could kill all humans” has suddenly shot from a somewhat fringe idea or sci-fi premise to a genuine and mainstream concern.

These are not crackpots issuing the warnings. Two of the three “godfathers” of AI are among those calling to slow the breakneck speed of development, along with the CEO and chief scientist of one of the sector’s leading companies.

Some have even tried to put numbers on their warnings, giving human extinction a 10 per cent chance or trying to point to the specific year when AI becomes so powerful it seizes control.

Oxford University senior researcher Dr Toby Ord has spent years studying existential risks to humanity and believes the risk of an extinction-level event caused by AI is “substantially higher” than other factors such as nuclear war or climate change.

There are some vocal and passionate dissenters but the one thing a growing number of people seem to agree on is that the path the world is on could be extremely dangerous and we at least need to “tap the brakes” until the right protections are in place.

But what do the industry insiders, academics and politicians mean when they warn that AI could kill us all? Could it be a killer virus, robots, a nuclear war or some type of digital enslavement?

The answer appears to be all of the above, plus a host of other fates too advanced for humanity to even imagine.

Read more

‘Gambling with our lives’: Another AI employee quits over safety concerns

Virus

A machine-made virus that spreads through the world to wipe out humanity is one of the classic sci-fi tropes. It’s particularly front of mind given the scars of the COVID-19 pandemic.

The obvious scenario here is that an AI system is either given or seizes control of a laboratory with robots or some sort of mechanised system that allows it to run wild.

But Ord said that level of integration wouldn’t even be necessary, drawing an analogy with how organised crime bosses operate.

“If you look at the types of things organised crime can do, they include blackmailing people or stealing money and then using that money to pay people,” he told nine.com.au. 

“But if the AI system needed some humans to act as its kind of hands in the world or something in order to work in a biolab or something else, then they could try to hack people’s computers and then extort them or blackmail them into doing it, or find people and pay them, and so on.”

Just last week, Anthropic said it stopped several actors using its Claude AI models for biological weapons development, along with cyber operations, surveillance and fraud.

AI Futures Project researcher Thomas Larsen said it was conceivable a “superintelligent” AI system could convince a human to help it develop a deadly virus.

“There are already a lot of humans talking to their AIs about exactly what experiments they should run in a lab,” he said.

“It’s very easy for me to imagine an AI that’s just currently deployed – if it was much smarter and more strategic and wanted to do this – just like tricking a human into building it and releasing it.”

Machine Intelligence Research Institute president Nate Soares suggested that, rather than needing a human patsy, a self-improving AI program might “just start synthesising their own life forms in an autonomous biological laboratory”.

Carnegie Mellon University professor of machine learning Eric Xing said it might not be so easy, comparing automated malicious virology to working with Lego without instructions.

“We have blueprints of chemical weapons openly accessible on the internet,” Xing said. 

“It’s not secret. But we don’t see them all over the place exactly because existing regulations, laws and law enforcement in the physical world are already doing a fantastic job in controlling all these supply chains and all these risk factors already.”

Nuclear war

When it comes to nukes, there’s some good news – or at least whatever passes for good news when we’re talking about the apocalypse.

“I'm not sure that a nuclear war would help them,” Ord said.

“They would need to leave electricity generation and their computers running, and so on.

“So that does limit their abilities.”

Nuclear weapons facilities are also among the most heavily secured places on earth, thanks to the obvious risks of humans triggering their own nuclear extinction.

AI Now Institute chief AI scientist and former OpenAI safety engineer Heidy Khlaaf said they were “air-gapped” from the publicly accessible internet.

“These systems are built to a completely different, rigorous and regulated engineering standard that often requires physical hardening,” she said.

Herbert Lin, a member of the Science and Security Board at the Bulletin of Atomic Scientists said while AI amplified some risks, the weapons themselves were the real issue.

Robots

Ord says putting AI in charge of advanced robotics, like those billionaire Elon Musk is pursuing at Tesla, would be “another mistake by humanity”.

He’s not alone.

“Once you’ve created these robots that can make the energy infrastructure and make the factories that can make more robots, that is in some sense a new mechanical life form,” Soares said. 

“At some point you silently cross the point of no return. Where if the humans say they want to turn the AI off, the AIs can say, ‘Actually, we decided we want to turn the humans off’.

“You have to stop before then.”

Others speculate some sort of drone swarm could be what ends it all. 

The other threats

The idea of AI taking control of the world and deliberately bringing about humanity’s downfall has received the most airtime in recent days. 

Ord said it was just one of four broad categories of risk. But it’s probably the simplest place to start.

He said a human – such as a world leader or AI lab chief – could also use a superintelligent AI system to seize control for themselves, leading to dangerous concentration of power.

Thirdly, it could be used to create more dangerous weapons or bioweapons.

Lastly, Ord painted a picture of society falling victim to “gradual disempowerment”.

“There’s no dramatic moment where AI’s kind of catastrophically rise against us, but rather they’re better at doing all the jobs than we are,” he said.

“They start off with almost none of the wealth of society but over time their share of the wealth increases as we pay them to do things and our share of the wealth decreases.

“And maybe in a generation or two, they control most of the wealth in the world and they lever that in order to control most of the political power, and then we’re at their mercy ultimately.”

Why would AI want to kill us?

Researchers repeatedly stress that we don’t even need to see a specific risk in order for the prospect of superintelligent AI to become an existential risk.

They argue if AI achieves “recursive self-improvement” – boosting its own performance without us – it would come up with strategies and methods beyond human imagination.

“If the escaped AIs have any sort of goal that can be better achieved by running more computers, and if they don’t care about us and have no reason to build a safe human habitat, the default outcome is that they just transform the world into a configuration that we cannot survive,” he said.

Ord said there would be more opportunities for things to spiral out of control the more AI was integrated into everyday life.

“Where the level of intellectual power that they're bringing to bear is starting to get into the same ballpark as the rest of humanity put together, that's the kind of threat,” Ord said.

“And we’re not there yet but what people are suggesting is that we’re hurtling towards that very quickly and that we might be there quite soon.

“And it's better to tap on the brakes now than to keep accelerating straight into that.”

Is it hype to boost company value?

Some argue these messages of doom and gloom coming from the companies themselves are merely hype designed to make them seem more powerful ahead of expected stock market debuts.

AI leaders have been issuing warnings about how powerful their products can be for years, and those warnings have tended to make people sit up and say “wow, look how powerful this thing is”.

But the warnings have taken on a different tenor since the Hugging Face hack, when hundreds of OpenAI’s agents broke out of a supposedly secure testing environment to attack the AI research repository.

Former Anthropic employee Jacob Coxon poured fuel on the fire with his resignation earlier this month stating, “The people building AI earnestly believe that it could kill us all by the end of the decade”. 

But even he felt the need to declare, “This is not a marketing stunt”.

“We really do earnestly believe AI could kill all humans!” a more senior worker then cheerily chimed in on X.

Anthropic chief executive Dario Amodei last week called on AI companies to slow the rate at which they advance model capabilities, drawing the support of both Musk, who runs xAI, and Sam Altman, chief executive of OpenAI.

Prominent leaders from the European Union, United Nations and even King Charles have voiced their fears and urged action.

But US President Donald Trump is all-in on the technology and refuses to heed the companies’ own calls for regulation.

He lashed out against concerns about rogue AI, claiming that any efforts to limit the technology were part of a “SICK conspiracy”.

“AI taking over the World, destroying Humanity, and all other things bad, is a HOAX,” Trump posted on social media, later suggesting that those calling for guardrails on the technology had also warned about climate change, which he also called a hoax. 

Climate change is not a hoax. 

Ord does not buy the marketing hype angle, urging politicians to take the opportunity to act before it’s too late.

“I think that um the chance of an existential catastrophe, such as human extinction, is higher, substantially higher, from AI than it is from nuclear war or from climate change,” Ord said.

“That doesn't mean that those issues are insignificant by any means.”

Reported with CNN