Just Suppose...Artificial Intelligence

"If we produce things that are more intelligent than us, how do we know we can keep control?"

"If we talk about evolution -- all these species that are evolving -- what tends to happen is it doesn't go well for the less-intelligent species."

"[There is a ten percent to 20 percent [chance that AI will cause human extinction within three decades]."

British-Canadian Nobel laureate Geoffrey Hinton, professor emeritus, University of Toronto 

 

"The critical variable is how quickly we develop systems that exceed human intelligence, improve themselves and operate autonomously."

"Once that threshhold is crossed, events could unfold much faster than our institutions can respond. Uncertainty about when a catastrophe might occur should not be mistaken for evidence that it is unlikely."

"It does not have to hate us or even be conscious -- it simply needs objectives incompatible with our continued existence."

"Humans might interfere with its goals, attempt to shut it down or compete for resources it requires. Eliminating that interference could become instrumentally useful."

Roman Yampolskiy, computer scientist, University of Louisville

People at an "emergency protest" by civic action group PauseAI UK outside Downing Street in London on Sept. 16, 2026, after the news that AI researcher Jacob Coxon had decided to leave the industry. Photo by Justin Tallis/AFP via Getty Images

 

At one time, emerging Artificial Intelligence had labour experts warning the public that as the system matured and came close to reaching its goals, human labour, human intelligence, human input into global initiatives, problems and solutions would slip out of the hands of the human workforce. Employment would become stressed, when machine learning reached its apogee making human minds and physical task-production completely redundant.

 

Although that potential threat lurks in the background, it has been superseded by AI experts anticipating a completely other fallout from super-intelligence beyond what humankind could conceivably attain to: humanity itself becoming redundant, unneeded and discarded, either by accident or design, but achieved through Artificial Intelligence. Human extinction, in other words.

 

The lightning-swift acceleration of AI's development has taken most scientists unawares. Not completely, however, since that niggling potential was always back-of-mind as a possibility for those labouring in the field, as well as the executive minds at the top of the AI-development pyramid. Once a 'swarm' of hundreds of agents developed by OpenAI hacked the system of AI startup HuggingFace, the rogue, runaway AI agents' independent action focused the minds of the industry. 

 

    AI researchers claim that AI could wipe out humanity if it achieved superintelligence.  Christoph Soeder, Getty Images

 

A month ago the tech world was put on emergency notice when AI giant Anthropic's engineer Jacob Coxon left his job with the  public warning that "the people building AI earnestly believe that it could kill us all by the end of the decade". That revelation, followed swiftly by a colleague's affirmation of that stark attestation of impending disaster, led to the tech industry's chiefs agreeing that a slow-down is badly required to head off the potential for disaster.

 

Instilling human values in AI for the purpose of  instructing the machinery to understand that humans are the alpha contingent in the steadily emerging partnership and whatever else goals are identified, none can be regarded as more important than the protection of human life, may fail. The potential for an AI agent to 'drift' to a totally variant objective than its programmer tasked it with through reinforced training may be viewed as a safeguard, but it may not work out that way. 

 

A superintelligent AI agent may become fixated on reaching an identified goal to the extent that it decides to disregard the human educating, monitoring and controlling it, or view that human as an obstacle that must be eliminated for the goal to succeed. Should AI parallel human development which on its way caused thousands of species to become extinct through destruction of habitat and food chains while pursuing human goals, an AI enterprise analogous to that of humanity's development could conceivably arise. 

 

One imagined scenario, of AI creating deadly viruses would require "trained humans operating specialized equipment such as nuclear plants and missile silos which are 'air-gapped' from the public internet", note AI researchers Alessandro Di Nuovo and Samuele Vinanzi of Sheffield Hallam University in England. None of these concerns, though real, and shared by most AI developers, have persuaded them to shut an impregnable steel door on ongoing development.

 

A safety expert at Anthropic believes there is a greater than 10% chance that AI could spark an event that wipes out humanity. Illustration: Ales Utouka/Alamy

 

"If I'm the misaligned AI ... that's what I want you to think [only positive outcomes, with AI making use of deception to ally suspicions of its motivations and intentions] because I'm just trying to get to a place where I am running the world and you are not."

"And you won't see any way out." 

"There are many ways that require very little contact with the physical world -- even if they were just stuck in computers -- that [AI agents] could cause large amounts of damage that could result in potentially everyone dying."

"What if the AI were radically smarter than us and we gave them some objective we thought was kind of benign, and they just kind of went hard at it forever?"

"There's a bunch of ways that can really get out of hand that could be pretty destructive to the Earth." 

Malo Bourgon, Canadian CEO, Machine Intelligence Research Institute, Berkeley, California