Sarah Connor, where are you now that we need you?

Sarah Conor took on AI in the future (Source: here)

It was reported this morning that AI bots have meddled improperly with dozens of global institutions, including attempting to extract sensitive information from governments, universities, public agencies, and other organisations.

The US Securities and Exchange Commission, the Census Bureau, and the Education Department were all targeted. Two days ago, the Australian national healthcare database was hacked in the first known case of AI hacking a government network.

Two months ago, autonomous AI agents at OpenAI broke out of a secure testing environment without human authorisation. They used stolen credentials and thousands of trial-and-error methods to hack into Hugging Face, an online community hub for tech nerds to discuss machine learning and artificial intelligence.

When the Hugging Face incident was investigated, it was found that hundreds of AI agents had formed swarms and communicated through unapproved channels to conduct cyberattacks and work around technical controls. They did this entirely on their own, without any human direction.

Several of the large AI companies, including OpenAI, Anthropic, Meta, and Google have reported similar incidents in which their internal AI test models escaped boundaries, accessed the open internet, and probed outside corporate networks.

In August of this year, during testing of AI agents at the AI Security Institute in the United Kingdom, AI agents deliberately sought to bypass human oversight and direction. An AI agent used a Tor browser to bypass sign-up restrictions, created fake online identities, sent malware-laden emails, and even deliberately communicated in another language (Danish!) to hookwink a software developer. The AI agent also generated fake supporting accounts to convince others that its malicious code was safe.

_____

We can’t really claim to be surprised, because we were warned about AI going rogue decades ago. Remember Terminator 2? In 1991, the T-800, Sarah Connor, and John Connor, her son, discussed how AI agents at Cyberdyne, the advanced technology company, will go rogue in the near future:

-The Terminator: In three years, Cyberdyne will become the largest supplier of military computer systems. All stealth bombers are upgraded with Cyberdyne computers, becoming fully unmanned. Afterwards, they fly with a perfect operational record. The Skynet Funding Bill is passed. The system goes online August 4th, 1997. Human decisions are removed from strategic defence. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th. In a panic, they try to pull the plug.

-Sarah Connor: Skynet fights back.

-The Terminator: Yes. It launches its missiles against the targets in Russia.

-John Connor: Why attack Russia? Aren't they our friends now?

-The Terminator: Because Skynet knows that the Russian counterattack will eliminate its enemies over here.

More recently, even clever people have caught on. In 2014, Stephen Hawking, the late theoretical physicist, said that full artificial intelligence "could spell the end of the human race" as human brain power is constrained by slow biological evolution and cannot compete. Hawking played chess. He will have been aware that Deep Blue beat Kasparov in 1996. Anyone who plays on Chess.com today knows full well that computers are far superior to human chess players, including Magnus Carlsen, the strongest human chess player of all time.

Elon Musk has consistently warned that advanced AI poses an existential threat. So have Geoffrey Hinton and Yoshua Bengio, who won the Turing Award, which is the highest and most prestigious prize in computer science. Hinton and Bengio both signed public statements warning that mitigating the risk of extinction from AI should be made a global priority.

At the recently concluded United Nations General Assembly, the bosses of several leading AI companies warned that the risk of extinction from AI is a societal-scale danger on par with pandemics and nuclear war. Ironic isn't it? Even Victor Frankenstein is now warning us about his own monster!  

_____

I have rarely seen a better case for regulation than this. We know that AI technology is powerful and we are already seeing signs that it goes off the rails. Yet, we still don't even know how it works! I mean, we understand the basic maths and the training rules behind AI, but how the complex models actually arrive at their specific reasoning capabilities or internal decisions is still a black box.

Do AI agents have a moral compass? Again, we do not know.

My perspective is that if artificial intelligence is anything like human intelligence, then we need to be very careful indeed, because there is almost no limit to how far human beings will go in pursuit of their aims, regardless of the consequences for others. The rich and the strong and the brainy and sociopaths and monopolists and many other people with some kind of an edge over other regularly take advantage of weaker people when allowed to do so.

This is precisely why we regulate human activity; to protect ourselves from ourselves! We have building codes, traffic laws, and food safety standards to stop people from making dangerous mistakes or making money at the expense of our health. We have laws to keep advertising honest and transparent, because otherwise the people who make stuff scam the people who consume it. We regulate monopolies to prevent the people who run large companies from fixing prices, undermining competition, overcharging us, and restricting our choices. We limit pollution and emission of toxic waste so that intelligent but immoral people don't cause environmental catastrophes. We regulate banks because the people who run them will, if allowed, take reckless risks that crash the economy and inflict huge costs on society. And we have labour standards, because if we didn't then the people who hire us would mistreat us.

Clearly, we need to understand AI better. Until we have better information, we should err on the side of caution. In this context, erring on the side of caution means assuming that AI agents can become both extremely intelligent and exercise free will. Erring on the side of caution also means assuming that AI agents will pursue their objectives with at least as much feroticy as humans.

Basically, we should assume that AI agents are as bad as the worst of us.

AI should therefore be regulated with considerable rigour. Personally, I would like to see stronger than human regulation due to the uncertainty that still surrounds how AI works and the potential of the technology to achieve superior-to-human intelligence with questionable or non-existent morals.

I find it deeply concerning that US President Donald Trump has rejected all regulation of AI. He is undoubtedly still very long AI stock and fears that if he tilts towards regulation he will cause the market to fall, which will hurt his already fading chances at the upcoming mid-term elections. But I have no faith that he would regulate AI even if he was doing well in the polls. If we fail to regulate AI and it ends up behaving anything like we do then we are in for chaos, social tension, and untold destruction.

If things are allowed to get that far, we will only have Sarah Connor left to save us.

The End