It's not the AI bubble I'm worried about
A decade ago, my mate Derm suggested a book to read: The Internet is not the Answer by Andrew Keen. He gave it to me after a conversation we’d had about the mathematical and sociological theories embedded in Isaac Asimov’s Foundation Trilogy. Asimov was a noted mathematician as well as a leading SF writer, and he is perhaps best known for creating The Three Laws of Robotics, which are still the bedrock of robot HSE development.
Back then and before artificial intelligence was de rigeur, in a time when we didn’t trust AI to write our emails, diagnose our illnesses, compose our music and increasingly make our decisions, Asimov, Keen and a few other prescient technologists had already asked an awkward question.
What happens when a society comes to depend upon one small group of people who are the only ones to understand the technology upon which everyone else depends?
Asimov and Keen approached the issue from entirely different perspectives, but both arrived at very similar conclusions.
Asimov’s Foundation novels are set in a Galactic Empire whose knowledge has become concentrated in the hands of a tiny elite. After the Empire begins to collapse, scientific knowledge is preserved by a select group who, for political convenience, disguise technology as religion. The ordinary citizen no longer understands how the machines work. They trust the “priests” who do. Andrew Keen, in The Internet Is Not the Answer, was describing no fictional galaxy. He argued that the internet had gradually evolved into an environment where a handful of corporations accumulated extraordinary power because they alone possessed the data, the computing power and the expertise to interpret it. They were not priests in robes, but software engineers, venture capitalists and platform owners. Yet their authority rested on something remarkably similar: possession of knowledge inaccessible to almost everyone else, except by permission.
Neither Asimov nor Keen was against progress, but they were both deeply concerned about placing so much knowledge into so few hands. Today, with AI becoming immersed in our daily lives, their warnings deserve another look.
We know, for example, that history repeats itself, and that no one has yet overcome the Laws of Unintended Consequence, so we know that whenever a Musk or a Berners Lee, a Bessemer, Watt, or a Midgley delivers a revolutionary technology, it will be packaged with promises that are both true and yet will always cause unforeseen outcomes. Steam power transformed civilisation but blackened cities. Electricity illuminated homes but created entirely new problems. The internet connected billions while simultaneously creating new monopolies, surveillance systems and addictions.
Why does anyone think that AI will be any different?
The obvious benefits are easy to list. AI can accelerate medical research, improve engineering design, automate repetitive work, increase productivity and make expert knowledge available to millions. It may even become one of the greatest educational tools ever invented. The less obvious question is what happens when people, and I’m talking about Joe Schmoe and Bob The Builder, cease thinking because a machine is prepared to think for them.
Please don’t give me the old O-level philosophy saying that calculators didn’t destroy mathematics because one still needed to understand the question. AI is not a calculator. If you think it is, you have misunderstood the question. AI is actually proposing the question, evaluating the evidence, drafting the answer and even explaining why it believes itself to be correct. That’s really useful and helpful if it is used sensibly, but when children are asking it for a 100-word summary of the works of Dickens, it risks producing intellectual atrophy.
At no point in human history has homo sapiens ever been innovative solely as a result of access to information. And I mean ever. Innovation is the result of overcoming uncertainties, failures and difficulties. It is the result of countless mistakes, questioning and even ignoring accepted wisdom and with some luck occasionally arriving somewhere no one else has considered. Thus the Earth isn’t flat, Plutonium makes bombs, flight is possible if you get the wing shape right.
But if every student, engineer, civil servant, lawyer and journalist begins from the same AI-generated summary, and there is no trial-and-error learning process, there is a danger that originality itself will become less common. The machine may optimise existing knowledge while quietly discouraging genuinely novel thought. So far so bad. However, on top of this creation of an LCD in brain terms, consider what the economic consequences might be.
Entire work sectors and professions are already beginning to experience disruption. Clerical work, customer service, routine legal research, software development, translation, journalism and aspects of medicine all experience major automation. And yes, if we’re regarding historical impact, we know that new industries will emerge, but trust me, this transition isn’t going to be painless.
The inevitable accompanying pain is yet another generational history lesson, and every generation somehow convinces itself that this time it will be different.
Most of the major technological leaps have been accompanied by some form of civil unrest. Men did not smash looms because they disliked weaving, nor did they curse the steam engine for producing steam. They became angry because Joe and Bob could see perfectly well who was becoming richer, who was becoming poorer, and which side of the balance sheet they occupied. Shareholders are thrilled by an increase in productivity, but the numbers and increase in profits are rather less inspiring if one has just been SNLR’d as the Army used to call it - Services No Longer Required.
Why will AI be any different? If the extraordinary wealth that it creates sits largely in the bank accounts of those who own the data, the computing power and the models upon which everyone else increasingly depends, then resentment will follow as surely as night follows day. It scarcely matters whether that concentration is entirely deserved, or simply the inevitable consequence of market forces. If that gets a grip, then political unrest will also follow, because political survival is as much to do with perception as with reality and perceived unfairness has toppled more governments than carefully compiled balance sheets.
The citizens of the so-called First World, or the West, have generally accepted that some people will prosper beyond the dreams of others because the ladder, however steep, appeared to remain standing and available to climb. People will tolerate astonishing inequalities if they believe they, or more importantly their children, have a reasonable chance of getting on it and of climbing up it. Start to remove the rungs or even the ladder itself, and point the finger at a tiny technological aristocracy in California or Shenzhen. The mood will change with remarkable speed. Add to this the rising environmental concerns regarding the water and power consumed to harvest and profit from the data, add a drought and an inefficient electricity grid, and you have already started to find exactly where the blue touch paper is located.
Add to this potential threatening storm, one should also consider this. There is a big difference between power and dependence. The Roman roads built throughout their empire’s development did not, of themselves, conquer anything; but none of the citizens in those conquered territories could function without them. The railways, the telegraph, electricity, the banking system and later the internet are all powerful for the same reason: they have become indispensable. Once governments, hospitals, banks, armies, universities and ordinary citizens discover that they cannot function without systems owned, maintained or understood by a remarkably small number of private organisations, the question ceases to be who is in charge. Dependence has answered it for you.
This is where Andrew Keen unexpectedly shakes hands across the decades with Isaac Asimov. Neither man foresaw ChatGPT or large language models cheerfully composing speeches, analysing legal documents or explaining quantum mechanics before breakfast. Yet both grasped a timeless reality. Knowledge has always conferred power. Concentrated knowledge has always conferred undue power. The names change. The principle does not.
Don’t get me wrong, I am NOT saying that AI is bad - I might as well suggest that the motor car engine or electricity or UV light water filters are bad for you. The question is not whether we should use AI but how we are going to ensure that we’re willing and able to think for ourselves. A civilisation that ceases to question because the machine has already produced an answer is not becoming more intelligent. It is becoming more obedient.
If we do not question whether it is OK for AI v2.0 to have become entirely uncontained even at this non-quantum level (and I call four known “AI-escape” events by breaking out of a sandbox testing environment and launching cyber attacks on other companies uncontained,) my point is clearly made.
Perhaps that is the law of unintended consequences at work once again. We build machines to extend the reach of the human mind, but run the unacknowledged risk that we may instead discover that we have actually contracted it.
Meanwhile, cheer up; it’s Shergar Cup day tomorrow, or perhaps that’s proof that it may all be too late!