It is all over the media - again. AI will kill us. Now, there are even very specific predictions - seemingly made by subject matter experts, over 10% probability that humanity will be extinct, within the next decade. This must be serious and how can one protect against such an event?
Mainstream media (including the BBC) as well as reputable experts are frenetically sharing this. Millions of views, public debates. So what to think about this?
To me this looks like a pattern, as old as humanity itself. End-time prophets that rise in times of social and technological disruption. We have been taught over millennia that human ambition that rises to high has to fall eventually - think about mythology like the Tower of Babel or Ikarus.
Every new technology creates its own catastrophe.
But lets dive into the narrative, the messengers and what to take serious and what not.
Claims versus Prophecies
First of all, there is a general problem with these kind of claims - that should better called prophecies. Why? Because, while they look like serious probabilistic and verifiable claims, they are actually unfalsifiable and immune to accountability in the event of fulfillment. If humanity is extincted, there is no one left to prove the case.
This is a type of claim I would always bet against. If humanity did not get extinct you win. If you loose the bet, it does not matter, because there is no one around to settle.
In addition, I reject the argument usually brought forth by doom sayers, that even the smallest chance of infinite loss justifies infinite precaution. If we would adhere to such logic we should better not get out of bed, and in consequence we would end up in total paralysis.
Who Benefits - The Economy of Alarmism
In case I hear something of this level of alarm and disturbance, I always ask myself if there are there any hidden interests that might influence the message, its framing and timing.
Cui bono? (Who benefits?).
What sticks out is that the loudest voices are people associated with (for-profit) frontier labs. This implies a natural interest in gaining attention, either for their organizations or themselves in order to promote their products, skills and foresight.
Still there might be some truth and just the way of framing is intentionally exaggerated.
Counter Arguments
However, there are several logical flaws to the argument. Advanced algorithms are accounted with human attributes like will, intent or will to power. Similar to how human attributes are given to humanoid robots. Several things do not add up:
Intelligence ≠Autonomy: A system can be very capable in solving a specific problem, without having a desire, ego or self-preservation. These are biological traits born from evolutionary pressure to compete for scarce resources. An algorithm (even advanced machine learning or AI) usually does what it is programmed for. And in case there is unexpected behavior - we used to call this a bug or malfunction and not intelligence.
Pragmatism: It is naiv to think that the existence of a real threat would not eventually alter behavior in order to make the outcome far less likely to happen. As we have seen with nuclear, this mobilized resources to control and restrict access. The prediction itself reduces its chance of fulfillment.
Risk and Uncertainty: Assigning a number of 10% probability creates the impression of a quantifiable, known risk assessment. In reality nobody knows, it is unknown territory. A number is just misleading or reflecting an intent and mood and not an objective probability.
Systemic Thinking: It is naive to isolate the growth of capabilities in one domain only. Capability and control (and understanding) advance together, not in isolation. Assuming AI capabilities grow exponentially, while understanding and control remain static does not make sense. Of course a modern soldier could defeat a knight equipped with sword and shield. Of course an advanced, dynamic AI algorithm can penetrate an old, static firewall. But we have reasons to assume that the firewall progresses as well.
Super Intelligence Paradox: It is almost comical how linear thinking is coupled with exponential predictions. Why would a system that achieves super intelligence think in a way as the most stupid humans do - how to harm others, how to promote itself at the cost of others? In case a system would ever achieve this level of superior intelligence, it is likely to say that it would think and act differently. If it maintains a goal of self-optimization and augmentation (which would be a precondition to fight for scarce resources), it likely would pursue more rewarding and productive tasks than eliminating humanity.
Hence these prophecies leave a bad taste in your mouth. Also there are striking similarity to entertaining science fiction stories in Dune or Terminator. The message reveals more about the thinking and value system of the teller, than it does about the system it claims to make predictions about. People driven by greed, competition and a will to dominate.
The Risk is the Mind and Not the Tool
Of course there are (immense) risks that come with every powerful technology. And AI is not an exception. However, the dangers will likely be a result of human design and way of application and not machine malice.
Some implications might not be obvious, but could have serious consequences, like competition for scarce resources (energy, water, rare materials) needed to power advanced compute capabilities. But also the consequences of economic concentration that is a consequence of advanced automation and wealth and capabilities accumulating with a small elite of tech monopolies and state actors. Also AI technologies can lead to pervasive surveillance, rendering people transparent and susceptible to manipulation and repression.
The existential debate on AI capabilities overshadows a much simpler reality:
AI is an amplifier of human intent.
The central challenge of the next decade will not be to prevent super intelligence or to be fearful of an AI wishing to kill us, but to prevent concentrated interests from using pervasive automation and AI to undermine human autonomy, safety, free will and integrity.