If Anyone Builds It, Everyone Dies
The Case Against Superintelligent AI
Eliezer Yudkowsky and Nate Soare
Having read it
★★★★★
Clear, honest and level-headed in its ideas and perspectives.
There are humans out there who will give Als power at the first opportunity, and who are already doing so, and who are unlikely to stop as Als get smarter. Some of them will get even more enthusiastic as the Als get power, and egg them on twice as hard if they act weird and ominous and mysterious. We doubt it will be hard for Als in real life to find enthusiastic assistance.
It is also, despite the naysayers against AI luddites no doubt saying it’s (mostly) unfounded, not very good etc. etc., refreshingly direct in its aims and, being an easy and informative read (certainly because of its links to further material and insights that expand and continue the conversations around artificial superintellegence) is well worth the (reading and thinking) time invested.
Can Earth survive if only some people do their part? Perhaps; perhaps not.
We have heard many people say that it’s not possible to stop Al in its tracks, that humanity will never get its act together. Maybe so. But a surprising number of elected officials have told us that they can see the danger themselves, but cannot say so for fear of the repercussions. Wouldn’t it be silly if really almost none of the decision-makers wanted to die of this, but they all thought they were alone in thinking so?
Where there’s life, there’s hope.
Plus, the book’s closing words are spot on.
A good passage
The world is not divided into a fake Digital Realm and a real Material Realm. Building a factory using the ripple effects from electrical signals in a computer is not fundamentally different from building a factory using the ripple effects from electrical signals in a biological brain. What a human can do depends on what they can affect with their hands. What an AI can do depends on what the AI can affect with devices that are connected to the internet, such as, for example, humans.
The internet is a rich and complicated setting. It’s connected to billions of phones, computers, and humans. It therefore offers billions of opportunities to affect the wider world.
Humanity is integrating Al into its economy at every opportunity. Elon Musk says his robot company will build a few hundred million or a billion robots and train Als to steer them around. Microsoft and Apple have declared their intention to integrate AI deeply into their devices.
A second good passage
Intelligences don’t need to be given a lot of power and resources to become dangerous. Humans started out naked in the savannah, and figured out how to exploit reality and compound advantages until they were building guns and nuclear weapons and supercomputers. An artificial superintelligence would be even more resourceful, at even greater speeds. It would have no limits but the laws of physics.
A third good passage
When a disaster is unthinkable ? when authority ngures insist with conviction that it’s not allowed to happen, when it’s not part of the usual scripts ? then human beings have difficulty believing in the disaster even after it has begun; even when the ship beneath their feet is taking on water.
This is the normal way humanity learns to surmount challenges: We deny the problem, reality smacks us around a bit, and then we start treating the problem with more respect. The Titanic sank, and most people who were aboard died. But nowadays passenger ships have enough lifeboats, and nowadays if the captain said to board them then you’d board them. We don’t hype ships up as unsinkable anymore. We make a mistake the first time, and learn from it the second time.
With ASI, there is no second time.