Wave

Every now and then within the book (and more so toward its middle and end), The Coming Wave by Mustafa Suleyman, some good insights and perspectives are explored and explained.

Aside from a self-inflicted downfall – potentially a beginning of the end of our planet and species! – through AI, synthetic biology and quantum computing et al. possibly happening partly because of their power, convenience, coolness or other excuse, we do and should be active participants (that learn, critique and debate this technology and its ethics and morals) in this new and growing world within the world to at least stand a chance of not being surplus to requirements.

There might be a chance…

Nationalism, capitalism, and science – these are, by now, embedded features of the world. Simply removing them from the scene is not possible in any meaningful time frame. Altruism and curiosity, arrogance and competition, the desire to win the race, make your name, save your people, help the world, whatever it may be: these are what propel the wave on, and these cannot be expunged or circumvented.

Moreover, these different incentives and elements of the wave compound. National arms races dovetail with corporate rivalries while labs and researchers spur each other on. A nested series of sub-races, in other words, adds up to a complex, mutually reinforcing dynamic. Technology ‘emerges’ through countless independent contributions all layering on top of one another, a metastasizing, entangled morass of ideas unraveling themselves, driven on by deep-rooted and dispersed incentives.

Without tools to spread information at light speed, people in the past could happily sit with new technologies staring them in the face sometimes for decades before they realised their full implications. And even when they did, it would take a lot of time, and ultimately imagination, to fully realize the broad ramifications. Today the world is watching everyone else react in real time.

Everything leaks. Everything is copied, iterated, improved. And because everyone is watching and learning from everyone else, with so many people all scratching around in the same areas, someone is inevitably going to figure out the next big breakthrough. And they will have no hope of containing it, for even if they do, someone else will come behind them and uncover the same insight or find an adjacent way of doing the same thing; they will see the strategic potential or profit or prestige and go after it.

This is why we won’t say no. This is why the coming wave is coming, why containing it is such a challenge. Technology is now an indispensable mega-system infusing every aspect of daily life, society, and the economy. No one can do without it. Entrenched incentives are in place for more of it, radically more. No one is in full control of what it does or where it goes next. This is not some far-out philosophical concept or extreme determinist scenario or wild-eyed California technocentrism. It is a basic description of the world we all inhabit, indeed the world we have inhabited for quite some time.

In this sense it feels like technology is, to use an unforgiving image, one big slime mold slowly rolling toward an inevitable future, with billions of tiny contributions being made by each individual academic or entrepreneur without any coordination or ability to resist. Powerful attractors pull it on. Where blocks appear, gaps open elsewhere, and the whole rolls forward. Slowing these technologies is antithetical to national, corporate, and research interests.

This is the ultimate collective action problem. The idea that CRISPR [clustered regularly interspaced short palindromic repeats] or Al can be put back in the box is not credible. Until someone can create a plausible path to dismantling these interlocking incentives, the option of not building, saying no, perhaps even just slowing down or taking a different path isn’t there.

Containing technology means short-circuiting all these mutually reinforcing dynamics. It’s hard to envisage how that might be done on any kind of timescale that would affect the coming wave. There is only one entity that could, perhaps, provide the solution, one that anchors our political system and takes final responsibility for the technologies society produces: the nation-state.

But there’s a problem. States are already facing massive strain, and the coming wave looks set to make things much more complicated. The consequences of this collision will shape the rest of the century.

...and there might not be...

By creating something smarter than us, we could put ourselves in the position of our primate cousins. With a long-term view in mind, those focusing on AGI scenarios are right to be concerned. Indeed, there is a strong case that by definition a super-intelligence would be fully impossible to control or contain. An ‘intelligence explosion’ is the point at which an Al can improve itself again and again, recursively making itself better in ever faster and more effective ways. Here is the definitive uncontained and uncontainable technology. The blunt truth is that nobody knows when, if, or exactly how Als might slip beyond us and what happens next; nobody knows when or if they will become fully autonomous or how to make them behave with awareness of and alignment with our values, assuming we can settle on those values in the first place.

Nobody really knows how we can contain the very features being researched so intently in the coming wave. There comes a point where technology can fully direct its own evolution; where it is subject to recursive processes of improvement; where it passes beyond explanation; where it is consequently impossible to predict how it will behave in the wild; where, in short, we reach the limits of human agency and control.

Ultimately, in its most dramatic forms, the coming wave could mean humanity will no longer be at the top of the food chain. Homo technologicus may end up being threatened by its own creation. The real question is not whether the wave is coming. It clearly is; just look and you can see it forming already. Given risks like these, the real question is why it’s so hard to see it as anything other than inevitable.

...but all of us taking responsibility responsibly, ethically and safely will be very important and helpful...

The fact that technology’s incentives are unstoppable does not mean that those building it bear no responsibility for their creations. On the contrary, they, we, I, do; the responsibility is crystal clear. No one is compelled to experiment with genetic modification or build large language models. Technology’s inevitable spread and development are not a get-out-of-jail-free card, a license to build what you want and see what happens. They are rather a hammering reminder of the need to get things right and the awful consequences of not doing so.