Introduction: Economics, the curse of dimensionality and reinforcement learning

Theoretical economics is the science of interaction among optimizing agents. Before the Cambrian explosion in reinforcement learning (RL), optimization was a hard mathematical problem: typically, when a dynamic optimization problem included more than five or six free variables, it became intractable: this is named “the curse of dimensionality”. For reasons I have never grasped, quantitative economists almost never surrender the optimizing behavior of economic agents, and consequently often they mutilate and twist the economic realism of the models to keep the tractability of optimal behavior at all costs.

Fortunately, optimization is no longer hard nor expensive. RL allows to populate any economic world we can imagine with superhuman optimizers. The painful algebra of Benveniste-Scheinkman conditions, envelope theorems and perturbative methods is now a barbaric relic. A Brave New World of modelling freedom opens under our eager eyes.

Now, cheap optimization does not mean that economics is no longer interesting. Economists still have to develop interactive settings (let’s name them “games”) and compare the equilibrium (=ergodic distribution) properties of the game under consideration with some real world phenomenon of interest. 

This introduction has a single goal: to direct the attention of the reader into a critical detail in the development of Artificial Intelligence: intelligence is not only a property of the optimizing kernel of a system: it emerges in the interaction between optimizing agent and the world it inhabits. 

The Moravec hypothesis and virtual reality

This has been well known since the 1990s. In “Mind Children”, the AI pioneer Hans Moravec provided both an explanation for the failure of symbolist AI and a road map for AI development. Moravec suggested that you cannot make AI in a symbolic world: to develop recognizably intelligent behavior you need a complete immersion in reality: Moravec, consequently suggested that robotics had to be developed before AI was created. 

Now, robots are too expensive for the kind of massive training that underlie in RL. But virtual worlds are cheap to make and populate. Virtual reality and Artificial Intelligence are dual technologies that have to be developed concurrently. If I had to value the relevant assets for AI development, the Open AI or Alpha routines and datasets would not be more important than the virtual world development tools owned by (v.g.) Epic Games.

Review of canonical AI risk research from the Moravec hypothesis perspective

After my previous note on the marginal contribution of AI-risk to total Existential Risk, I decided to review the canonical literature on AI risk. I read the Eliezer Yudkowsky FAQ, and two additional papers (“The Alignment Problem from a Deep Learning Perspective” and “Is Power-Seeking AI an Existential Risk?”) that were kindly suggested by Jackson Wagner. 

I had not read before about AI risk because I expected a very technical literature, only suitable for AI developers. This is not the case: if you know dynamic macroeconomics (in particular dynamic programming) and you have been a heavy science fiction reader the main arguments in the papers above are not only accessible but also familiar. 

And those arguments are simple and persuasive: First, intelligence is what gives us our ecological supremacy, and by growing an artificial intelligence we risk losing it. Secondly, we “grow”, do not “design” AI, and we really do not understand what we are building. Finally, any truly intelligent agent will tend to accumulate power as an intermediate goal to deploy that power for its final goals.

I find those arguments persuasive, but unspecific: AI is by far the most uncertain technology ever produced: currently, we don’t know even if AGI is feasible (a new AI winter is perfectly possible), and if it is feasible, it can be heaven or it can be hell. In my view, long chains of reasoning about superintelligence tend to be mainly unreliable, given the massive uncertainty involved.

But the main problem with catastrophist AI arguments is that current AI tools are far away from AGI level by construction. Let’s take the crown jewel: chat GPT. Chat GTP does not “understand” what it says, because it has been mainly trained with texts. The universe chat GPT inhabits is made of words, and for chat GTP words are not related to their external world referents (because chat GTP does not live in the physical world). Words, for chat GTP are linked and clustered to other words. For chat GTP Language is a self-referential reality, and it is an incredible predator in that ocean made of interlinked words, not in the bio-physical reality made of flows of energy, material cycles and gene pool competition.

Chat GTP is not as alien as a giant squid, but far more: it has not been even trained for self-preservation. Its universe and goals are totally orthogonal to ours.  All the AI systems developed so far are extremely specific and no matter how powerful is the underlining optimizing/agentic technology, they live in very constrained realities with goals idiosyncratic to those realities. Currently, AI models are not as animals, but only as specific brain tissues. 

Until AIs are immersed in the real world or in a virtual world designed for realism, and their goals are substantially based on that realist virtual world, AGI is not close (no matter how powerful is the core pseudo-neural technology), and existential AI-risk is too low to measure.

The relative paucity of the current alignment literature derives from the reality that we are too far away from AGI and its real technical challenges. We cannot “solve” the alignment problem in a blackboard; AI development and the risk control measures to deal with it shall be developed in parallel. In fact, there is not any “alignment problem” to be “solved”. There is an AGI development problem where (among other challenges) AI existential risk shall be monitored and addressed.

Any “pause” in the current stage of AI development would be obviously useless, at least if you are interested in the “controlled development” of AGI. 

All together now

Now, I want to summarize the results of my little trip into AI risk. 

First of all, if feasible, AI is an extremely hard to ban technology. There is nothing as the “enrichment” bottleneck that makes nuclear proliferation a difficult industrial challenge. A consequential ban on AI development would imply worldwide draconian restrictions on the research and use of IT. There shall be an extremely strong reason for such a massive decision. But AI alignment literature is not truly technical. It is mainly based in high level visions, philosophical positions and other non-operational, non-technical arguments. 

If your assessment of risk (even informed by that kind of arguments) is extremely high, you can ask for a complete ban of the technology. But in my view, a total ban is impossible, and were it possible, it would imply a massive slowdown of technological progress in general.  Now, without progress we are left in age of acute nuclear war risk with nothing else than our primitive social systems and some environmental and social crises to deal with. 

Nuclear war is not existential in the “one-off” sense: even a NATO-Russia full exchange in the worst nuclear winter case would not kill everybody. But what kind of societies would be left after the first major nuclear war? Military aristocracies, North Korea like totalitarian regimes, large tracts of Somalian anarchy awaiting to be invaded by their imperialist neighbors, etc. Nothing else can keep political coherence after such a shock.

To simplify, suppose one thousand years are needed to recover from a major nuclear war, and (given the apparent intractability of the “human alignment” problem) a major nuclear war happens every 150 years. Then Humanity returns to a new kind of Malthusian trap (more specifically, a nuclear fueled Hobbesian trap). In reality I don’t expect a post nuclear war world to be one of one thousand years of recovery and then a major nuclear war (the “Canticle for Leibowitz” typical story), but more a world of totalitarian militarism with frequent nuclear exchanges and the whole society oriented for war. At some point, if AGI is possible, some country will develop it, with the kind of purpose that guarantees it to be Skynet. We have been lucky enough to chain 77 years in a row without a nuclear war. In my view the pre-nuclear war Mankind (more specifically, the democratic countries) is the best suited to develop a benefic AGI.  

Consequently, an AI ban is probably impossible and would be counterproductive. A pause on AI research would be too premature to be useful: we are still far away from AGI, and AI research and AI alignment efforts shall run in parallel. AI alignment shall not be an independent effort, but a part of the AI development. As long as AI alignment researchers do not have more substantial results, they are not legitimized to regulate (far less pause) an infant industry still far away from being risky and that can be bring either extinction or salvation to Mankind.