Previously: Samotsvety's AI risk forecasts.

Our colleagues at Epoch recently asked us to update our AI timelines estimate for their upcoming literature review on TAI timelines. We met on 2023-01-21 to discuss our predictions about when advanced AI systems will arrive.

Forecasts:

Definition of AGI

We used the following definition to determine the “moment at which AGI is considered to have arrived,” building on this Metaculus question:

The moment that a system capable of passing the adversarial Turing test against a top-5%^[1] human who has access to experts on various topics is developed.

More concretely:

A Turing test is said to be “adversarial” if the human judges make a good-faith attempt to unmask the AI as an impostor, and the human confederates make a good-faith attempt to demonstrate that they are humans.

An AI is said to “pass” a Turing test if at least half of judges rated the AI as more human than at least third of the human confederates.

This definition of AGI is not unproblematic, e.g., it’s possible that AGI could be unmasked long after its economic value and capabilities are very high. We chose to use an imperfect definition and indicated to forecasters that they should interpret the definition not “as is” but “in spirit” to avoid annoying edge cases.

Individual forecasts

Aggregate

Epistemic status:

• For Samotsvety track-record see: https://samotsvety.org/track-record/
  • Note that this track record comes mostly from questions about geopolitics and technology that resolve within 12 months.
• Most forecasters have at least read Joe Carlsmith’s report on AI x-risk, Is “Power-Seeking AI an Existential Risk?”. Those who are short on time may have just skimmed the report and/or watched the presentation. We discussed the report section by section over the course of a few weekly meetings.
• Note also that there might be selection effects at the level of which forecasters chose to participate in this exercise; for example, Samotsvety forecasters who view AI as an important/interesting/etc. topic could have self-selected into the discussion.
  • (Though, the set of forecasters who participated this time and participated last time is very similar.)

Update from our previous estimate

The last time we publicly elicited a similar probability from our forecasters, we were at 32% that AGI would be developed in the next 20 years (so by late 2042); and at 73% that it would be developed by 2100. These are a bit lower than our current forecasts. The changes since then can be attributed to

• We have gotten more time to think about the topic, and work through considerations and counter-considerations, e.g., the extent to which we should fear selection effects in the types of arguments to which we are exposed.
• Some of our forecasters still give substantial weight to more skeptical probabilities coming from semi-informative priors, from Laplace’s law, etc. But overall, we have also moved into being guided by more direct considerations: thinking about how AI progress will look year-by-year, what is the probability of a slowdown, what is the probability of a China/Taiwan war, etc.
• We have also spent significant time interacting with state of the art systems. We have seen other people producing new and exciting applications on top of these models. Just recently, Microsoft was in talks with OpenAI to invest renewed billions. This all gives us another round of information.

Acknowledgments

Thanks to Aaron Ho, Nuño Sempere, Greg Justice, Pablo Stafforini, Vidur Kapur, Misha Yagudin, Jared Leibowich, Tolga Bilge, Jonathan Mann, and Eli Lifland. for contributing to the discussion and/or submitting forecasts.

1. ^{^}

    A person who answers a range of general and specific knowledge questions better than 95% of the general population.

2. ^{^}

    P(AGI by ∞) is between 80% and 90% due to non-recoverable catastrophic risks, though this is very uncertain.

3. ^{^}

    P(AGI by ∞) is between 80% and 90% due to x-risks other than AI misalignment. They note that P(x-risks) goes up with more advanced AI systems being available and used (e.g., used in nuclear command & control).

4. ^{^}

    This depends significantly on whether China will blockade or invade Taiwan in a relevant timeframe. We think probability conditional on no Cross-Strait troubles is roughly 20% more likely, but this is highly uncertain. See also http://metaforecast.org/?query=taiwan.

5. ^{^}

    Just mean ± 1.96 stdev.

6. ^{^}

    “When pooling forecasts, use the geometric mean of odds."