Edit: At the suggestion of a commenter I am including some definitions for clarity.
Existential Risk (x-risk) [Bostrom's definition]: An existential risk is one that threatens the premature extinction of Earth-originating intelligent life or the permanent and drastic destruction of its potential for desirable future development.
Extinction risk: Future events that threatens the premature extinction of Earth-originating intelligent life.
S-risks: Future events with the potential capacity to produce an astronomical amount of suffering.
I feel like people sometimes use x-risk when they mean extinction risk because they want to use an acronym. Plus it would be nice for extinction risk to have an acronym anyway (if it doesn't already - I did a quick search and couldn't find anything).
Some possible candidates:
- e-risk (obvious choice but could be confusing since existential starts with e)
- d-risk (death, or disappearance)
- a-risk (annihilation)
Feel free to propose a different acronym, or why this is a bad idea.
So what exactly is wrong with x-risk? I feel like it has already been pretty well established?
Presumably he means because x-risk is short for 'existential risk' and can refer to things other than extinction.
for transparency I strong agreement downvoted you for the reason sam said.
I think x-risk should remain as meaning extinction risk, but "doom" better encompasses what people refer to when they use x-risk colloquially. Doom encompasses x-risk (extinction risk), any permanent drastic curtailment of potential (permanent stagnation or totalitarian lock-in), and s-risk (suffering risk; fates worse than extinction).
EDIT: I fell prey to the original confusion. x-risk was originally existential risk, which encompasses extinction risk and permanent drastic curtailment of future potential. Perhaps x-risk is better as just extinction risk, and ex-risk should be used for existential risk? Doom is still useful as a broader term for common usage (also including s-risk).
Would be good if OP included a definition of existential risk/x-risk!
I really like this! doom feels right for existential risks and x risk definitely could be used for extinction risk since it also has the phonetic x. However I disagree with the notion that x-risk already means extinction risk, see this paper for instance, or Sam's comment.
Cool, and thanks for editing the OP. I think the definition of s-risk you give is off though. S-risks are risks of suffering on an astronomical scale (e.g. a future filled with tortured beings). They are sometimes classed as a sub-category of existential risk, but given they are (much) worse than extinction or a drastic curtailment of potential, they are also often talked about separately.
Wait so something can be an S-risk but not an X-risk?
Let me know if you agree with the following:
extinction risk but not existential risk: Something that kills everything on earth but it turns on the future would have been worse if we survived
S-risk but not existential risk: Factory Farming
S-risk and existential risk: Stable totalitarianism
Existential risk but not S-risk or extinction risk: Permanent civilization collapse.
All three: AI kills us all and then runs horrible simulations
Yes, I agree with that, although perhaps not your first example (I would say that is an existential risk), and for the second one (Factory Farming), to be astronomical in size and therefore an s-risk it would have to be "Space colonisation locking in Factory Farming on a cosmic scale".
Interesting... So the first scenario is an x-risk that we would want to increase.
edit: I had sort of written about this in a separate post but I was told that scenario one is extinction risk and not x-risk. Slightly confused but I agree with you based on the definition of x-risk I used.
This is getting into button-pushing territory. "Would have wanted to increase were we certain of the future being much worse", maybe. But I don't think we can ever be certain enough.
Re extinction vs existential, I'd say it is both (extinction being a subset of existential).
Perhaps "safety from AGI doom" (or "safety from ASI doom", or "ASI doom-safety") is better than "AGI x-safety"?
I think we should avoid using acronyms where possible.
It makes sense to occasionally use them when you're abbreviating common phrases or names of organisations that would otherwise be long/unwieldy to say or write in full every time. But too often acronyms just needlessly introduce barriers to understanding.
Would you be in favor of phasing out s-risk and x-risk?
There's a somewhat standard argument I've heard before that we shouldn't separate these out because they're already covered by existential risks. For example, this is already a complaint lobbied against s-risks since they're a subtype of x-risk. Trying to focus on a subtype seems like mostly a bid to allow something to rise to higher importance than it would otherwise be under standard x-risk framings.
For example, many people try to claim climate change is an extinction risk because fast, significant climate change could cause an extinction event, but that doesn't mean all of life on Earth dies, just that some large percentage of species are wiped out. Existential risk helps keep the lines clear, since climate change is generally not considered an existential risk since plenty of life would be left around after major climate change since that's what's happened in every past instance of major climate change on Earth.
Not having an acronym for extinction risk seems fine then since I think existential risks matter much more, and we should do what we can as EAs to focus on the highest leverage stuff rather than get distracted by things that might kill lots of species but not wipe out all life, since in comparison this is a lower priority type of risk due to lesser impact.
I don't necessarily think s-risks and extinction risks are strictly subtypes of x-risks (if by subtype you mean subset), although it seems like the community may have a few definitions swirling around for each term.
Does giving something an acronym = trying to focus on? It could explicitly help you focus less on it by clarifying via making it easier to communicate, for example. Even if it adds focus, if it also adds clarity (which it totally may not), there is at least the notion of some tradeoff.
This still seems more important than almost anything else that isn't an x-risk to me. So is the implication here that existential risk is the sole term that gets an acronym? I feel ok about letting the global dev and animal welfare communities have acronyms (conditioning on acronyms being useful) even though one might say they are orders of magnitude less important than x-risk reduction.