Yeah again I just think this depends on one's definition of EA, which is the point I was trying to make above.

Many people have turned away from EA, both the beliefs, institutions, and community in the aftermath of the FTX collapse. Even Ben Todd seems to not be an EA by some definitions any more, be that via association or identification. Who is to say Leopold is any different, or has not gone further? What then is the use of calling him EA, or using his views to represent the 'Third Wave' of EA? 

I guess from my PoV what I'm saying is that I'm not sure there's much 'connective tissue' between Leopold and myself, so when people use phrases like "listen to us" or "How could we have done" I end up thinking "who the heck is we/us?"

I'm not sure to what extent the Situational Awareness Memo or Leopold himself are representatives of 'EA'

In the pro-side:

• Leopold thinks AGI is coming soon, will be a big deal, and that solving the alignment problem is one of the world's most important priorities
• He used to work at GPI & FTX, and formerly identified with EA
• He (probably almost certainly) personally knows lots of EA people in the Bay

On the con-side:

• EA isn't just AI Safety (yet), so having short timelines/high importance on AI shouldn't be sufficient to make someone an EA?^[1]
• EA shouldn't also just refer to a specific subset of the Bay Culture (please), or at least we need some more labels to distinguish different parts of it in that case
• Many EAs have disagreed with various parts of the memo, e.g. Gideon's well received post here
• Since his EA institutional history he moved to OpenAI (mixed)^[2] and now runs an AGI investment firm.
• By self-identification, I'm not sure I've seen Leopold identify as an EA at all recently.

This again comes down to the nebulousness of what 'being an EA' means.^[3] I have no doubts at all that, given what Leopold thinks is the way to have the most impact he'll be very effective at achieving that.

Further, on your point, I think there's a reason to suspect that something like situational awareness went viral in a way that, say, Rethink Priorities Moral Weight project didn't - the promise many people see in powerful AI is power itself, and that's always going to be interesting for people to follow, so I'm not sure that situational awareness becoming influential makes it more likely that other 'EA' ideas will

1. ^{^}

   Plenty of e/accs have these two beliefs as well, they just expect alignment by default, for instance

2. ^{^}

   I view OpenAI as tending implicitly/explicitly anti-EA, though I don't think there was an explicit 'purge', I think the culture/vision of the company was changed such that card-carrying EAs didn't want to work there any more

3. ^{^}

   The 3 big defintions I have (self-identification, beliefs, actions) could all easily point in different directions for Leopold

I sort-off bounced of this one Richard. I'm not a professor of moral philosophy, so some of what I say below may seem obviously wrong/stupid/incorrect - but I think that were I a philosophy professor I would be able to shape it into a stronger objection than it might appear on first glance.

Now, when people complain that EA quantifies things (like cross-species suffering) that allegedly “can’t be precisely quantified,” what they’re effectively doing is refusing to consider that thing at all.

I don't think this would pass an ideological Turing Test. I think what people who make this claim are saying is often that previous attempts to quantify the good precisely have ended up having morally bad consequences. Given this history, perhaps our takeaway shouldn't be "they weren't precise enough in their quantification" and should be more "perhaps precise quantification isn't the right way to go about ethics".

Because the realistic alternative to EA-style quantitative analysis is vibes-based analysis: just blindly going with what’s emotionally appealing at a gut level.

Again, I don't think this is true. Would you say that before the publication of Famine, Affluence, and Morality that all moral philosophy was just "vibes-based analysis"? I think, instead, all of moral reasoning is in some sense 'vibes-based' and the quantification of EA is often trying to present arguments for the EA position.

To state it more clearly, what we care about is moral decision-making, not the quantification of moral decisions. And most decisions that have been made or have ever been made have been done so without quantification. What matters is the moral decisions we make, and the reasons we have for those decisions/values, not what quantitative value we place on said decisions/values.

the question that properly guides our philanthropic deliberations is not “How can I be sure to do some good?” but rather, “How can I (permissibly) do the most (expected) good?”

I guess I'm starting to bounce of this because I now view this as a big moral commitment which I think goes beyond simple beneficentrism. Another view, for example, would be a contractualism, where what 'doing good' means is substantially different from what you describe here, but perhaps that's a base metaethical debate.

It’s very conventional to think, “Prioritizing global health is epistemically safe; you really have to go out on a limb, and adopt some extreme views, in order to prioritize the other EA stuff.” This conventional thought is false. The truth is the opposite. You need to have some really extreme (near-zero) credence levels in order to prevent ultra-high-impact prospects from swamping more ordinary forms of do-gooding.

I think this is confusing two forms of 'extreme'. Like in one sense the default 'animals have little-to-no moral worth' view is extreme for setting the moral value of animals so low as to be near zero (and confidently so at that). But I think the 'extreme' in your first sentence refers to 'extreme from the point of view of society'.

Furthermore, if we argue that quantifying expected value in quantitative models is the right way to do moral reasoning (as opposed to sometimes being a tool), then you don't have to accept the "even a 1% chance is enough", I could just decline to find a tool that produces such dogmatism at 1% acceptable. You could counter with "your default/status-quo morality is dogmatic", which sure. But it doesn't convince me to accept strong longtermism any more, and I've already read a fair bit about it (though I accept probably not as much as you).

While you’re at it, take care to avoid the conventional dogmatism that regards ultra-high-impact as impossible.

One man's "conventional dogmatism" could be reframed as "the accurate observation that people with totalising philosophies promising ultra-high-impact have a very bad track record that have often caused harm and those with similar philosophies ought to be viewed with suspicion"

Sorry if the above was a bit jumbled. It just seemed this post was very unlike your recent Good Judgement with Numbers post, which I clicked with a lot more. This one seems to be you, instead of rejecting the ‘All or Nothing’ Assumption, actually going "all in" on quantitative reasoning. Perhaps it was the tone with which it was written, but it really didn't seem to actually engage with why people have an aversion to over-quantification of moral reasoning.

Thanks for sharing your thoughts. I'll respond in turn to what I think are the two main parts of it, since as you said this post seems to be a combination of suffering-focused ethics and complex cluelessness.

On Suffering-focused Ethics: To be honest, I've never seen the intuitive pull of suffering-focused theories, especially since my read of your paragraphs here seems to tend towards a lexical view where the amount of suffering is the only thing that matters for moral consideration.^[1] 

Such a moral view doesn't really make sense to me, to be honest, so I'm not particularly concerned by it, though of course everyone has different moral intuitions so YMMV.^[2] Even if you're convinced of SFE though, the question is how best to reduce suffering which hits into the clueless considerations you point out.

On complex cluelessness: On this side, I think you're right about a lot of things, but that's a good thing not a bad one!

• I think you're right about the 'time of perils' assumption, but you really should increase your scepticism of any intervention which claims to have "lasting, positive effects over millennia" since we can't get the feedback on the millennia long impact of our interventions.
• You are right that radical uncertainty is humbling, and it can be frustrating, but it is also the state that everyone is in, and there's no use beating yourself up for the default state that everyone is in.
• You can only decide how to steer humanity toward a better future with the knowledge and tools that you have now. It could be something very small, and doesn't have to involve you spending hundreds of hours trying to solve the problems of cluelessness.

I'd argue that reckoning with the radical uncertainty should point towards moral humility and pluralism, but I would say that since that's the perspective in my wheelhouse! I also hinted at such considerations in my last post about a Gradient-Descent approach to doing good, which might be a more cluessness-friendly attitude to take.

1. ^{^}

   You seem to be asking e.g. "will lowering existential risk increase the expected amount of future suffering" instead of "will lowering existential risk increase the amount of total preferences satisfied/non frustrated" for example.

2. ^{^}

   To clairfy, this sentence specifically referred to lexical suffering views, not all forms of SFE that are less strong in their formulation

Edit: Confused about the downvoting here - is it a 'the Forum doesn't need more of this community drama' feeling? I don't really include that much of a personal opinion to disagree with, and I also encourage people to check out Lincoln's whole response 🤷

For visibility, on the LW version of this post Lincoln Quirk - member of the EV UK board made some interesting comments (tagging @lincolnq to avoid sub-posting). I thought it'd be useful to have visibility of them on the Forum. A sentence which jumped out at me was this:

Personally, I'm still struggling with my own relationship to EA. I've been on the EV board for a year+ - an influential role at the most influential meta org - and I don't understand how to use this role to impact EA.

If one of the EV board members is feeling this way and doesn't know what to do, what hope for rank-and-file EAs? Is anyone driving the bus? Feels like a negative sign for the broader 'EA project'^[1] if this feeling goes right to the top of the institutional EA structure.

That sentence comes near the end of a longer, reflective comment, so I recommend reading the full exchange to take in Lincoln's whole perspective. (I'll probably post my thoughts on the actual post sometime next week)

1. ^{^}

   Which many people reading this might feel positive about

A thought about AI x-risk discourse and the debate on how "Pascal's Mugging"-like AIXR concerns are, and where this causes confusion between those concerned and sceptical.

I recognise a pattern where a sceptic will say "AI x-risk concerns are like Pascal's wager/are Pascalian and not valid" and then an x-risk advocate will say "But the probabilities aren't Pascalian. They're actually fairly large"^[1], which usually devolves into a "These percentages come from nowhere!" "But Hinton/Bengio/Russell..." "Just useful idiots for regulatory capture..." discourse doom spiral.

I think a fundamental miscommunication here is that, while the sceptic is using/implying the term "Pascallian" they aren't concerned^[2] with the percentage of risk being incredibly small but high impact, they're instead concerned about trying to take actions in the world - especially ones involving politics and power - on the basis of subjective beliefs alone. 

In the original wager, we don't need to know anything about the evidence record for a certain God existing or not, if we simply Pascal's framing and premisses then we end up with the belief that we ought to believe in God. Similarly, when this term comes up, AIXR sceptics are concerned about changing beliefs/behaviour/enact law based on arguments from reason alone that aren't clearly connected to an empirical track record. Focusing on which subjective credences are proportionate to act upon is not likely to be persuasive compared to providing the empirical goods, as it were.

1. ^{^}

   Let's say x>5% in the rest of the 21st century for sake of argument

2. ^{^}

   Or at least it's not the only concern, perhaps the use of EV in this way is a crux, but I think it's a different one

Something which has come up a few times, and recently a lot in the context of Debate Week (and the reaction to Leif's post) is things getting downvoted quickly and being removed from the Front Page, which drastically drops the likelihood of engagement.^[1]

So a potential suggestion for the Frontpage might be:

• Hide the vote score of all new posts if the absolute score of the post is below some threshold (I'll use 20 as an example)
  • If a post hits -20, it drops off the front page
  • After a post hits 20+, it's karma score is permanently revealed
  • Galaxy-brain version is that Community/Non-Community grouping should only take effect once a post hits these thresholds^[2]
• This will still probably leave us with too many new posts to fit on the front page, so some rules to sort which stay and which get knocked off:
  • Some consideration to total karma should probably count (how much to weight it is debatable)
  • Some consideration to how recent the post is should count too (e.g. I'd probably want to see a new post that got 20+ karma quickly than 100+ karma over weeks)
  • Some consideration should also go to engagement - some metric related to either number of votes or comment count would probably indicate which posts are generating community engagement, though this could lead to bikeshedding/Matthew effect if not implemented correctly. I still think it's directionally correct though
  • Of course the user's own personal weighting of topic importance can probably contribute to this as well
• There will always be some trade-offs when designing some ranking on many posts with limited space. But the idea above is that no post should quickly drop off the front page because a few people quickly down-vote it into negative karma.

Maybe some code like this already exists, but this thought popped into my head and I thought it was worth sharing on this post.

1. ^{^}

   My poor little piece on gradient descent got wiped out by debate week 😭 rip

2. ^{^}

   In a couple of places I've seen people complain about the use of the Community tag to 'hide' particular discussions/topics. Not saying I fully endorse this view.

I think 'meat-eating problem' > 'meat-eater problem' came in my comment and associated discussion here, but possibly somewhere else.^[1]

1. ^{^}

   (I still stand by the comment, and I don't think it's contradictory with my current vote placement on the debate week question)

On the platonic/philosophical side I'm not sure, I think many EAs weren't really bought into it to begin with and the shift to longtermism was in various ways the effect of deference and/or cohort effects. In my case I feel that the epistemic/cluelessness challenge to longtermism/far future effects is pretty dispositive, but I'm just one person.

On the vibes side, I think the evidence is pretty damning:

• The launch of WWOTF was almost perfectly at the worst time possible and the idea seems indelibly linked with SBF's risky/naïve ethics and immoral actions.
• Do a Google News or Twitter search for 'longtermism' in its EA context and it's ~broadly to universally negative. The Google trends data also points toward the term fading away.
• No big EA org or "EA leader" however defined is going to bat for longtermism any more in the public sphere. The only people talking about it are the critics. When you get that kind of dynamic, it's difficult to see how an idea can survive.
• Even on the Forum, very little discussion on the Forum seems to be based on 'longtermism' these days. People either seem to have left the Forum/EA, or longtermist concerns have been subsumed into AI/bio risk. Longtermism just seems superfluous to these discussions.

That's just my personal read on things though. But yeah, seems very much like that SBF-Community Drama-OpenAI board triple whammy from Nov22-Nov23 marked the death knell for longtermism at least as the public facing justification of EA.

For the avoidance of doubt, not gaining knowledge from the Carl Shulman episodes is at least as much my fault as it is Rob and Carl's!^[1] I think similar to his appearance on the Dwarkesh Podcast, it was interesting and full of information, but I'm not sure my mind has found a good way to integrate it into my existing perspective yet. It feels unresolved to me, and something I personally want to explore more, so a version of the post written later in time might include those episodes high up. But writing this post from where I am now, I at least wanted to own my perspective/bias leaning against the AI episodes rather than leave it implicit in the episode selection. But yeah, it was very much my list, and therefore inherits all of my assumptions and flaws.

I do think working in AI/ML means that the relative gain of knowledge may still be lower in this case compared to learning about the abolition of slavery (Brown #145) or the details of fighting Malaria (Tibenderana #129), so I think that's a bit more arguable, but probably an unimportant distinction.

1. ^{^}

   (I'm pretty sure I didn't listen to part 2, and can't remember how much I listened to of part 1 over reading some of the transcript on the 80k website, so these episodes may be a victim of the 'not listened to fully yet' criteria)