This piece from Gideon Lewis-Kraus (the writer for the MacAskill piece) is a recent overview of how EA has reacted to SBF and the FTX collapse.
Lewis-Kraus's articles are probably the most in-depth public writing on EA, and he has had wide access to EA members and leadership.
The New Yorker is highly respected and the narratives and attitudes in this piece will influence future perceptions of EA.
This piece contains inside information about discussions or warnings about SBF. It uses interviews from a "senior EA", and excepts from an internal Slack channel used by senior EAs.
When my profile of MacAskill, which discussed internal movement discord about Bankman-Fried’s rise to prominence, appeared in August, Wiblin vented his displeasure on the Slack channel. As he put it, the problem was with the format of such a treatment. He wrote, “They don’t focus on ‘does this person have true and important ideas.’ The writer has no particular expertise to judge such a thing and readers don’t especially care either. Instead the focus is more often on personal quirkiness and charisma, relationships among people in the story, ‘she said / he said’ reporting of disagreements, making the reader feel wise and above the substantive issue, and finding ways the topic can be linked to existing political attitudes of New Yorker readers (so traditional liberal concerns). This is pretty bad because our great virtue is being right, not being likeable or uncontroversial or ‘right-on’ in terms of having fashionable political opinions.”
There are claims of a warning about SBF on the Slack channel:
This past July, a contributor to the Slack channel wrote to express great apprehension about Sam Bankman-Fried. “Just FYSA,”—or for your situational awareness—“said to me yesterday in DC by somebody in gov’t: ‘Hey I was investigating someone for [x type of crime] and realized they’re on the board of CEA’ ”—MacAskill’s Centre for Effective Altruism—“ ‘or run EA or something? Crazy! I didn’t realize you could be an EA and also commit a lot of crime. Like shouldn’t those be incompatible?’ (about SBF). I don’t usually share this type of thing here, but seemed worth sharing the sentiment since I think it is not very uncommon and may be surprising to some people.” In a second message, the contributor continued, “I think in some circles SBF has a reputation as someone who regularly breaks laws to make money, which is something that many people see as directly antithetical to being altruistic or EA. (and I get why!!). That reputation poses PR concerns to EA whether or not he’s investigated, and whether or not he’s found guilty.” The contributor felt this was a serious enough issue to elaborate a third time: “I guess my point in sharing this is to raise awareness that a) in some circles SBF’s reputation is very bad b) in some circles SBF’s reputation is closely tied to EA, and c) there’s some chance SBF’s reputation gets much, much worse. But I don’t have any data on these (particularly c, I have no idea what types of scenarios are likely), though it seems like a major PR vulnerability. I imagine people working full-time on PR are aware of this and actively working to mitigate it, but it seemed worth passing on if not since many people may not be having these types of interactions.” (Bankman-Fried has not been charged with a crime. The Department of Justice declined to comment.)
The suggestion is that EA leadership, while not knowing of any actual crime, accepted poor behavior and norm breaking because of the resources Bankman-Fried provided.
In other words, it seems as though the only thing that truly counts [...] is the inviolate sphere of ideas—not individual traits, not social relationships, not “she said” disagreements about whether it was wise to throw in one’s lot with billionaire donors of murky motive, and certainly not “traditional liberal concerns.”
...
Effective altruism did not create Sam Bankman-Fried, but it is precisely this sort of attitude among E.A.’s leadership, a group of people that take great pride in their discriminatory acumen, that allowed them to downweight the available evidence of his ethical irregularities. This was a betrayal of the E.A. rank and file, which is, for the most part, made up of extremely decent human beings.
What’s worse, however, is what was effectively communicated to Bankman-Fried himself. Ideas in the abstract are influential, but practical social norms constrain what individual actors think they can get away with. The message from E.A. leadership to Bankman-Fried seemed clear: as long as your stated ideals, not to mention your resources, are in alignment with ours, we might not bother ourselves with the other dimensions of your behavior. After all, “our great virtue is being right.” (MacAskill insisted to me that the movement’s leaders have always emphasized acting with integrity.) There was every incentive to look the other way. By the beginning of the summer, Bankman-Fried’s FTX Future Fund, for which MacAskill was serving as an adviser, had promised grants in excess of twenty-eight million dollars to E.A.’s institutional pillars, including C.E.A.; these outfits were the largest recipients of the new foundation’s largesse. It’s not that E.A. institutions were necessarily more irresponsible, or more neglectful, than others in their position would have been; the venture capitalists who worked with Bankman-Fried erred in the same direction. But that’s the point: E.A. leaders behaved more or less normally. Unfortunately, their self-image was one of exceptionalism.
What's stunning to me is the following:
Leaking private slack conversations to journalists is a 101 on how to destroy trust. The response to SBF and FTX betrayal shouldn't be to further erode trust within the community.
EA should not have to learn every single group dynamic from first principles - the community might not survive such a thorough testing and re-learning of all social rules around discretion, trust and why its important to have private channels of communication that you can assume will not be leaked to journalists.
If the community ignores trust, networks and support for one another - then the community will not form, ideas will not be exchanged in earnest and everyone will be looking over their shoulder for who may leak or betray their confidence.
Destroying trust decimates communities - we've all found that with SBF. The response to that shouldn't be further, even more personal and deep betrayals. I will now have to update against how open I am in discussions with other EAs - which is a shame as the intellectual freedom, generosity, honesty and subtlety are what I love about this community - but it seems I will have to consider "what may a journalist think of this if this person leaked it?" as a serious concern.
I don't know about others, but I find it deeply uncomfortable there's an invite-only conference and a private slack channel where, amongst other things, reputational issues are discussed. For one, there's something weird about, on the one hand, saying "we should act with honesty and integrity" and also "oh, we have secret meetings where we discuss if other people are going to make us look bad".
I think its completely fine for invite-only slacks to exist and for them to discuss matters that they might not want leaked elsewhere. If they were plotting murders or were implicated in serious financial crime, or criminal enterprise, or other such awful unforgiveable acts, then yes I can see why we would want to send a clear signal that anything like that is beyond the pale and discretion no longer protects you. In that instance I think no one would object to breach of trust.
However, we aren't discussing that scenario. This is a breach of trust, which erodes honest discussion in private channels. The more this is acceptable in EA circles, the less honesty of opinion you will get - and the more paranoia will set in.
Acting with honesty and integrity does not mean opening up every discussion to the world, or having an expectation that chats will leak in the event that you discuss "if other people are going to make us look bad". Nevermind the difficulty that arises in then attempting to predict what else warrants leaks if that's the bar you've set.
The thing that most keenly worries me about this is the lack of openness and accountability of this. We are a social movement, so of course we will have power dynamics and leadership. But with no transparency or accountability, how can anyone know how to make change?
I think its wrong to say there's no transparency or accountability (this isn't to say we should just assume all checks now are enough, but I don't think we should conclude that none so far exist). Obviously for anything actually criminal then proper whistleblowing paths exist and should be used! At the moment, I think even checks like this discussion are far more effective than in most other communities because EA is still quite small, so it hasn't got the issues of scale that other institutions or communities may experience.
On transparency: Transparency is a part of honesty, but has costs and I don't think its at all clear in this instance that that cost was remotely required to be paid. Again, this will only cause future discussions to be slower, more guarded and less honest - the community response to this will similarly decide how much we should guard ourselves when talking with other EAs. As a side point: its also the case that this instance isn't actual "transparency" but fed lines to a journalist, then selectively quoted and given back to us.
The cost of transparency in every discussion at a high-level of leadership (for example) is that the cost of new ideas becomes prohibitively high as everyone can pick you apart, weigh in, misrepresent or re-direct discussion entirely. Compare e.g. local council meetings with the public and those without, and decisions made in committee vs those made by individual founders. Again transparency is a part of honesty but I can put my trust in you - for example - without needing you to be transparent about every conversation you have about me. If, however, the norm is that we expect total transparency of information and constant leaks - then we should expect a community of paranoia, dishonest conversation and continuous misrepresentations of one another.
I think you may be assuming what I am calling for here is much more wide-ranging. There still doesn't seem to be good justification for not knowing who is in the coordination forum or on these leadership slack channels. Making the structures that actually exist apparent to community members would probably not come at such a prohibitively high cost as you suggest
Yes, I don't know what I think of that, but you're right that I implied you were thinking of something much more wide reaching.
This strikes me as weirdly one-sided. You're against leaking, but presumably, you're in favour of whistleblowing - people being able to raise concerns about wrongdoing. Would you have objected to someone leaking/whisteblowing that e.g. SBF was misusing customer money? If someone had done so months ago, that could have saved billions, but it would have a breach of (SBF's) trust.
The difference between leaking and whistleblowing is ... I'm actually not sure. One is official, or something?
This fundamentally misunderstands norms around whistleblowing. For instance, UK legislation on whistleblowing does not allow you to just go to journalists for any and all issues - they have to be sufficiently serious. This isn't just for "official" reasons but because it's understood that trust within institutions is necessary for a functioning society/group/community/company and norms that encourage paranoia over leaks lead to non-honest conversations filtered through fears of leaks.
Even in the event that a crime is being committed, you are expected to first go to authorities, rather than journalists - to journalists only if you believe authorities won't assist. In the SBF example I'd hope someone would have done precisely that. Moreso, to protect trust, whistleblowing is protected only for issues that warrant that level of trust breach - ie my comment is that this is disproportionate breach of trust, with long term effect on community norms.
Furthermore, whistleblowing on actual crimes is entirely different to leaking private messages about managing PR. And - again - is something one should do first to authorities - not necessarily to journalists!
Essentially you are eliding very serious whistleblowing of crimes to police or public bodies, to leaked screenshots to journalists of private chats about community responses.
I'm a bit confused: Which authorities would the whistleblower report to, especially if they aren't reporting any crimes?
I'd guess the distinction would be more 'public interest disclosure' rather than 'officialness' (after all, a lot of whistleblowing ends up in the media because of inadequacy in 'formal' channels). Or, with apologies to Yes Minister: "I give confidential briefings, you leak, he has been charged under section 2a of the Official Secrets Act".
The question seems to be one of proportionality: investigative or undercover journalists often completely betray the trust and (reasonable) expectations of privacy of its subjects/targets, and this can ethically vary from reprehensible to laudable depending on the value of what it uncovers (compare paparrazzi to Panorama). Where this nets out for disclosing these slack group messages is unclear to me.
Even if you're not concerned about leaks, the possibility of compelled disclosure in a lawsuit has to be considered. So if would be seriously damaging for information to show up in the New Yorker, then phone, in-person, and Inspector Gadget telegram should be the preferred method of communication anyway.
I definitely appreciate the point about trust, just wanted to add that people should consider the risks of involuntary disclosure through legal process (or hacking) before putting stuff in writing.
Editorial/Speculation/Personal comments:
This article might be good and satisfying to many people because it gives a plausible sense of what happened in EA related to SBF, and what EA leaders might have known. The article goes beyond the "press releases" we have seen, does not come from an EA source, and is somewhat authoritative.
Rob Wiblin appears quite a few times and is quoted. In my opinion, he is right and most EAs and regular people would agree with him. New Yorker articles include details to suggest a sense of intimacy and understanding, but the associated narrative is not always substantive or true. This style is what Wiblin is reacting to.
Gideon-Lewis makes some characterizations that don't seem that insightful. As in his last piece, Gideon-Lewis maintains an odd sense of surprise that a large movement with billions of dollars, and a history of dealing with bad actors, has an "inner circle".
Gideon-Lewis has had great access to senior EAs, and inside documents. After weeks of work, there is not that much that he shows he has uncovered, that isn't available after a few conversations or even just available publicly on the EA forum.
My intuitions differ some here. I don't know about Will MacAskill's notion of moral pluralism. But my notion of moral pluralism involves assigning some weight to views even if they're informed by less data or less reflection, and also doing some upweighting on outsider views simply because they're likely to be different (similar to the idea of extremization in the context of forecast aggregation).
If a regular person thinks "our great virtue is being right" sounds like hubris, that's evidence of actual hubris. You don't just replace "our great virtue is being right" with "our focus is being right" because it sounds better. You make that replacement because the second statement harmonizes with a wider variety of moral and epistemic views.
PR is more corrosive than reputation because "reputation" allows for the possibility of observers outside your group who can form justified opinions about your character and give you useful critical feedback on your thinking and behavior.
There's a sense in which reputational harm from a vignette like this is justified. People who read it can reasonably guess that the speaker has few instinctive misgivings to ally with a "semi-fascist" who's buying political power and violating widely held "common sense" American morality.
One would certainly hope that deontological considerations (beyond just PR) would come up at some point, were EA considering an alliance with Thiel. But it concerns me that Lewis-Kraus quotes so much "PR" discussion, and so little discussion of deontological safeguards. I don't see anything here that reassures me ethical injunctions would actually come up.
And instinctive misgivings actually matter, because it's best to nip your own immoral behavior in the bud. You don't want to be in a situation where each individual decision seems fine and you don't realize how big their sum was until the end, as SBF put it in this interview (paraphrased). That's where Lewis-Kraus' references to Schelling fences and "momentum" come in.
The best time to get a "hey this could be immoral" mental alert is as soon as you have the idea of doing the thing. Maybe you do the thing despite the alert. I'm in favor of redirecting the trolley despite the "you will be responsible for the death of a person" alert. But an alert is generally a valuable opportunity to reflect.
Finally, some meta notes:
I doubt I'm the only person thinking along these lines. Matt Yglesias also seems concerned, for example. The paragraphs above are an attempt to steelman what seems to be a common reaction on e.g. Twitter in a way that senior EAs will understand.
The above paragraphs, to a large degree, reflect updates around my thinking about EA which have occurred over the past years and especially the past weeks. My thinking used to be a lot closer to yours and the thinking of the quoted Slack participants.
I've noticed it's easy for me to get into a mode of wondering if I'm a good person and trying to defend my past actions. Generally speaking it has felt more useful to reflect on how I can improve. Growth mindset over fixed mindset, essentially.
That said, I think it is a major credit to EAs that they work so hard to do good. Lack of interest in identifying and solving the world's biggest problems strikes me as a major problem with common-sense morality. So I don't think of EAs in the Slack channel as bad people. I think of them as people working hard to do good, but the notion of "good" they were optimizing for was a bit off. (I used to be optimizing for that thing myself!)
I've also noticed that when experiencing an identity threat (e.g. sunk cost fallacy), it's useful for me to write a specific alternative plan, without committing to it during the process of writing it. This could look like: Make a big list of things I could've done differently... then circle the ones I think I should've done, in hindsight, with the benefit of reflection. Or, if I'm feeling doubtful about current plans, avoid letting those doubts consume me and instead outline one or more specific alternative plans to consider.
This is true, but it's about as good as can be expected since it's an online New Yorker piece. Their online pieces are much closer to blog posts. The MacAskill profile that ran in the magazine was the result of months of reporting, writing, editing, and fact-checking, all with expenses for things like travel.
I've worked to pitch (and in some cases, been the target of) investigative pieces for the last 15 or so years of my life, and honestly, nothing here strikes me as particularly troubling. These are routine errors in communication, or cognitive biases (e.g., salience bias), and probably not indications of any sort of wrongful conduct.
The main thing that I would find concerning in this piece is the excessive focus on PR by EA leaders. Don't focus on PR. Focus on trying to get a true and accurate account out there in the media. It's very hard to manipulate or even strategize about how to portray yourself. It's much easier to be real, because you don't have to constantly perform. That should be a norm within EA, especially among leaders.
I think your point about the various "warning flags" is well-taken. Of course, in retrospect, we've been combing the forums for comments that could have given pause. But the volume of comments is way too large to imagine we would have actually updated enough on a single comment to make a difference.
That said, I think the mass exodus of Alameda employees in 2018 should have been a bigger warning flag, cause for more scrutiny on the business, to the extent where those with a concern for the risks should have tried to dig deeper on those employees, even with the complications that NDAs can pose. We can't say we weren't aware of it -- that episode even made it into SBF's fawning 80k interview, albeit mostly framed as "how do you pick yourself up after hardships?".
The best case scenario conclusion of such an investigation very likely wouldn't have been "SBF is committing massive fraud" especially as that might not have happened until years later. But I think it still would have been useful for the community to know that SBF had a reckless appetite for risk, so we could anticipate at least the potential for FTX to just outright collapse, especially as the crypto industry turned sour earlier this year.
I am open to the idea that private slack messages can be shared to make a point, but it seems that someone just shared a tonne of them (they range across Rob's comments, FTX early warnings etc) and I dislike that - it damages people's ability to communicate freely if they think a chunk of messages are gonna get shared.
What a strange line - did I miss some event where people expelled Will from EA to make ourselves look better? Seems like this would make more sense of SBF (but context rules that interpretation out).
(for reference, the book of Jonah is pretty short and you can read it here)
It's open to interpretation, but I don't think"thrown overboard" is there to suggest much about the EA community, though I'm sure some wish there were a way to distance EA from someone who was so deeply entangled with SBF.
Whatever the case, I think the reference primarily serves to set up the following:
Will's responses in the piece fall far short from "accepting the burden of duty." First, on his propagating the myth of SBF's frugality:
and this, in reference to the Slack message:
Perhaps Will doesn't deserve much blame (that's certainly a theme running through his comments so far). But if he isn't able to tell the truth about what happened, or isn't equipped to grapple with it, it's bad news for the movements and organizations he's associated with.
I am purely quibbling with whether the Biblical allusion fits.
Like, for one, Nineveh is super alien to Jonah, and he hates the fact that they actually repent, which seems like a bad analogy for Will speaking truth to EAs in order to get us to do better. Also Nineveh's sins don't seem like they have much to do with Jonah's (altho Jonah certainly doesn't seem to have a totally properly reverent attitude). So the paragraph just doesn't really make all that much sense.
It doesn't.
Don't read too much into this piece of journalistic flair. It's common practice to end pieces like this (in a paragraph or two known as a "kicker") with something that sounds poignant on first glance to the reader, even if it would fall apart on closer scrutiny, like this does.
I think it doesn't even make sense at first glance! Anyway I retain my right to complain about bad things that are common.
Were I being charitable to Lewis-Kraus I might say that he moves on to talk about the belly of the fish so in the story EA is God rather than the folks on the boat. Ie that Will is currently in the fish and that denotes uncertainty about the future.
Note that the ship is already effective altruism so in this reading the ship is also God (which is actually an interesting twist on the story).
FWIW my read of the text is that the king of Nineveh is the only one who is said to sit in ashes. This wasn't that hard to check!
Actually sackcloth and ashes go together so maybe we're supposed to assume that the Ninevites did the ashes as well? I maybe retract this remark.
Alternate link: https://web.archive.org/web/20221201200615/https://www.newyorker.com/news/annals-of-inquiry/sam-bankman-fried-effective-altruism-and-the-question-of-complicity
I edited this in. Thank you!
This article makes one specific point I want to push back on:
Anyone who has ever interacted with EA knows that this is not true. EAs are constantly, even excessively, criticizing the movement and trying to figure out what big flaws could exist in it. It's true, a bit strange, and a bad sign, that these exercises did not for the most part highlight this flaw of reliance on donors who may use EA to justify unethical acts - unless you counted the reforms proposed by Carla Zoe Cremer that were never adopted cited by the article. Yes, there is some blame that could be assigned for never adopting these reforms, but the pure quantity of EA criticism and other potential fault points suggests IMO that it's really hard to figure out a priori what will fail a movement. EA is not perfect, nobody has ever claimed as much, and to some extent I think this article is disingenuous for implying this. "People focused on doing the most good" =/= "moral saints who think they are above every human flaw".
Some reasons I disagree:
I think internal criticism in EA is motivated by aiming for perfection, and is not motivated by aiming to be as good as other movements / ideologies. I think internal criticism with this motivation is entirely compatible with a self-image of exceptionalism.
While I think many EAs view the movement as exceptional and I agree with them, I think too many EAs assume individual EAs will be exceptional too, which I think is an unjustified expectation. In particular, I think EAs assume that individual EAs will be exceptionally good at being virtuous and following good social rules, which is a bad assumption.
I think EA also relies too heavily on personal networks, and especially given the adjacency to the rationalist community, EA is bad at mitigating against the cognitive biases this can cause in grantmaking. I expect that people overestimate how good their friends are at being virtuous and following good social rules, and given that so many EAs are friends with each other at a personal level, this exacerbates the exceptionalism problem.
I mean, of course Effective Altruism is striving for perfection, every movement should, but this is very different than thinking that EA has already achieved perfection. I think you listed a couple of things that I had read as EA self-criticism pre-FTX collapse, suggesting that EAs were aware of some of the potential pitfalls of the movement. I just don't think many people thought EA exceptional in the "will avoid common pitfalls of movements" perfection sense as implied by the article.
Edit (3/12/2022): On further reflection, I think these accusations are quite shocking, and likely represent (at best) significant incompetence.
[Low quality- midnight thoughts]
Not sure to what extent to update here, someone unnamed said something about sbf in some slack thread.
It would be good to have some transparency on this issue - perhaps by those who have access to said slack workspace - to know how many people read it, how they reacted, and why they reacted that way.
Although this is unlikely at the moment because of proceeding legal concerns
Relevant manifold market question
Post summary (feel free to suggest edits!):
Linkpost and key excerpts from a New Yorker article overviewing how EA has reacted to SBF and the FTX collapse. The article claims there was an internal slack channel of EA leaders where a warning that SBF “has a reputation [in some circles] as someone who regularly breaks laws to make money” was shared, before the collapse.
(If you'd like to see more summaries of top EA and LW forum posts, check out the Weekly Summaries series.)