Andrea Salinas just beat EA candidate Carrick Flynn in the Democratic primary for Oregon's 6th congressional district. She got about 38% and he got about 19% of the vote. This outcome was (very) disappointing but expected.
Individual EAs donated some $900K (my estimate) to Carrick's campaign, presumably mostly because they thought it was highly effective in expectation. Several prominent EAs publicly donated the maximum amount, and the general feeling on this forum (and elsewhere, like twitter) was "I . . . recommend this highly to people looking for impactful individual donations" (source). In retrospect, this investment didn't pay off. But independent of this particular election, what can we learn about prioritizing donations?
I wouldn't want everyone to spend forever researching donations of a few thousand dollars. But when the community is donating on the scale of $100K or more, particularly in novel ways, and particularly when many donors are driven by prominent EAs' enthusiasm, we should really have reason to believe that the intervention is effective. Before donating and encourage others to donate, we should share reason to believe that the intervention has a reasonable probability of success, rather than just the promise of great outcomes if it does succeed. I almost think it's irresponsible for community leaders to say things like "I donated $5800 and recommend this highly to people looking for impactful individual donations" without (1) pointing to at least a back-of-the-envelope reason to believe that marginal donations have a reasonable probability of improving the outcome and (2) checking to make sure at least a couple people with relevant subject knowledge agree. Whether donating is effective depends not just on whether electing Carrick would be good, but also on how effective donations are, and I saw little discussion in the community of expected value of marginal dollars. I would guess that relatively few EAs donating to Carrick's campaign had either a good understanding of elections that told them that donating was highly effective or meta-level justification such as the opinion of an independent person with relevant knowledge. Again setting aside this specific campaign, I think we should beware of bandwagoning before perceiving much reason to believe that an intervention is likely to succeed.
Also, I think we should be particularly cautious around causes that feel tribal-y. I noticed that a couple objectively bad pro-Carrick election takes got highly upvoted in the endorsement comments, while some objectively good Carrick-skeptical election takes and some objectively reasonable meta-level comments got downvoted. We should be good enough at scout mindset to welcome informed criticism without seeing it as an attack on our values; I'm not saying that went wrong here, but the forum votes suggest it's possible. In such situations, we should be particularly encouraging of counterevidence (see Habryka's comment). It should be easy to disagree with the community without it feeling hard, much less feeling like you might risk social status. Regardless of the actual consequences of expressing dissent, a reasonable commenter thought it was "obvious" that dissent would entail "severe" consequences, so others likely had similar fears.
I happen to think that donating to Carrick's campaign was totally reasonable, as an effective intervention, but I fear that the community's decision-making process wasn't very grounded in donations' probability of tipping the election, so I'm concerned that future causes might get support despite being predictably ineffective. Anyway, I'm excited to see Carrick's future research and policy work.
Edit: this post is getting several downvotes and strong-downvotes, so I just want to emphasize that I'm excited to hear particular disagreements you have or what you think I got wrong. Indeed, this post is kind of about how we should share disagreement more!
Edit 2: now there's some relevant criticism, thank you!
Shouldn’t we know better than to update in retrospect based on one highly uncertain datapoint?
We have a number of political data people in EA who thought donating to Flynn was a good investment early in the campaign cycle (later on I was hearing they thought it was no longer worth it). There was also good reason to believe Flynn could be high-impact if elected. Let’s not overthink this.
I agree we shouldn't update much on the effectiveness of campaign donations. Despite being less enthusiastic about donations to Carrick than [literally every prominent community member I saw comment on the election], I actually think donating to Carrick was totally reasonable, as I mentioned in the post. This post is just about how we collectively reach such decisions; I'm concerned about the process.
I don’t understand the premise of the post especially given what you just wrote.
Would you write this if Flynn won? I’m a little skeptical you would as your post leads heavily with this outcome.
This post was originally going to be more about elections, but I didn't have time to write that. I kept the lede because it is relevant; Carrick lost by quite a lot, and in retrospect marginal donations were wasted, and I think some people wouldn't have donated if they knew more at the time. (And because the lede explains why I'm posting this now.)
If Carrick won--or if the election was close--I would have written something similar but with slightly different framing/tone.
As a logical point, it also seems like you could have written up an analysis or BOTEC when it seems like it mattered (maybe anonymously under an account “Definitely Not Zach Stein-Perlman”. This is because it could have influenced $10M (or even more).
You could then say “Ah ha, it was I all along”, and this would be a little more convincing that this reasoning was possible.
Fair. If a similar scenario arises in the future, and I disagree with the consensus, I'll strongly consider sharing analysis. (Also note that major bandwagoning happened quickly, within a day of ASB's post. Slowing down would allow for people to do more analysis.)
Thanks for writing this. While I don’t personally enjoy being featured, I appreciate the post as a Forum reader and former mod.
A few notes on my approach to donating, since I was quoted:
Other thoughts:
*I won’t say “no reputational hit at all”, because thousands of people read the Forum and some of them would probably be annoyed. Public online discussion is rough.
Thanks very much for your comments. I almost entirely approve/agree, and I think this is all useful. (And I'm sorry to quote you in particular, but that quote was one good example of the phenomenon). I'd just add two things:
I wasn't surprised that Protect Our Future intervened (though I was surprised by how much it spent). Others with relevant knowledge might have been able to confidently predict that or other relevant factors in advance. I think donating was correct in your epistemic state. But in general, even if one believes donating to X is higher EV than donating to anything else, it doesn't imply that one should donate to X, if there's also the option to learn more first. Repeating something from my post, it's not worth spending so much time to guide a donation of $3K, but for a community donation of ~$900K it seems worth being more methodological.
I tentatively agree. But harm arises even if there's no social punishment-- just from the fear of social punishment leading to self-censorship.
You don’t understand the premise of my comments in that thread (I am guessing) you reported where I contravened _pk.
The goal of my comments there is to mitigate issues of anonymous accounts that don’t contain new or verifiable information but are mainly emotional or appeal to norms or authority of various kinds (“I can see OR-6 from my house”). Anonymous comments like _pk (even if sweetly written and are >70% likely to be authentic) are problematic when emotional and influencing behaviour. This is extremely so when it involves movement of money which has no precedent on an Internet forum. I distrust your judgement about this issue described by this paragraph.
As part of an number of considerations, I take ownership of the repercussions by sandbagging this persona, which I believe mitigates issues. It is possible that inept countervening will simply cause me to relax this sandbag, to achieve the aims in the above paragraph, but the resulting envelope of achievable results will be much worse (e.g if this establishes some wretched authority or something to this account, the issues of which are sort of being pointed at by the post and your own comment).
The forum is really complicated and the norms under the previous moderator set things up for failure.
I definitely agree that we need reflection on the questions, but I frankly think that this post needed a bit more basic fact checking.
Sure, but I think the first bar - expectation of impact conditional on success - was correct, and don't think you are disagreeing. And the second claim, that we need "reasonable probability of success," is a different and contentious claim, given all the discussion of hits-based giving, and the idea of expected value.
But 1) they did the BOTEC, if you read the post you linked to, and 2) yes, GAP and the various other people who focus on public policy and supporting the campaign personally were both working with plenty of people with subject matter expertise.
All of that said, we should certainly update based on the information gained from this loss, and I definitely think that more reflection is critical - but the reflexive take that "Carrick lost, so this was a bad decision" is unhelpful, and I'd welcome more specific thoughts on how and why "the community's decision-making process wasn't very grounded in donations' probability of tipping the election" - which seems plausible, but unclear, and far from the only thing that matters.
Thanks for your comments. I think you're mostly right, and I apologize for not being able to write more or more carefully at the moment.
I did, a few times. The BOTEC is... extremely BOTE-y, and significantly too optimistic in my opinion, but absolutely better than no discussion of effectiveness.
I totally agree.
I guess I'm mostly reacting to sentiment on the forum in early February, which did not feel very concerned with how likely Carrick was to win (and iirc there was no analysis other than ASB's shared on the forum), and independent of donations partly reacting to discourse similarly to Habryka.
Epistemic status: very tired.
As others mentioned, this feels like too much of an update based on one data point.
One of the largest advantages EAs running for office will have is their ability to fundraise from other EAs. I worry that skepticism of EAs in politics and/or slowness to act on time sensitive donation oppos will kneecap the success of future candidates.
Big picture, I think the impact case was pretty solid. The US govt is enormously influential. It moves a lot of money, regulates important industries, has the largest military, and can uniquely affect x risk. Members of congress exert significant control over the govt. Senators more, president most.
Having an extremely committed EA in govt seems worth A LOT to me.
Raising some amount of money is essential to winning, no matter how much outside money is committed to a race. Campaigns need to hire staff, get on the ballot, and do other things that super PACs can't do. They also get much more favorable rates on TV ad buys, can make better ads, etc. "Hard money", i.e. that raised by campaigns by retail donors and governed by donor caps, is way more valuable than "soft money", i.e. independent expenditure made by super PACs.
It seems clear to me that marginal hard dollars increase the odds of success, and it doesn't have to be that big of an increase for it to be a good bet in expected value terms.
I would guess that almost no EAs donating to GiveWell charities really understand the evidence base and models going into the recommendation, but we outsource our thinking to people/orgs we trust. Obviously, there's way less of a track record with running EAs for office and a lot of uncertainty baked into politics. But the most experienced, aligned people in the political data science world were supportive of this particular race happening, and A LOT of thinking went into this decision.
Another thing to consider is the enormous amount of info value we got out of this campaign. It looks like large amounts of money are not a sufficient condition for victory, but if Carrick hadn't been able to raise the amount of hard money needed to make the campaign happen, we would've learned a lot less.
Some of us already knew that "large amounts of money are not a sufficient condition for victory," but I agree there are other things we can learn.
If you had to do it yourself, how would you go about a back-of-the-envelope calculation for estimating the impact of a Flynn donation?
Asking this question because I suspect that other people in the community won't actually do this, and since you are maybe one of the best-positioned people to do this since you seem interested in it.
I'm not sure what my estimate would be -- probably ballpark of a marginal $1 buying 1/500M of a counterfactual election in a race with as much money as this one and where the candidate was a dark horse, more effective in races with less money already or where the race was closer. To actually make an estimate I'd feel comfortable with, I'd have to look for studies on the effect of money on elections; if studies gave a consistent picture, there wouldn't really be any more work to do.
Hey folks, good back and forth here. Just thought this story we just published on Protect Our Future and SBF might be interesting to the question about the value of a donation for Carrick and the cost-per-vote in this race.
My back of the envelope. Back back back, like maybe even outhouse of the envelope. It's very hard to calculate marginal cost per vote, in part bc there's sort of an efficient markets thing going on w donations in some cases and not others. A senate race in Wyoming costs roughly the same as one in California, because the seat is just as valuable, despite vastly different numbers of votes. But activists getting worked up about a race can change the numbers, and in solid blue states it seems you can win a House seat more cheaply than in swing states, where you have to win a primary and a general and the national partisan orgs dgaf about the primary.
:
Approx 10m voters who matter in a Presidential campaign (i.e. the relevant pop of swing states). Takes about a billion to win. Naively this would suggest $100/vote, but multiply by 10x to $1000 for the marginal vote (iirc there IS some fancy econometrics way to estimate marginal dollar's impact better than this and I have seen it before; this is just dumdum math and I am dumdum).
Do these numbers hold up for campaigns at other levels? Gonna set aside the Flynn campaign bc freakishly large amounts were spent. Normal House primary, maybe 1m spent. Maybe 100k votes cast in a blue district? Idk; it varies widely. I got this estimate from Ayanna Pressley's race, but it looks like in AOC's race only 30k were case, and 63k in Chris Pappas's primary race in a swing district. So, taking AOC as a sort of worst case, 30k votes cost 1m, ($33k each) assume the marginal vote is 10x that, it's $330 each.
Or, thinking in terms of $ per election and ignoring numbers of votes, assuming 1m gets you a 50% chance of winning, $1=1/2m probability of winning election in a normal election.
But Flynn campaign cost more like $2000/vote (not marginal, avg) if reports of 8-figure SBF spend are to be believed. So under the same methodology this would imply ~20k per marginal vote.
Thanks for sharing! But I think this may be too BOTEC-y to be useful...
This is too low by a couple orders of magnitude. The marginal dollar in a presidential election has much less than a one-in-a-billion chance of tipping the election.
More generally, I think it's quite misleading to think in terms of buying votes. (In retrospect, Carrick wouldn't have won even if he had $100M.)
(And yes, Protect Our Future did spend $11M.)
"More generally, I think it's quite misleading to think in terms of buying votes. (In retrospect, Carrick wouldn't have won even if he had $100M.)"
I think people are confused about the difference between a PAC spending money, and Carrick's campaign having money. Because of rules about donations and coordination, money given To Carrick's campaign was far more helpful. Ad of the end of April, his campaign has raised just under $1m. That's a lot, but Salinas, who won, had raised $600k by then, and 2 other candidates who lost had raised even more, or lent their campaign the money, which is allowed - so it's not nearly the same outsized advantage as it seems when you talk about PAC spending.
I agree, and you're right that my comment could have been clearer. (Regardless, my point was that here and in other posts people are talking about 'how much votes cost' in an incautious manner, and e.g. the fact that there was much less pro-Salinas spending per Salinas vote than pro-Carrick spending per Carrick vote doesn't mean that pro-Salinas spending was more effective.)
I'm curious how much more helpful you think it is; I would have thought <2x. The fact that campaigns sometimes spend money on basically the same thing PACs would--ads--seems to suggest that marginal campaign spending can't be much more effective (or campaigns wouldn't buy ads).
Also, no such thing as generic "too BOTEC-y to be useful." If you have a more rigorous calculation offer it. Otherwise BOTEC is the best available estimate and you should show it more respect until you do have an alternative.
This is why I suggest the marginal dollar is only 1/10 as effective as the avg dollar. I don't have any particular reason to think my est is off by an order of magnitude or more. If you do I'd like to hear it, and I suspect so would every campaign in the world.
Um, $5.7B was spent on the 2020 presidential election, and I have no idea where "Takes about a billion to win" comes from? Marginal spending doesn't have much effect in presidential elections.
Joe Biden raised 1.69 bn, Trump 1.96 b https://www.npr.org/2020/05/20/858347477/money-tracker-how-much-trump-and-biden-have-raised-in-the-2020-election. Little more than I thought but not a whole OOM. Closer to 1b than 10b. Call it 2bn to win if u prefer.
"Doesn't have much effect" is too vague a statement to be meaningful. 1/1b increase in chance of winning is simultaneously "not much" and also enough to spend money on where the consequences are large enough.