So this is an empirical post to warn others and to discuss these practices. I think it's essential, especially since many orgzs/companies have people writing and reading this forum. I am not writing out of bitterness; just stating what I've experienced. I hope they will engage in this conversation!
A bit of context: I am finishing a Ph.D. in History. An underdeveloped discipline in the EA field, although McAskill's recent book might have a good effect on changing this. So, I'm trying to sell my research skills in a more reality-anchored, useful field than medieval state-building (happy to chat about it if that sounds interesting!).
I have applied to many, many jobs. The success rate being 5% (a number I heard during a 'celebrate our failure' gig), I am not surprised to struggle. However, the most frustrating thing is being chosen for the first, second, even third, or fourth or fifth (last one I had) interview round, completing tasks for free more than half of the time, and just....raising my hopes for nothing as I'm never chosen in the end.
Let me explain. I had to fulfill very practical and useful tasks for the organizations to which I applied. Without naming institutions, I'll cite one where I had to choose relevant candidates to speak for a panel or another where I had to devise a whole plan to manage a new research unit. I spent hours on these tests. My most rational approach was to do my best because I knew competition to be fierce and that I couldn't just mildly apply for something and have a single chance to be reinvited for an interview without giving my best.
As I'm finishing my thesis, it took much time and energy of thought to complete these tests. It took hope. And yet, I was barely compensated for my time despite giving back, what I believe, was work of very high quality. I've never seen this phenomenon outside of EA, e.g., the requirement of working on intellectually intensive tasks that take more than a few hours. Maybe it exists: but it seems to me that because EA-related companies are more likely to raise feelings of hope and admiration than those that are not EA-oriented (in sum, those that only exist for profit without any other aim), EA-related companies invite more people than what they anticipate to employ to complete some labor for free.
And it doesn't feel right. I know EA isn't (directly) about unions and fair rights for employees, but it feels that organizations use my work without them having the intention to hire me.
Thoughts?
I'm really sorry you had such negative experiences with your applications.
I'd like to state for the record that Rethink Priorities makes heavy use of work tests because we find they best correlate with on-the-job performance.
However, we:
(1) always ensure generous and competitive compensation for work tests done regardless of whether the applicant moves on in the process
and (2) never make actual use of work test output other than to assess the applicant (e.g., we only ask questions that we've already researched internally)
While I'm sympathetic to organizations who may not have the funding, I think it would make sense in the future for candidates to refuse to engage with multi-hour work trials without fair compensation.
Same for the organization I run.
Hi Vaipan, thanks for writing this. As someone who's applied for (many) dozens of EA and non-EA jobs over the past 2 years, I can identify and empathise with your story here.
From my experience, I think EA orgs do use 'work tests' way more often than non-EA orgs (I very rarely had to do a work test for non-EA job hiring, only interviews). There are obviously pros and cons to using work tests, and in my short time at a couple of different EA orgs, I've been part of pretty frequent discussion about how/when/why we should use work tests in hiring rounds (to their credit, I think).
The main thing that I'm surprised by in your post is the high frequency that you report to have done unpaid work tests. In recent times I've been really happy to see most EA orgs offering compensation for work tests, and I think the vast majority of work tests that I did were compensated.
^This quote is pretty concerning .In general, I don't think I've seen anyone use the product of work tests for anything other than assessing them for hiring purposes. Are you saying that orgs have used the product of your work tests for other purposes? In that case, it does seem especially bad for that work test to have been uncompensated, and I'd assume (and hope) that most other EA's would agree with me.
If you don't feel comfortable sharing which orgs they were publicly, I'd strongly recommend speaking to the community health team at CEA about this.
Hi Kaleem!
I hope you have found your fit by now! No, I don't have any evidence that my work has been used, but the assignments were so specific that it gave me this impression. However, it is only an impression!
Some anecdata:
I’ve found work tests to be very worthwhile in hiring (often the single most predictive) . In return (as well as paying) I’ve offered feedback to all candidates who’ve done the work test.
I’m glad you raised this and while I’m not sure of the prevalence of the problem within the EA ecosystem I think it’s important we have norms around paying people for their work (unless it is explicitly charitable volunteering which is something that I think is good to provide opportunities for people to do and has been valuable in my own impact journey).
Hi Luke,
I'm happy to know that your org has better practices. It's also the feedback part that sucks: despite succeeding at many rounds, I never gotten any feedback!
FWIW, I found this surprisingly high, assuming that the denominator is something like: "all candidates applying to all jobs on 80k board," which excludes internal hiring/closed hiring/etc.
I personally only offer paid work trials, and this is the norm in the orgs I've seen (e.g. OpenPhil). I hope the answer is that the ones you experienced actually can't afford to do this (but I'm sure some could).
I’m sorry to hear you’ve had these negative experiences.
At CEA, trial tasks (minutes or hours) and work trials (days) are key components of our typical hiring rounds because we believe them to be better predictors of performance in the role than other assessments such as resumes or interviews. We do, however, always offer to compensate candidates for their time for any longer trial tasks and work trials if it's legally possible. (Visa rules mean this is sometimes not possible for candidates who travel, e.g. to the UK, for a trial without a work visa.)
I think work tests are a great way to hire people (because they are less biased than the alternatives) but I agree they should be paid for. I didn't know unpaid work tests were a thing. Which orgs had extensive unpaid work tests?
I think work tests depend on the industry. For example when I was working as a teacher, my interview usually included teaching a real class of students, but in the Civil Service I've never been asked to do anything more than prepare a presentation.
Following other commenters here, I'm heavily pro work tests, but always advocate strongly for (what I believe to be) fair compensation. If organisations are having you do major unpaid (or e.g. minimum-wage) work trials, that seems like a serious problem.
There are also two broad categories of paid work tests I've seen, and I think the distinction between them is important:
In my opinion, both of these should be paid, with the importance of providing payment increasing with the length and arduousness of the trial. I also think (2) should only be used with a small number of finalist candidates the org is genuinely excited about. If an org is instead giving these kinds of trials to significant numbers of candidates with lower probability of hiring, then I agree that seems quite problematic, especially if the trials aren't fairly compensated (the need for fair compensation also creates an incentive to be selective with the candidates that get asked to take a trial, which I think is helpful for reducing the concerns raised here).
On (1), the organization needs to clearly explain that this is a task that is used for assessment and that the organization will not benefit from the task. Otherwise, the candidate will likely walk away feeling used, and may share information about the trial task with others (why wouldn't they if they were not aware it was a standardized instrument that will be reused?)
I have mixed feelings about being "selective with the candidates that get asked to take a trial" -- if you weight performance on the trial very highly, then trimming the number of candidates able to take a trial denies people who may have scored very well (but whose resume is middle-of-the-road) a chance to prove themselves. Obviously, one should be more selective with invitations to more time-consuming and arduous trials.
We generally try to be pretty clear that this is a standardised test, yeah. Including asking people not to share it with others.
I agree that this is a tradeoff -- one that is made sharper by the expense of paying fairly for trial tasks (if we didn't pay, we could have a lower bar for sending out trial tasks). I think it's fairly common to try to address this by having multiple trial tasks / work trials of increasing length & selectivity. (Though this has its own cost, namely an unusually lengthy and gruelling application process.)
Do all candidates necessarily start at the same stage of trial tasks in most organizations? I could imagine a very short "stage 0" trial task requested only from candidates who would be on the bubble of moving on to the first, moderately time-consuming task if there were no "stage 0" task to evaluate. The "stage 0" task would only need enough power to tip someone on the bubble over into either the "it's respectful to ask them to do a more demanding task" or "respect their time and move on" buckets. Of course, you'd need to find a quick "stage 0" task that correlates well enough with either job performance or later-stage test performance.
Ideally there'd be a supervisor-org where you could tell them about this, nothing bad would happen to you, and they'd check with the hiring-org what's going on.
With all the discussion on EA governance lately, this is something that seems relevant to me
This could be something to bring to the CEA community health team? It's not a central example of the kind of case I think they normally handle, but it does seem like a kind of "EAs treating each other badly".
(Disclosure: I'm married to Julia Wise, but that doesn't mean I know whether the team would consider this within their scope)
Thanks! I'll refer this comment to relevant people
Why an org and not a public platform?
I'm not against
I don't really know how to set up a structure like this, I assume some people thought a lot about such topics
I've come across the same thing in both EA and non-EA orgs (notably including some higher level poisitons at the UN that had a multiple hour long work test as part od second and third rounds of the hiring process), although for the most part the EA ones have been compensated, and compensated fairly generously. I personally consider any uncompensated work test that takes more than about an hour to be a serious red flag for how I expect the org to treat overtime, work life balance and compensation.
I know that these positions are very competitive and that not engaging with them is not a choice for most people looking for a job, but I think that that might be the best way to disincentivize these practices- a drop in the quality and quantity of applicants while letting them know the reason why will probably be the most effective way to discourage this practice. I'm also generally a fan of naming and shaming for exploitative practices but that unfortunately tends to have an outsized negative effect on job seekers.
(Speaking in a personal capacity of course) I like the idea of using interviews and work tasks to help the organisation and help EA priorities etc. Consulting a variety of perspectives and backgrounds can be very useful, and otherwise we can end up wasting a lot of time, effort and insight in the job seeking and hiring process.
As as an interviewee I was sometimes frustrated when I put in a lot of work into a task I ~knew was not actually used/useful.
BUT I think these must be compensated and perhaps even generously compensated. Generously: to avoid the impression of being exploitative, and the temptation to do so without considering the value of the interviewee’s time.