Thanks so much for writing this! 

I just went through the process of trying to hire an executive assistant in the US so I thought I'd contribute a bit. My sample size is relatively low (I only tried 1-2 executive assistants from ~3 services), but here are some things I found helpful: 

* In my experience additional cost --> additional quality. Boldly (the firm we will likely end up using - $50 / hour) seemed to have higher service quality than places like Time Etc. ($30 / hour)

* I tried a series of 'work tests'. Here are the two I found most predictive: 
a) I'm trying to [find an Airbnb here, find a conference center, ship my bags using a remote service -- example]. Could you help me figure out the lowest cost provider that has good client reviews? Here are some other criteria that are important (X, Y). 
This allowed me to see presentation of information, accuracy, and reasoning. 

b) I'm interested in services from [X company]. Could you write an email inquiry about whether they are taking new clients & price? It would be great to try to find a personal email for a staff member. 
This allowed me to see basic email composition skills, spelling, and resourcefulness online. 

* I found it helpful to test out two companies in parallel for a month, that way I could have a direct head to head comparison. I would ask both executive assistants to perform the work tests above. I would then ask for a new executive assistant from the firm that was performing to a lower quality standard, and keep using the one that was performing well on one off tasks to get more data. I think this particularly worked in my situation because we had some lead time. 

Appendix: 

Jan 2022 survey of Oxford/Cambridge/Stanford organizers 

We surveyed some full-time group organizers on how valuable they’d found various aspects of CEA support, versus support from non-CEA people (GCP, Lightcone, Buck Shlegeris – EAIF, Claire Zabel – Open Phil, EAIF, Stanford residencies). We gave them the option to be anonymous.

We split this up into 13 types of CEA support (UK group leaders retreat, US retreat, calls, etc.), and 8 types of non-CEA support. They rated things on a 1-7 scale, based on how useful they found them. 

Ignoring N/As, CEA activities got an average score of 4.2/7. Non-CEA activities got an average score of 5.1/7. Summing up scores (which doesn’t have a clean interpretation), CEA totaled 246 points and non-CEA people (GCP, Icecone (a winter retreat hosted by Lightcone), Stanford team, Cambridge’s online course) totaled 201 points.** This maybe indicates that CEA is providing a wider breadth of less intensely valued services. On the other hand, we asked more detailed questions about CEA’s services so the whole ‘total number’ could be biased upwards. 

Looking in more detail at scores, it seems that support calls with CEA staff members were less useful than support calls from non-CEA staff members, retreats were generally more useful, and various forms of funding were quite useful. Different leaders found quite different things useful.

Some more direct comparisons:

• 1:1s: 
  • CEA 1:1s were rated 3.8/7
  • CEA in person campus visits 4.3/7
  • GCP 1:1s were rated 4.7/7
  • 1:1s with others (e.g. Claire, Buck) were rated 5/7
• Retreats/events:
  • CEA’s summer retreat and EAG London retreats averaged 4.3/7
  • Icecone averaged 4.9/7
  • GCP’s summer residency averaged 5.0/7
  • Stanford’s residencies were 5.5/7
• Funding:
  • CEA’s revised expense policy/Soldo cards were rated 4.1/7
  • CEA’s Funding for Campus Specialist Interns was rated 5.0/7
  • EAIF funding was 5.8/7
• Other resources:
  • CEA’s remote community building fellowship was 3.0/7
  • GCP’s handbook was rated 4.3/7
  • (CEA) Lifelabs management calls were 4.4/7
  • GCP’s advice on how to do 1:1s was rated 4.5/7
  • Cambridge’s online cause specific programs were rated 6.0/7

Overall, this suggests that others provided more targeted, useful support. I think they suggest that CEA did provide some meaningful value to these group leaders, but that it might be better to cede this space to others if others have interest and capacity to take it on. 

** Notes on interpreting this: I think we split CEA activities up in a more fine-grained way, which may biased scores for individual activities downwards. I also think that some of these activities (e.g. UK/US retreats) were not aimed at these organizers, but at getting less involved organizers more excited. Also, it might be fine to have low averages, with a lot of things, e.g. if the things you’re providing are really useful to some organizers but useless (and easy to ignore) for other organizers.

Summary: CEA support for earlier stage focus unis group organizers 

We surveyed attendees of our January Groups Coordination Summit, both on that particular event, and also on what support had been more generally useful to them.

Key figures:

Ignoring N/As, a similar gap remains. CEA activities got an average score of 4.8/7. Non-CEA activities got an average score of 5.4/7. The average scores are overall higher – this indicates that earlier stage groups can be more intensively helped by outside support. 

Summing-up scores (which doesn’t have a clean interpretation), CEA totaled 297 points and non-CEA people (GCP, Icecone, Stanford team, Cambridge’s online course) totaled 345 points. 

Some more direct comparisons:

• 1:1s: 
  • CEA 1:1s were rated 4.0/7
  • GCP 1:1s were rated 5.0/7
  • Calls with others (e.g. Claire, Buck) were rated 5.0/7
• Retreats/events:
  • CEA’s summer retreats and EAG London retreats average 4.9/7
  • Icecone average 5.7/7
  • Stanford’s residencies were 6.0/7
  • GCP’s summer residency averaged 6.3/7
• Funding:
  • CEA’s revised expense policy/Soldo cards were rated 5.1/7
  • CEA’s Funding for Campus Specialist Interns was rated 4.8/7
  • EAIF funding was 5.9/7
• Various forms of Resources:
  • GCP’s handbook was rated 4.4/7
  • GCP’s advice on how to do 1:1s was rated 4.8/7
  • (CEA) Lifelabs management calls were 5.0/7
  • CEA’s remote community building fellowship was 5.3/7
  • (CEA) University Group Accelerator Program (UGAP) was rated 5.3/7
  • Cambridge’s online cause specific programs were rated 5.8/7

For this group, retreats/events seem better when longer and/or focused on a narrow project (Icecone, Summer residency, Stanford residency) compared to our shorter retreats.

2. yes
3. correct
4. yes
5. we think focusing will improve quality in the short term, which will enable more potential scale / impact in the long term

Thanks for your questions! As mentioned before, I’m excited for others to consider full time community building via the infrastructure fund, and hope that you and others would peruse this option if you feel well positioned.

I don’t think CEA is covered all the net positive opportunities in this space — just the ones we think are the best given our view of our core competencies, staff capacity, and theory of change.

Within the CEA Groups team, we have several different sub-teams. Two of the sub-teams are focused on experimenting and understanding what a model looks like with full-time community builders in a focused set of locations (one sub-team for university groups, another sub-team for city/national groups). This is because the type of centralized support CEA might provide and the type of skills/characteristics required of someone working full-time running a university group or a city/national professional network might look very different depending on the ultimate model. 
 

Our staff capacity is limited (to either hiring, piloting, scaling) and we think that this focus will enable faster scaling in the long term. 

I also want to note a couple things: 
* In addition to the sub teams mentioned above, we have two sub teams supporting part-time organizers. One team provides foundational support to all part-time/volunteer group organizers (basic funding, resources hub, EA slack, phone calls), and another team runs the University Groups Accelerator Program to help part-time university organizers launch their group. 

* Additionally, just because the CEA Groups team building up the ‘full-time’ model is prioritizing certain locations, that doesn't mean we want to stop experiments in other locations. We'd encourage people interested in full-time organizing in places that aren't on the locations list above to apply to the EAIF, help us innovate on the community building model in different locations, and share back your learnings with other organizers and on the forum.

Thanks Michael, these are good points. We should have been more careful here, and plan to edit the post to be more nuanced.  

Thanks for the recommendations, Ryan. I'll pass this on to my team and we'll look into it the historical outcomes of undergraduates. 

Thanks for your comments Mathias, 
 

Just to echo your point about supporting university groups - beyond supporting a subset of university groups with full-time organizers via the CBG program, we just released a job posting for someone to help us develop a scalable university support program that I think is high impact. This will further support volunteer-led university groups. 

> I don't have any a priori reason to believe that Austin and Warsaw have much less 'ea-potential' than Stockholm and Prague. It seems to me that many places have potential to grow as big as the communities you're focusing on, but for some reason have not.

We’d agree with that. Apart from India, our key locations were chosen because of their existing large groups of engaged EAs, not because of the particular potential of that location. As we noted in the post, about half of these groups with large EA populations currently don’t have paid community builders, so we’d like to make sure there are paid community builders in these areas before considering locations to grow EA in further. I’ll make a note to add the locations you suggested to consider as part of our analysis. 

There are projects other than the CBG programme that are intended to support the community members in other locations, including the EA Forum, conferences, virtual programs, and non-salary funding for all groups. We’re pleased that the EAIF is able to assess funding applications from other groups, and we expect volunteer-run and EAIF funded groups will also be able to grow.
 

For the large EA Virtual Program round, at first we were worried about having enough facilitators. But then we actually had quite a number of volunteer facilitators (over 100!) so then we focused on getting more participants. In the end we ended up having participant demand that matched available facilitators. As we mentioned, we're working to build more operations capacity for the virtual programs version of our fellowship. Once we do this, we hope to be able to offer them on a more consistent basis so more people can sign up.

On #1 and #2: In our report in footnote 5 where we reported this data we said:  "As some students leave fellowships before finishing, and fellowships are run independently through groups, our estimates of the number of fellowships in Q4 and participants across Q1 / Q4 are somewhat uncertain."

I do think the benefits of reporting estimates are more valuable than only reporting precise information, but we do try to add additional detail about where the uncertainty comes from. I'll keep this comment in mind when we do our Q2 report as well. 

Hi Brian, 

Great to hear about your enthusiasm for fellowships! 

1. Q1 data - we count fellowships based on when they start
2. We collect data on this at the end of each fellowship, so the non-EAVP participant numbers in the report are guesses based on Marie’s conversations with group leaders. For groups Huw was not in touch with on a regular basis, Marie assumed an average of around 12 participants/group, so it’s possible that the number is higher based on EA Philippines numbers (although a number of fellowships are smaller than 12 participants).
3. Marie is planning to make a spreadsheet like this from next quarter and will post it on the EA Groups Slack.
4. We’re collecting data on both starting and finishing numbers of participants at the end of each fellowship.
5. Marie is planning to include future fellowships in the spreadsheet. Adding a tab for other reading groups seems like a good idea. 

Feel free to reach out to Marie directly on the EA Groups slack if you’d like to discuss more