Good points.

We should try to track the uncertainty in our all-things-considered beliefs, and we should take a portfolio approach.

It's not enough to just track the uncertainty, you also have to have visibility into current resource allocation. The "defer if there's an incentive to do so" idea helps here, because if there's an incentive, that suggests someone with such visibility thinks there is an under-allocation.

And this all-things-considered belief is what guides my research and career decisions.

A few arguments for letting your independent impression guide your research and career decisions instead:

• If everyone in EA follows the strategy of letting their independent impression guide their research and career decisions, our distribution of research and career decisions will look like the aggregate of everyone's independent impressions, which is a decent first approximation for what our all-things-considered belief should be as a community. By contrast, if everyone acts based on a similar all-things-considered belief, we could overweight the modal scenario.
• You have more detailed knowledge of your independent impression than your all-things-considered belief. If you act on your all-things-considered belief, you might take some action and then later talk to a person you were deferring to in taking that action, and realize that a better understanding of their view actually implies that the action you took wasn't particularly helpful.
• Working based on your independent impression could also be a comparative advantage if it feels more motivating since your path to impact seems more intuitively plausible.

IMO, good rules of thumb are:

• Carefully consider other peoples' beliefs, but don't update too much on them if you don't find the arguments for them persuasive. (There's a big difference between "people are unconcerned about unrecoverable dystopia because of a specific persuasive argument I haven't heard yet" and "people are unconcerned about unrecoverable dystopia because they haven't thought about it much and it doesn't seem like a fashionable thing to be concerned about".)
• Defer to your all-things-considered belief in research/career decisions if there's an incentive to do so (e.g. if you can get a job working on the fashionable thing, but not the thing you independently think is most helpful).

I think offering funding & advice causes more people to work with you, and the closer they are working with you, the larger the influence your opinion is likely to have on the question of whether they should shut down their project.

I don't think risk goes up linearly with time. Many people quit their PhDs when they aren't a good fit.

Fair enough.

Maybe a pragmatic solution here is to emphasize to people who get a grant to do independent research that they can quit and give back the remainder of their grant at any time?

Sure. Well when the LTFF funds graduate students who aren't even directly focused on improving the long-term future, just to help them advance their careers, I think that sends a strong signal that the LTFF thinks grad school should be the default path. Counterfactually, if grad school is 5-10x the risk of independent research, it seems like you should be 5-10x as hesitant to fund grad students compared to independent researchers. (Assuming for the moment that paternalism is in fact the correct orientation for a grantmaker to have.)

My model for why there's a big discrepancy between what NIH grantmakers will fund and what Fast Grants recipients want to do is that NIH grantmakers adopt a sort of conservative, paternalistic attitude. I don't think this is unique to NIH grantmakers. For example, in your comment you wrote:

we want to avoid funding people for independent research when they might do much better in an organization

The person who applies for a grant knows a lot more about their situation than the grantmaker does: their personal psychology, the nature of their research interests, their fit for various organizations. They seem a lot better equipped to make career decisions for themselves than busy grantmakers.

It seems worth considering the possibility that there are psychological dynamics to grantmaking that are inherent in the nature of the activity. Maybe the NIH has just had more time to slide down this slope than EA Funds has.

The feedback loops in grantmaking aren't great. There's a tendency for everyone to assume that because you control so much money, you must know what you're doing. (I talked to an ex-grantmaker who said that even after noticing and articulating this tendency, he continued to see it operating in himself.) And people who want to get a grant will be extra deferential:

once you become a philanthropist, you never again tell a bad joke… everyone wants to be on your good side. And I think that can be a very toxic environment…

source

So I think it's important to be extra self-skeptical if you're working as a grantmaker.

I think you're splitting hairs here--my point is that your "hesitation" doesn't really seem to be justified by the data.

trying to pursue an independent research path will be a really big waste of human capital, and potentially cause some pretty bad experiences

I think this is even more true for graduate school:

• https://www.benkuhn.net/grad/

• https://scholar.harvard.edu/files/bolotnyy/files/bbb_mentalhealth_paper.pdf

Independent research seems superior to graduate school for multiple reasons, but one factor is that the time commitment is much lower.

In my opinion it's not enough to carefully think through independent research grants... with so much longtermist funding centralized through your organization, you also have to carefully think through a default of funneling people through another thing that can waste a lot of human capital and cause a lot of bad experiences, but lasts 5-10x longer.

It’s hard to find great grants

Pardon me if this is overly pedantic, but I think you might be missing a map/territory distinction here. "It's hard to find great grants" seems different than "It's hard to find grants we really like". For example, the LTFF managers mentioned multiple times in this post that they're skeptical of funding independent researchers, but this analyst found (based on a limited sample size) that independent researchers outperformed organizations among LTFF grant recipients. Similarly, a poll of Fast Grants recipients found that almost 80% would make major changes to their research program if funders relaxed constraints on what their grants could be used for, suggesting that the preferences of grantmakers can diverge wildly from the preferences of researchers applying for grants.

Sure. I guess I don't have a lot of faith in your team's ability to do this, since you/people you are funding are already saying things that seem amateurish to me. But I'm not sure that is a big deal.