Investing into researching psychedelics will damage the "brand value" of EA. While many peoples' opinions of EA would likely improve if the organization began investing into researching psychedelic, I imagine most ultra-wealthy donors would not fall into this group and are more likely to dislike this pursuit. Donating may even become associated with promoting drug use, and even if the ultra-wealthy donors are personally okay with this, they may see donating to EA as damaging the reputation of their business ventures.

Furthermore, while researching psychedelic therapies likely would be beneficial, there already exist (many?) organizations pursuing this line of avenue. Hence, EA's impact likely wouldn't totally transform the field, and this non-massive impact comes at the cost of EA being able to secure funding for other cause areas and being able to serve as an advocate for effective philanthropy.

EDIT: I see that agdfoster is already making an argument along the lines of this (and posted this argument before I did). Although, perhaps my focus on "damage to ultra-wealthy donor recruitment" is crucial to drilling home the importance of EA maintaining it's brand.