There's not going to be a one-size-fits-all answer to this. EA (implicitly and explicitly) criticises how many other worldviews see the world, and as such we get a lot of criticism back. However, it is a topic I've thought a bit about, so here are some best guesses at the 'visions' of some of our critics put into four groups. [Note: I wrote this up fairly quickly, so please point out any disagreements, mistakes, or suggest additional groups that I've missed]
1: Right-of-centre Sceptics: Critics from this school may think kind of well of EAs intentions, but think we are naïve and/or hubristic, and place us in the a tradition of thought that relies on central planning rather than market solutions. They'd argue along the lines of the most efficient interventions being the spread of markets and the rule of law rather than charities. They may also, if on the more social conservative end, believe that social traditions capture cultural knowledge than can't be captured by quantification or first-principles reasoning. Example critic: Tyler Cowen
2: Super Techno-Optimistic Libertarians: This set thinks that EA has been captured by 'wokeness'/'AI doomers'/whatever Libertarian boogeyman you can think of here. Generally dismissive of EAs, EA institutions, and not really willing to engage on object-level discussions in my experience. Their favoured intervention is probably cutting corporate taxes, removing regulations, and increased funding on AI capabilities so we can go as fast as possible to reap the huge benefits they expect.
In a way, this group acts as a counter-point to some other EA critics, who don't see a true distinction between us and this group, perhaps because many of them live in the Bay and are socially similar to/entangled with EAs there. Example critic: Perry Metzger/Mark Andreessen
3: Decentralised Democrats: There are some similarities to group 1 here, in the sense that critics in this group think that EAs are too technocratic. Sources of disagreement here include pragmatic ones: they are likely to believe that social institutions are not adapted to the modern world to such a degree that fixing them is higher priority than 'core EA' think, normative ones: they likely believe that decisions that will have a large impact over the future deserve the consent of as much of the world as possible and not just the acceptance of whatever EA thinks, and sociological ones: if I had to guess, I'd say they're more central-left/liberaltarian than other EA critics. Very likely to think that distinguishing from EA-as-belief and EA-as-institutions is a false distinction, and very supportive of reforms to EA including community democratisation. Example critic: E. Glen Weyl/Zoe Cremer
4: Radical Progressives/Anti-capitalists: This group is probably the one that you're thinking of in terms of 'our biggest critics', and they've been highly critical of EA since the beginning. They generally believe EA to be actively harmful, and usually ascribe this to either deliberate design or EA being blind to its support of oppressive ideologies/social structures. There's probably a lot of variation in what kind of world they do want, but it's likely to be a very radical departure, probably involving mass cultural and social change (perhaps revolutionary change), ending capitalism as it is currently constituted, and more money, power, and support being given to the State to bring about positive changes.
There is a lot of variation in this group, though you can pick up on some common themes (e.g. a more Hickel-esque view of human progress, compared to a more 'Pinkerite' view that EA might have), common calls-to-action (climate change is probably the largest/most important cause area here). I suggest you don't take my word for it and read them yourself, but I think you won't find much in terms of practical policy suggestions - perhaps because that's seen as "working within a fatally flawed system", but some in this group are more moderate. Example critic: Alice Crary/Emile Torres/Jason Hickel
I think this is generally right but misunderstands how 3 and 4 are often a continuum. I think the biggest change post-FTX is that people who are on the high-status left (e.g. Amia Srinivasan who wrote a critical but collegial critique in the LRB in 2015) now have switched to a more critical tack (e.g. the prelude to Crary's book).
There's a version of the critique that is a soft-left critique of Effective Altruism being too friendly to capitalism and existing power structures versus a critique of EA as actively disingenuous and bad faith (e.g. Torres).
This reply is really thorough and I appreciate the clarity in world views you describe (without strawmanning!) in addition to examples of specific critics. Thank you!
If you find reading criticism from the last group demotivating and "bad faith", try this podcast with the great Habiba Banu on EA and the Left:
https://forum.effectivealtruism.org/posts/6NnnPvzCzxWpWzAb8/podcast-the-left-and-effective-altruism-with-habiba-islam
I think it does a great job pointing out both agreements and disagreements between EA and the Left.