I have battled with cause prioritization for years. I took a hard turn early in my career that set me back years. I remember it being emotionally difficult to be in the process of a potential large career change, even more difficult to lock in the decision. Hopefully I can say something useful. 

First of all, emotion drive us. It's a force multiplier of all other factors boiling into the amount of impact you can have. Think of all other parameters except for your drive as being a lever you can pull, and your drive is the force you can put on the lever. What will happen if your drive (applied force) is small vs large? 

There are many parameters going into the equation of how much impact you can have in a field, but to mention a few, your experience in the field / track-record, your reputation and strength of your network, more general skills and knowledge, and, of course, your drive to get stuff done. I would think about what drives you the most of:

1. Raw impact, independent of cause area (as partially defined by your very core assumptions / values). 
2. Cause area (your "flavor" / personal preference)

Re-training is a sour apple, for sure, but I wouldn't think too much about that at this (exploration?) phase -- mapping over one fields system of thinking into new domains is often very fruitful in serendipitous ways (I don't know the effect of this in your particular case, but it is generally so). 

And I wouldn't think about this as being irrational, at all -- emotion spurs action. How we are driven to make the world a better place shouldn't be discounted. 

If you really are burning for a specific cause, then I would definitely take that as a strong signal that this question should be further investigated, even in case it isn't seen effective, altruistic or scores high on the ITN framework.