This is a linkpost for Ten Commandments for Aspiring Superforecasters by Good Judgment[1]. I think the lessons also apply outside of explicit forecasting.
In Superforecasting: The Art and Science of Prediction, Good Judgment co-founder Philip Tetlock and his co-author Dan Gardner summarize the Good Judgment Project research findings in the form of “Ten Commandments for Aspiring Superforecasters.” These commandments describe behaviors that have been “experimentally demonstrated to boost [forecasting] accuracy.” You can learn more about these commandments—and practice applying them under the guidance of professional Superforecasters—at one of Good Judgment’s training workshops.
Here are the 10 commandments:
- Triage. "Focus on questions where your hard work is likely to pay off".
- Break seemingly intractable problems into tractable sub-problems.
- Strike the right balance between inside and outside views.
- Strike the right balance between under- and overreacting to evidence.
- Look for the clashing causal forces at work in each problem.
- Strive to distinguish as many degrees of doubt as the problem permits but no more.
- Strike the right balance between under- and overconfidence, between prudence and decisiveness.
- Look for the errors behind your mistakes but beware of rearview-mirror hindsight biases.
- Bring out the best in others and let others bring out the best in you.
- Master the error-balancing bicycle. "Implementing each commandment requires balancing opposing errors".
Last but not least, there is the “11th commandment”:
“It is impossible to lay down binding rules,” Helmuth von Moltke warned, “because two cases will never be exactly the same.”
- ^
I wanted this to be a crosspost, but it was not possible due to copyright restrictions.
I think this leaves out what is perhaps the most important step in making a quality forecast, which is to consider the baserates!
Nice point, Mathias! I agree reference class forecasting is super important. I think it is supposed to be included in the 3rd commandment about the inside and outside view: