Title:
Just Effect Sizes? Risking Distributive Injustice in Evidence-Based Decision-Making
Abstract:
Researchers often report how effective tested interventions are using effect size measures, and such effect sizes are widely used to inform evidence-based decision-making. In this talk, I expose a risk with this practice: reporting effect sizes to decision-makers risks distributing their opportunities to learn decision-relevant information about the effectiveness of interventions unfairly. Using expected utility theory, we can show that reporting effect sizes omits information that is more decision-relevant for more risk-averse rational decision-makers than for less risk-averse ones. In this talk, I develop this result for a common effect size measure, the mean difference. Combining this formal result with empirical findings on how risk aversion correlates with socio-demographic factors such as gender reveals how reporting effect sizes like the mean difference risks providing already disadvantaged groups with worse opportunities to learn about the effectiveness of interventions. Following recent work on distributive epistemic justice in science, we can recognise such disadvantages as pro tanto injustices in how science distributes opportunities to learn about the effectiveness of interventions. How could science avoid such injustices? I conclude by sketching various reform proposals for studying and communicating the effectiveness of interventions, and by discussing how their feasibility and desirability depend on largely neglected philosophical and empirical questions.