Hierarchical Bayesian Regression for experimental psychology: A case study of cognitive control

Arising from the so-called 'replication crisis' in the experimental psychology literature, there has been a growing call to reassess whether specific analytic practices might lead to the improved accuracy and precision of given findings. This issue is explored here, through a case study examination of two previously collected datasets from the Dual Mechanisms of Cognitive Control (DMCC) task battery. This case study highlights the unique advantages afforded by Hierarchical Bayesian Regression (HBR) models as a potentially more rigorous analytic approach to statistical inference. In the DMCC datasets, two sets of HBR models are presented, with the estimates of the former used as priors for the latter. In addition to systematically generating cumulative posterior distributions for all effects of theoretical interest, we further illustrate how our particular application of HBR models provides novel insights regarding specific proactive/reactive control indicators in each of the four DMCC tasks, by: 1) accurately modeling the specific properties of response time distributions; 2) appropriately modeling trial-level accuracy effects; 3) assessing point estimate inter-sample reliability; and 4) estimating the relative strength of null effects.