Machine Learning and Psychological Research: The Unexplored Effect of Measurement
Published in Perspectives on Psychological Science, 2020
Recommended citation: Jacobucci, R., & Grimm, K. J. (2020). "Machine Learning and Psychological Research: The Unexplored Effect of Measurement." Perspectives on Psychological Science, 15(3), 809-816.
Abstract
Machine learning techniques have gained popularity in psychological research for their flexibility in model fitting and superior predictive performance. However, measurement errors prevent machine-learning algorithms from accurately modeling nonlinear relationships, if indeed they exist. Through simulated examples, we demonstrate that model selection between a machine-learning algorithm and regression depends on measurement quality, regardless of sample size.
Key Findings
- Measurement Quality is Critical: “Garbage in, garbage out” principle applies crucially to machine learning in psychology
- Nonlinear Relationships: ML algorithms struggle to detect true nonlinear relationships when measurement error is present
- Sample Size Limitations: Even large samples cannot overcome poor measurement quality
- Model Selection Impact: Choice between ML and traditional regression depends heavily on measurement reliability
Research Impact
- Highly Cited: Ranks in top 25% of all research outputs on Altmetric
- Methodological Influence: Changed how researchers think about ML applications in psychology
- Replication Implications: Highlights why ML “promise” has been “somewhat unmet” in psychology
Practical Implications
This work demonstrates that before applying sophisticated machine learning algorithms, psychological researchers must first ensure high-quality measurement. The findings suggest that improving measurement instruments may be more valuable than developing more complex algorithms.
Recommended citation: Jacobucci, R., & Grimm, K. J. (2020). “Machine Learning and Psychological Research: The Unexplored Effect of Measurement.” Perspectives on Psychological Science, 15(3), 809-816.