Machine Learning and Psychological Research: The Unexplored Effect of Measurement

Abstract

Machine learning techniques have gained popularity in psychological research for their flexibility in model fitting and superior predictive performance. However, measurement errors prevent machine-learning algorithms from accurately modeling nonlinear relationships, if indeed they exist. Through simulated examples, we demonstrate that model selection between a machine-learning algorithm and regression depends on measurement quality, regardless of sample size.

Key Findings

• Measurement Quality is Critical: “Garbage in, garbage out” principle applies crucially to machine learning in psychology
• Nonlinear Relationships: ML algorithms struggle to detect true nonlinear relationships when measurement error is present
• Sample Size Limitations: Even large samples cannot overcome poor measurement quality
• Model Selection Impact: Choice between ML and traditional regression depends heavily on measurement reliability

Research Impact

• Highly Cited: Ranks in top 25% of all research outputs on Altmetric
• Methodological Influence: Changed how researchers think about ML applications in psychology
• Replication Implications: Highlights why ML “promise” has been “somewhat unmet” in psychology

Practical Implications

This work demonstrates that before applying sophisticated machine learning algorithms, psychological researchers must first ensure high-quality measurement. The findings suggest that improving measurement instruments may be more valuable than developing more complex algorithms.

Recommended citation: Jacobucci, R., & Grimm, K. J. (2020). “Machine Learning and Psychological Research: The Unexplored Effect of Measurement.” Perspectives on Psychological Science, 15(3), 809-816.