Closing the Loop to Discover Psychological Theories with an Automated Cognitive Scientist

Abstract

Theory generation in cognitive science has remained a manual process even as data collection, modeling, and experiment design have been automated. We present AutoCog (Automated Cognitive Scientist), a fully autonomous agentic-AI system that closes this loop: large-language-model agents advocate competing theories, each expressed as an executable cognitive model, design experiments that best discriminate them, collect behavioral data from participants recruited online, score theories against collected data based on their generative performance, diagnose why they fail, and synthesize a better successor. Tested in decision-making domains, the system recovered known strategies from simulated data and outperformed established theories when working with human participants, discovering an unconventional theory about multi-cue decision-making that subsequent preregistered research confirmed.