Dissecting curious exploration with self-supervised machine learning

Our drive to seek new information is powerful. As children, we play — experimenting to establish the relationship between our actions and the environment. As adults, we consume media — searching for news and entertainment. As scientists, we explore — carrying out complicated experiments that probe and refine our understanding of nature. Sometimes this drive has clear underlying motivations: improving job performance, obtaining food, finding shelter. In other cases, the direct connection to application is less apparent: staying awake to read a suspenseful book, pursuing basic science research, poking your head through an unlocked door.

Though influential — daily and throughout our lives — the drive of curiosity is poorly understood. What are the principles that guide curiosity-based exploration? What is the neural circuitry that implements curiosity? How can insights related to the phenomenon of curiosity improve the education and capabilities of humans and artificially intelligent agents?

To address these questions, I will take an interdisciplinary approach — positioned at the intersection of computer science, neuroscience, and psychology. One primary goal of this project is to build upon and improve emerging computer models of intrinsically motivated, self-supervised artificial agents. In doing so, we will establish a testbed for comparing different models and types of curiosity. Importantly, however, we will explicitly design our testbed such that it can be translated to rodent models, where we will investigate comparisons between animal behavior and various curiosity formulations.

Insights from this initial work will provide a framework for designing experimental tasks for animal models to enable in vivo study of neuronal circuit dynamics underlying curiosity. Ultimately, in addition to informing educational strategies and the capabilities of artificial intelligence, a mechanistic description of the desire and reward associated with information-seeking may also impact treatment of mental disorders such depression and ADHD.