Non-paramet Bayesian models of optimal learning and control: applications for sequential decision-making and decoding for bra - Rehabilitation Institute of Chicago

Skip to Content

Fri, Apr 1

Speaker: Daniel E. Acuna (Dr. Kording's group)

Title: Non-parametric Bayesian models of optimal learning and control: applications for sequential decision-making and decoding for brain machine interfaces

Abstract:   One of the key features of the Central Nervous System (CNS) is its ability to extract much information from very few noisy observations and yet be flexible enough to adapt to new situations. One way to understand this feature is through the idea that the CNS is equipped with structured but flexible probabilistic representations of how the environment works--models with a sparse set of strong assumptions that can grow with more data. Non-parametric Bayesian statistics offers the ability to create such probabilistic models in a simple, coherent, and scientifically appealing fashion. And coupled with recent algorithmic advances in Statistics and Machine Learning, non-parametric Bayesian objects enable us to analyze large, noisy, and complex data.In this talk, I will first describe my research on human sequential decision-making under uncertainty. A large body of research has favored the hypothesis that human decision-making is based on "model-free" controllers--i.e., fundamentally limited for learning and generalization--but I argue that it is also largely based on strong but flexible models. I develop an optimal controller using the hierarchical Dirichlet Process and test its ability to mimic human choices in a challenging reinforcement learning task. The results suggest that the model better fits human behavior while offering greater interpretability than previous standard model-free controllers.

Secondly, I will discuss our ongoing research on decoders for brain machine interfaces. The task is to predict hand position based on spike recordings from the primary motor cortex of a monkey. We have constructed a probabilistic model of the evolution of hand kinematics and its interaction with spike rates. The model essentially assumes an unbounded set of switching modes of movement, each controlling a linear dynamical system for kinematics and a spike-to-kinematics regression. I will show preliminary results and highlight the advantages of the approach.