Past events

The spiking of dopamine neurons in animals, and apparently analogous BOLD signals at dopaminergic targets in humans, appear to report predictions of future reward. Prominent computational theories of these responses suggest that they both support and reflect trial-and-error learning about which actions have been successful, based on simple associations with past rewards. This is essentially a neural implementation of Thorndike's (1911) behaviorist principle that reinforced behaviors should be repeated. However, it has long been known that organisms are not condemned merely to repeat previously successful actions, but instead that even rodents' decisions can under some circumstances reflect other sorts of knowledge about task structure and contingencies. The neural and computational bases for these additional effects, and their interaction with the putative reinforcement systems in the basal ganglia, are poorly understood. Such interactions are of considerable practical importance because, for instance, disorders of compulsion in humans, such as substance abuse, are thought to arise from runaway reinforcement processes unfettered by more deliberative influences. I first discuss how such extra-reinforcement effects – e.g., planning novel routes based on cognitive maps, or incorporating "counterfactual" feedback about foregone actions – can be incorporated in the framework of existing computational theories, via algorithms for “model-based reinforcement learning." Rather than learning about actions' past successes directly, such algorithms learn a representation of the task structure, and can use it to evaluate candidate actions via mental simulation of their consequences. This computational characterization allows reasoning about (and explaining empirical data concerning) under which circumstances the brain might efficiently adopt either this strategy or the reinforcement one. It also allows quantifying and dissociating either strategy's effects on decision making and associated neural signaling. Next, I discuss human fMRI experiments characterizing these influences in learning tasks. By fitting computational models to decision behavior and BOLD signals, we demonstrate that neither choices nor (putatively dopamine-related) BOLD signals in striatum can be explained by past reinforcement alone, but instead that both reflect additional learning and reasoning about task structure and contingencies. That such influences are prominent even at the level of striatum challenges current models of the computations there and suggest that the system is a common target for many different sorts of learning. Additional experiments examine individual variation in the tendency to employ either system; the patterns of both spontaneous and experimentally induced variation suggest that the dominance of model-based decision influence over simpler reinforcement systems employs cognitive control mechanisms that have previously been studied in other areas of cognitive neuroscience. Finally, I report results showing that patients with several disorders involving compulsion show abnormally reinforcement-bound choices on our tasks, supporting a link between these neurocomputational learning mechanisms and pathological habits.

Visual perception is commonly viewed as a stimulus-driven process, whereby neural representations of increasing complexity are hierarchically assembled from primary sensory areas through category-selective regions to high-level association areas. Vision provides a great opportunity to study cortical mechanisms of perception, as the ordered hierarchical organization has been amply demonstrated and modeled formally in many computational models. Despite their success, however, computational models rarely perform as well as the biological system, and often fail to take account of the highly interactive nature of cortical networks - involving interactions between different processing pathways as well as across different levels of the hierarchy. In the current talk, I will present a series of neuroimaging studies, which demonstrate how representations in dedicated brain regions in visual cortex emerge from interactions with large-scale networks, exemplifying both functional and neuroanatomical constraints. Specifically, I will describe recent investigations of object- and scene-selective cortex that reveal (1) the large impact that top-down factors, such as experience and task demands have on the neural representations of visual objects and (2) how the distinction between object and scene representations can be accounted for by the patterns of connectivity within and across the ventral and dorsal visual processing pathways.

Time delay is pervasive in neural information processing. To achieve real-time tracking, it is critical to compensate the transmission and processing delays in a neural system. In the present study we show that dynamical synapses with short-term depression can enhance the mobility of a continuous attractor network to the extent that the system tracks time-varying stimuli in a timely manner. The state of the network can either track the instantaneous position of a moving stimulus perfectly (with zero-lag) or lead it with an effectively constant time, in agreement with experiments on the head-direction systems in rodents. The parameter regions for delayed, perfect and anticipative tracking correspond to network states that are static, ready-to-move and spontaneously moving, respectively, demonstrating the strong correlation between tracking performance and the intrinsic dynamics of the network.

The work in our lab focuses on understanding the neural basis of behavior, particularly spatial cognition, in freely-moving, freely behaving mammals – employing the echolocating bat as a novel animal model. I will describe our recent studies, including: (i) recordings of 3-D head-direction cells in the presubiculum of crawling bats, as well as recordings from hippocampal 3-D place cells in freely-flying bats, using a custom neural telemetry system – which revealed an elaborate 3-D spatial representation in the mammalian brain; and (ii) recordings of 'grid cells' in the bat's medial entorhinal cortex, in the absence of theta oscillations – which strongly argues against the prevailing computational model of grid formation. I will also describe our recent studies of spatial memory and navigation of fruit bats in the wild, using micro-GPS devices, which revealed outstanding navigational abilities and provided the first evidence for a large-scale 'cognitive map' in a mammal.