Publications

How is speech like birdsong? What do we mean when we say an animal learns their vocalizations? Questions like these are answered by studying how animals communicate with sound. As in many other fields, the study of acoustic communication is being revolutionized by deep neural network models. These models enable answering questions that were previously impossible to address, in part because the models automate analysis of very large datasets. Acoustic communication researchers have developed multiple models for similar tasks, often implemented as research code with one of several libraries, such as Keras and Pytorch. This situation has created a real need for a framework that allows researchers to easily benchmark multiple models, and test new models, with their own data. To address this need, we developed vak (https://github.com/vocalpy/vak), a neural network framework designed for acoustic communication researchers.(" vak" is pronounced like " talk" or" squawk" and was chosen for its similarity to the Latin root voc, as in " vocal".) Here we describe the design of the vak, and explain how the framework makes it easy for researchers to apply neural network models to their own data. We highlight enhancements made in version 1.0 that significantly improve user experience with the library. To provide researchers without expertise in deep learning access to these models, vak can be run via a command-line interface that uses configuration files. Vak can also be used directly in scripts by scientist-coders. To achieve this, vak adapts design patterns and an API from other domain-specific PyTorch libraries such as torchvision

Learning new rules and adopting novel behavioral policies is a prominent adaptive behavior of primates. We studied the dynamics of single neurons in the dorsal anterior cingulate cortex and putamen of monkeys while they learned new classification tasks every few days over a fixed set of multi-cue patterns. Representing the rules and the neuronal selectivity as vectors in the space spanned by a set of stimulus features allowed us to characterize neuronal dynamics in geometrical terms. We found that neurons in the cingulate cortex mainly rotated toward the rule, implying a policy search, whereas neurons in the putamen showed a magnitude increase that followed the rotation of cortical neurons, implying strengthening of confidence for the newly acquired rule-based policy. Further, the neural representation at the end of a session predicted next-day behavior, reflecting overnight retention. The novel framework for characterization of neural dynamics suggests complementing roles for the putamen and the anterior cingulate cortex.

Miniaturized microscopes for head-mounted fluorescence imaging are powerful tools for visualizing neural activity during naturalistic behaviors, but the restricted field of view of first-generation miniscopes limits the size of neural populations accessible for imaging. Here we describe a novel miniaturized mesoscope offering cellular-resolution imaging over areas spanning several millimeters in freely moving mice. This system enables comprehensive visualization of activity across entire brain regions or interactions across areas.

Learning to execute precise, yet complex, motor actions through practice is a trait shared by most organisms. Here we develop a novel experimental approach for the comprehensive investigation and characterization of the learning dynamics of practiced motion. Following the dynamical systems framework, we consider a high-dimensional behavioral space in which a trial-by-trial sequence of motor outputs defines a trajectory that converges to a fixed point - the desired motor output. In this scenario, details of the internal dynamics and the trial-by-trial learning mechanism cannot be disentangled from behavioral noise for nonlinear systems or even well estimated for linear systems with many parameters. To overcome this problem, we introduce a novel approach: the sporadic application of systematic target perturbations that span the behavioral space and allow us to estimate the linearized dynamics in the vicinity of the fixed point. The steady-state Lyapunov equation then allows us to identify the noise covariance. We illustrate the method by analyzing sequence-generating neural networks with either intrinsic or extrinsic noise, at time resolutions that span from spike timing to spiking rates. We demonstrate the utility of our approach in experimentally plausible and realizable settings and show that this method can fully characterize the linearized between-trials learning dynamics as well as extract meaningful internal properties of the unknown mechanism that drives the motor output within each trial. We then illustrate how the approach can be extended to nonlinear learning dynamics through a flexible choice of the basis and magnitude of perturbations.

Coordinated skills such as speech or dance involve sequences of actions that follow syntactic rules in which transitions between elements depend on the identities and order of past actions. Canary songs consist of repeated syllables called phrases, and the ordering of these phrases follows long-range rules¹ in which the choice of what to sing depends on the song structure many seconds prior. The neural substrates that support these long-range correlations are unknown. Here, using miniature head-mounted microscopes and cell-type-specific genetic tools, we observed neural activity in the premotor nucleus HVC²⁴ as canaries explored various phrase sequences in their repertoire. We identified neurons that encode past transitions, extending over four phrases and spanning up to four seconds and forty syllables. These neurons preferentially encode past actions rather than future actions, can reflect more than one song history, and are active mostly during the rare phrases that involve history-dependent transitions in song. These findings demonstrate that the dynamics of HVC include hidden states that are not reflected in ongoing behaviour but rather carry information about prior actions. These states provide a possible substrate for the control of syntax transitions governed by long-range rules.

Microelectrode arrays that consistently and reliably record and stimulate neural activity under conditions of chronic implantation have so far eluded the neural interface community due to failures attributed to both biotic and abiotic mechanisms. Arrays with transverse dimensions of 10 μm or below are thought to minimize the inflammatory response; however, the reduction of implant thickness also decreases buckling thresholds for materials with low Young's modulus. While these issues have been overcome using stiffer, thicker materials as transport shuttles during implantation, the acute damage from the use of shuttles may generate many other biotic complications. Amorphous silicon carbide (a-SiC) provides excellent electrical insulation and a large Young's modulus, allowing the fabrication of ultrasmall arrays with increased resistance to buckling. Prototype a-SiC intracortical implants were fabricated containing 8 - 16 single shanks which had critical thicknesses of either 4 μm or 6 μm. The 6 μm thick a-SiC shanks could penetrate rat cortex without an insertion aid. Single unit recordings from SIROF-coated arrays implanted without any structural support are presented. This work demonstrates that a-SiC can provide an excellent mechanical platform for devices that penetrate cortical tissue while maintaining a critical thickness less than 10 μm.

Objective. Foreign body response to indwelling cortical microelectrodes limits the reliability of neural stimulation and recording, particularly for extended chronic applications in behaving animals. The extent to which this response compromises the chronic stability of neural devices depends on many factors including the materials used in the electrode construction, the size, and geometry of the indwelling structure. Here, we report on the development of microelectrode arrays (MEAs) based on amorphous silicon carbide (a-SiC). Approach. This technology utilizes a-SiC for its chronic stability and employs semiconductor manufacturing processes to create MEAs with small shank dimensions. The a-SiC films were deposited by plasma enhanced chemical vapor deposition and patterned by thin-film photolithographic techniques. To improve stimulation and recording capabilities with small contact areas, we investigated low impedance coatings on the electrode sites. The assembled devices were characterized in phosphate buffered saline for their electrochemical properties. Main results. MEAs utilizing a-SiC as both the primary structural element and encapsulation were fabricated successfully. These a-SiC MEAs had 16 penetrating shanks. Each shank has a cross-sectional area less than 60 μm² and electrode sites with a geometric surface area varying from 20 to 200 μm². Electrode coatings of TiN and SIROF reduced 1 kHz electrode impedance to less than 100 kω from ∼2.8 Mω for 100 μm² Au electrode sites and increased the charge injection capacities to values greater than 3 mC cm^-2. Finally, we demonstrated functionality by recording neural activity from basal ganglia nucleus of Zebra Finches and motor cortex of rat. Significance. The a-SiC MEAs provide a significant advancement in the development of microelectrodes that over the years has relied on silicon platforms for device manufacture. These flexible a-SiC MEAs have the potential for decreased tissue damage and reduced foreign body response. The technique is promising and has potential for clinical translation and large scale manufacturing.

History dependent behavior is a key readout of neural processing. In skills, like speech or dance, motor sequences follow syntactic rules in which transitions between motor elements rely on past actions. Canary songs are defined by syllable repeats, called phrases, whose syntax exhibits long range order. The phrase sequence neural underpinnings must either rely on fixed action patterns or maintain historic context to influence ongoing transitions. To discriminate such mechanisms, we recorded Ca2+ signals from the premotor nucleus HVC in freely behaving canaries. We find that song history is reflected in identified ROIs up to 4 phrases apart, spanning up to 3 seconds and 40 syllables and that some ROIs exhibit mixed history selectivity. Moreover, we find that signals, reflecting sequence history information are more frequent during phrase transitions that are history dependent compared to history insensitive ones. These findings suggest that the network dynamics reflects historic context relevant to flexible transitions. Additionally, we find ROIs whose signals last several seconds and span 3-4 phrases. These signals are rarely modulated by syllable or phrase boundaries and initiate mostly during stereotyped sequences, suggesting distinct network dynamics during stereotyped and variable behavior.

Behavioral flexibility requires the brain to maintain and rely on cognitive contexts for dictating appropriate responses. Saez et al. (2015) demonstrate that such abstract rule-based representations co-exist in prefrontal cortices and in the amygdala, with the latter being surprisingly crucial for correct performance.

Pattern classification learning tasks are commonly used to explore learning strategies in human subjects. The universal and individual traits of learning such tasks reflect our cognitive abilities and have been of interest both psychophysically and clinically. From a computational perspective, these tasks are hard, because the number of patterns and rules one could consider even in simple cases is exponentially large. Thus, when we learn to classify we must use simplifying assumptions and generalize. Studies of human behavior in probabilistic learning tasks have focused on rules in which pattern cues are independent, and also described individual behavior in terms of simple, single-cue, feature-based models. Here, we conducted psychophysical experiments in which people learned to classify binary sequences according to deterministic rules of different complexity, including high-order, multicue-dependent rules. We show that human performance on such tasks is very diverse, but that a class of reinforcement learning-like models that use a mixture of features captures individual learning behavior surprisinglywell. Thesemodels reflect the important role of subjects' priors, and their reliance on high-order features even when learning a low-order rule. Further, we show that these models predict future individual answers to a high degree of accuracy. We then use these models to build personally optimized teaching sessions and boost learning.