Publications

2023

vak: a neural network framework for researchers studying animal acoustic communication
Nicholson D. & Cohen Y. (2023) Proceedings of the 22nd Python in Science Conference (SciPy 2023). Calloway C., Niederhut D. & Agarwal M.(eds.). p. 59-67 Abstract
How is speech like birdsong? What do we mean when we say an animal learns their vocalizations? Questions like these are answered by studying how animals communicate with sound. As in many other fields, the study of acoustic communication is being revolutionized by deep neural network models. These models enable answering questions that were previously impossible to address, in part because the models automate analysis of very large datasets. Acoustic communication researchers have developed multiple models for similar tasks, often implemented as research code with one of several libraries, such as Keras and Pytorch. This situation has created a real need for a framework that allows researchers to easily benchmark multiple models, and test new models, with their own data. To address this need, we developed vak (https://github.com/vocalpy/vak), a neural network framework designed for acoustic communication researchers.(" vak" is pronounced like " talk" or" squawk" and was chosen for its similarity to the Latin root voc, as in " vocal".) Here we describe the design of the vak, and explain how the framework makes it easy for researchers to apply neural network models to their own data. We highlight enhancements made in version 1.0 that significantly improve user experience with the library. To provide researchers without expertise in deep learning access to these models, vak can be run via a command-line interface that uses configuration files. Vak can also be used directly in scripts by scientist-coders. To achieve this, vak adapts design patterns and an API from other domain-specific PyTorch libraries such as torchvision …

2022

Recent Advances at the Interface of Neuroscience and Artificial Neural Networks
Cohen Y., Engel T., Langdon C., Lindsay G., Ott T., Peters M., Shine J., Breton-Provencher V. & Ramaswamy S. (2022) The Journal of Neuroscience. 42, 45, p. 8514-8523 Abstract
Biological neural networks adapt and learn in diverse behavioral contexts. Artificial neural networks (ANNs) have exploited biological properties to solve complex problems. However, despite their effectiveness for specific tasks, ANNs are yet to realize the flexibility and adaptability of biological cognition. This review highlights recent advances in computational and experimental research to advance our understanding of biological and artificial intelligence. In particular, we discuss critical mechanisms from the cellular, systems, and cognitive neuroscience fields that have contributed to refining the architecture and training algorithms of ANNs. Additionally, we discuss how recent work used ANNs to understand complex neuronal correlates of cognition and to process high throughput behavioral data.
Automated annotation of birdsong with a neural network that segments spectrograms
Cohen Y., Nicholson D. A., Sanchioni A., Mallaber E. K., Skidanova V. & Gardner T. J. (2022) eLife. 11, e63853. Abstract
Songbirds provide a powerful model system for studying sensory-motor learning. However, many analyses of birdsong require time-consuming, manual annotation of its elements, called syllables. Automated methods for annotation have been proposed, but these methods assume that audio can be cleanly segmented into syllables, or they require carefully tuning multiple statistical models. Here we present TweetyNet: a single neural network model that learns how to segment spectrograms of birdsong into annotated syllables. We show that TweetyNet mitigates limitations of methods that rely on segmented audio. We also show that TweetyNet performs well across multiple individuals from two species of songbirds, Bengalese finches and canaries. Lastly, we demonstrate that using TweetyNet we can accurately annotate very large datasets containing multiple days of song, and that these predicted annotations replicate key findings from behavioral studies. In addition, we provide open-source software to assist other researchers, and a large dataset of annotated canary song that can serve as a benchmark. We conclude that TweetyNet makes it possible to address a wide range of new questions about birdsong.

2021

The geometry of neuronal representations during rule learning reveals complementary roles of cingulate cortex and putamen
Cohen Y., Schneidman E. & Paz R. (2021) Neuron. 109, 5, p. 839-851.e9 Abstract
Learning new rules and adopting novel behavioral policies is a prominent adaptive behavior of primates. We studied the dynamics of single neurons in the dorsal anterior cingulate cortex and putamen of monkeys while they learned new classification tasks every few days over a fixed set of multi-cue patterns. Representing the rules and the neuronal selectivity as vectors in the space spanned by a set of stimulus features allowed us to characterize neuronal dynamics in geometrical terms. We found that neurons in the cingulate cortex mainly rotated toward the rule, implying a policy search, whereas neurons in the putamen showed a magnitude increase that followed the rotation of cortical neurons, implying strengthening of confidence for the newly acquired rule-based policy. Further, the neural representation at the end of a session predicted next-day behavior, reflecting overnight retention. The novel framework for characterization of neural dynamics suggests complementing roles for the putamen and the anterior cingulate cortex.

Accepted version
A novel approach to the empirical characterization of learning in biological systems
Cohen Y., Cvitanović P. & Solla S. A. (2021) BioRxiv. Abstract
Learning to execute precise, yet complex, motor actions through practice is a trait shared by most organisms. Here we develop a novel experimental approach for the comprehensive investigation and characterization of the learning dynamics of practiced motion. Following the dynamical systems framework, we consider a high-dimensional behavioral space in which a trial-by-trial sequence of motor outputs defines a trajectory that converges to a fixed point - the desired motor output. In this scenario, details of the internal dynamics and the trial-by-trial learning mechanism cannot be disentangled from behavioral noise for nonlinear systems or even well estimated for linear systems with many parameters. To overcome this problem, we introduce a novel approach: the sporadic application of systematic target perturbations that span the behavioral space and allow us to estimate the linearized dynamics in the vicinity of the fixed point. The steady-state Lyapunov equation then allows us to identify the noise covariance. We illustrate the method by analyzing sequence-generating neural networks with either intrinsic or extrinsic noise, at time resolutions that span from spike timing to spiking rates. We demonstrate the utility of our approach in experimentally plausible and realizable settings and show that this method can fully characterize the linearized between-trials learning dynamics as well as extract meaningful internal properties of the unknown mechanism that drives the motor output within each trial. We then illustrate how the approach can be extended to nonlinear learning dynamics through a flexible choice of the basis and magnitude of perturbations.
Large-scale cellular-resolution imaging of neural activity in freely behaving mice
Leman D., Chen I., Bolding K., Tai J., Wilmerding L., Gritton H., Cohen Y., Yen W., Perkins L. N., Liberti, 3rd W. A., Kilic K., Han X., Cruz-Martín A., Gardner T. J., Otchy T. M. & Davison I. (2021) BioRxiv. Abstract
Miniaturized microscopes for head-mounted fluorescence imaging are powerful tools for visualizing neural activity during naturalistic behaviors, but the restricted field of view of first-generation ‘miniscopes’ limits the size of neural populations accessible for imaging. Here we describe a novel miniaturized mesoscope offering cellular-resolution imaging over areas spanning several millimeters in freely moving mice. This system enables comprehensive visualization of activity across entire brain regions or interactions across areas.

2020

TweetyNet: A neural network that enables high-throughput, automated annotation of birdsong
Cohen Y., Nicholson D., Sanchioni A., Mallaber E., Skidanova V. & Gardner T. J. (2020) BioRxiv. Abstract
Songbirds have long been studied as a model system of sensory-motor learning. Many analyses of birdsong require time-consuming manual annotation of the individual elements of song, known as syllables or notes. Here we describe the first automated algorithm for birdsong annotation that is applicable to complex song such as canary song. We developed a neural network architecture, “TweetyNet”, that is trained with a small amount of hand-labeled data using supervised learning methods. We first show TweetyNet achieves significantly lower error on Bengalese finch song than a similar method, using less training data, and maintains low error rates across days. Applied to canary song, TweetyNet achieves fully automated annotation of canary song, accurately capturing the complex statistical structure previously discovered in a manually annotated dataset. We conclude that TweetyNet will make it possible to ask a wide range of new questions focused on complex songs where manual annotation was impractical.
Hidden neural states underlie canary song syntax
Cohen Y., Shen J., Semu D., Leman D. P., Liberti, 3rd W. A., Perkins L. N., Liberti D. C., Kotton D. N. & Gardner T. J. (2020) Nature (London). 582, 7813, p. 539-544 Abstract
Coordinated skills such as speech or dance involve sequences of actions that follow syntactic rules in which transitions between elements depend on the identities and order of past actions. Canary songs consist of repeated syllables called phrases, and the ordering of these phrases follows long-range rules¹ in which the choice of what to sing depends on the song structure many seconds prior. The neural substrates that support these long-range correlations are unknown. Here, using miniature head-mounted microscopes and cell-type-specific genetic tools, we observed neural activity in the premotor nucleus HVC^2–4 as canaries explored various phrase sequences in their repertoire. We identified neurons that encode past transitions, extending over four phrases and spanning up to four seconds and forty syllables. These neurons preferentially encode past actions rather than future actions, can reflect more than one song history, and are active mostly during the rare phrases that involve history-dependent transitions in song. These findings demonstrate that the dynamics of HVC include ‘hidden states’ that are not reflected in ongoing behaviour but rather carry information about prior actions. These states provide a possible substrate for the control of syntax transitions governed by long-range rules.

2018

Amorphous silicon carbide platform for next generation penetrating neural interface designs
Deku F., Frewin C. L., Stiller A., Cohen Y., Aqeel S., Joshi-Imre A., Black B., Gardner T. J., Pancrazio J. J. & Cogan S. F. (2018) Micromachines (Basel). 9, 10, 480. Abstract
Microelectrode arrays that consistently and reliably record and stimulate neural activity under conditions of chronic implantation have so far eluded the neural interface community due to failures attributed to both biotic and abiotic mechanisms. Arrays with transverse dimensions of 10 μm or below are thought to minimize the inflammatory response; however, the reduction of implant thickness also decreases buckling thresholds for materials with low Young's modulus. While these issues have been overcome using stiffer, thicker materials as transport shuttles during implantation, the acute damage from the use of shuttles may generate many other biotic complications. Amorphous silicon carbide (a-SiC) provides excellent electrical insulation and a large Young's modulus, allowing the fabrication of ultrasmall arrays with increased resistance to buckling. Prototype a-SiC intracortical implants were fabricated containing 8 - 16 single shanks which had critical thicknesses of either 4 μm or 6 μm. The 6 μm thick a-SiC shanks could penetrate rat cortex without an insertion aid. Single unit recordings from SIROF-coated arrays implanted without any structural support are presented. This work demonstrates that a-SiC can provide an excellent mechanical platform for devices that penetrate cortical tissue while maintaining a critical thickness less than 10 μm.
Amorphous silicon carbide ultramicroelectrode arrays for neural stimulation and recording
Deku F., Cohen Y., Joshi-Imre A., Kanneganti A., Gardner T. J. & Cogan S. F. (2018) Journal of Neural Engineering. 15, 1, 016007. Abstract
Objective. Foreign body response to indwelling cortical microelectrodes limits the reliability of neural stimulation and recording, particularly for extended chronic applications in behaving animals. The extent to which this response compromises the chronic stability of neural devices depends on many factors including the materials used in the electrode construction, the size, and geometry of the indwelling structure. Here, we report on the development of microelectrode arrays (MEAs) based on amorphous silicon carbide (a-SiC). Approach. This technology utilizes a-SiC for its chronic stability and employs semiconductor manufacturing processes to create MEAs with small shank dimensions. The a-SiC films were deposited by plasma enhanced chemical vapor deposition and patterned by thin-film photolithographic techniques. To improve stimulation and recording capabilities with small contact areas, we investigated low impedance coatings on the electrode sites. The assembled devices were characterized in phosphate buffered saline for their electrochemical properties. Main results. MEAs utilizing a-SiC as both the primary structural element and encapsulation were fabricated successfully. These a-SiC MEAs had 16 penetrating shanks. Each shank has a cross-sectional area less than 60 μm² and electrode sites with a geometric surface area varying from 20 to 200 μm². Electrode coatings of TiN and SIROF reduced 1 kHz electrode impedance to less than 100 kω from ∼2.8 Mω for 100 μm² Au electrode sites and increased the charge injection capacities to values greater than 3 mC cm^-2. Finally, we demonstrated functionality by recording neural activity from basal ganglia nucleus of Zebra Finches and motor cortex of rat. Significance. The a-SiC MEAs provide a significant advancement in the development of microelectrodes that over the years has relied on silicon platforms for device manufacture. These flexible a-SiC MEAs have the potential for decreased tissue damage and reduced foreign body response. The technique is promising and has potential for clinical translation and large scale manufacturing.
Calcium imaging in canary (serinus canaria) HVC reveals latent states supporting behavioral sequencing with long range history dependence
Cohen Y., Shen J., Semu D., Otchy T. M. & Gardner T. J. (2018) Abstract
History dependent behavior is a key readout of neural processing. In skills, like speech or dance, motor sequences follow syntactic rules in which transitions between motor elements rely on past actions. Canary songs are defined by syllable repeats, called phrases, whose syntax exhibits long range order. The phrase sequence neural underpinnings must either rely on fixed action patterns or maintain historic context to influence ongoing transitions. To discriminate such mechanisms, we recorded Ca2+ signals from the premotor nucleus HVC in freely behaving canaries. We find that song history is reflected in identified ROIs up to 4 phrases apart, spanning up to 3 seconds and 40 syllables and that some ROIs exhibit mixed history selectivity. Moreover, we find that signals, reflecting sequence history information are more frequent during phrase transitions that are history dependent compared to history insensitive ones. These findings suggest that the network dynamics reflects historic context relevant to flexible transitions. Additionally, we find ROIs whose signals last several seconds and span 3-4 phrases. These signals are rarely modulated by syllable or phrase boundaries and initiate mostly during stereotyped sequences, suggesting distinct network dynamics during stereotyped and variable behavior.

2015

It All Depends on the Context, but Also on the Amygdala
Cohen Y. & Paz R. (2015) Neuron. 87, 4, p. 678-680 Abstract
Behavioral flexibility requires the brain to maintain and rely on cognitive contexts for dictating appropriate responses. Saez et al. (2015) demonstrate that such abstract rule-based representations co-exist in prefrontal cortices and in the amygdala, with the latter being surprisingly crucial for correct performance.

2013

High-order feature-based mixture models of classification learning predict individual learning curves and enable personalized teaching
Cohen Y. & Schneidman E. (2013) Proceedings of the National Academy of Sciences of the United States of America. 110, 2, p. 684-689 Abstract
Pattern classification learning tasks are commonly used to explore learning strategies in human subjects. The universal and individual traits of learning such tasks reflect our cognitive abilities and have been of interest both psychophysically and clinically. From a computational perspective, these tasks are hard, because the number of patterns and rules one could consider even in simple cases is exponentially large. Thus, when we learn to classify we must use simplifying assumptions and generalize. Studies of human behavior in probabilistic learning tasks have focused on rules in which pattern cues are independent, and also described individual behavior in terms of simple, single-cue, feature-based models. Here, we conducted psychophysical experiments in which people learned to classify binary sequences according to deterministic rules of different complexity, including high-order, multicue-dependent rules. We show that human performance on such tasks is very diverse, but that a class of reinforcement learning-like models that use a mixture of features captures individual learning behavior surprisinglywell. Thesemodels reflect the important role of subjects' priors, and their reliance on high-order features even when learning a low-order rule. Further, we show that these models predict future individual answers to a high degree of accuracy. We then use these models to build personally optimized teaching sessions and boost learning.