with Edward S Boyden, Anthony M Zador, George M Church, Young-Gyu Yoon, Richie Kohman, Alex Vaughan and many collaborators
We have been working to help initiate the field of optical connectomics using the combination of expansion microscopy, in-situ sequencing, and machine learning. This would enable high-throughput tracing of neurons using information-rich barcodes (see the work of the Zador lab at CSHL and our IARPA MICRONS project) in combination with high spatial resolution (via improved forms of expansion microscopy combined with fluorescent in-situ sequencing) and molecular annotations such as in-situ transcriptomics and highly multiplexed proteomics.
Computational paper modeling the combination of cell barodes and 20x expansion microscopy for optical connectomic reconstruction: Frontiers
Early Rosetta Brain preprint explaining the principles and goals of in-situ barcode connectomics: ArXiv
Early calculations outlining scalability considerations for optical, molecular, electron microscopy and other connectomics approaches: bioRxiv
news: Some of our recent results on in-situ neuronal barcoding are featured in this article about the IARPA MICRONS project. Here is an image from that article showing in-situ sequencing of RNA barcodes in expanded mouse brain tissue -- the red and green represent two different RNA letters:
news: The Rosetta Brain has been featured in an excellent popular book, The Future of the Brain, edited by Gary Marcus and Jeremy Freeman. Highly recommended!
news: The Rosetta Brain has been featured in a proposal from the Kavli 6 for a National Network of Neurotechnology Centers, as a possible approach for molecularly annotated optical connectomics.
news: FISSEQ-BOINC connectomics approaches will be a part of the IARPA MICRONS program to derive new machine learning algorithms by studying mesoscale cortical circuitry. I am MIT's PI on the project. Science picked up the story here.
with Bradley M. Zamft*, Yael G. Maguire, Mikhail G. Shapiro, Ted Cybulski, Joshua I. Glaser, P. Benjamin Stranges, Reza Kalhor, David A. Dalrymple, Dongjin Seo, Elad Alon, Michel M. Maharbiz, Jose Carmena, Jan Rabaey, Edward S. Boyden**, George M. Church** and Konrad P. Kording**
Simultaneously measuring the activities of all neurons in a mammalian brain at millisecond resolution is a challenge beyond the limits of existing techniques in neuroscience. Entirely new approaches may be required, motivating an analysis of the fundamental physical constraints on the problem. We outline the physical principles governing brain activity mapping using optical, electrical, magnetic resonance, and molecular modalities of neural recording.
Publisher link: Front. in Comp. Neurosci.
Commentary in Nature Physics.
news: A related paper on "Spatial Information in Large-Scale Neural Recordings" has been published, studying the statistics of large-scale neural recordings.
news: Several collaborators on this work are tapped by the neurotechnology company Kernel to help develop the next generation of scalable human brain interfaces. See my EMTech talk about the approach I led as Kernel's Chief Strategy Officer, leading to their current focus on novel forms of non-invasive brain interfacing.
with Sam Rodriques, Max Mankin, Lowell Wood and Ed Boyden
In a followup to our work on physical principles for scalable neural recording, we introduce a fiber-optic architecture for neural recording without contrast agents, and study its properties theoretically. Our sensor design is inspired by electrooptic modulators. A key concept of the design is the ability to create an "intensified" electric field inside an optical waveguide by applying the extracellular voltage from a neural spike over a nanoscopic distance. Implementing this concept requires the use of ultrathin high-dielectric capacitor layers. If suitable materials can be found, then such sensing fibers could, in principle, be scaled down to few-micron cross-sections for minimally invasive neural interfacing. Custom-designed multi-material optical fibers, probed using a reflectometric readout, may therefore provide a powerful platform for neural sensing.
Journal of Biomedical Optics paper: html
with Greg Wayne, Ken Hayworth and Konrad Kording
Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures. Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion and various forms of short- and long-term memory storage. Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas. We hypothesize that (1) the brain optimizes cost functions, (2) these cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses.
with Gary Marcus and Tom Dean
The goal of this project is to catalyze the characterization of an integrative taxonomy and phylogeny of neural computations: what such circuits achieve, how they are realized both neurally and computationally, and how they have diverged over evolutionary time. A focus is on exploring the potential neural underpinnings of "variable binding", a problem of relevance to modern artificial intelligence.
Short essay: PDF
Publisher link: html
Summary in NY Times Sunday Review.
These analyses led the Kavli Foundation to sponsor a series of workshops organized by Gary, Tomaso Poggio and myself. The purpose of the workshops was to: "bring together leading theorists and experimentalists to work together to (a) evaluate our state of knowledge of cortical computation from an integrative perspective, bridging levels of analysis, (b) to develop and evaluate a taxonomy of candidate cortically implemented computations and (c) to outline potential pathways towards experimentally resolving the longstanding question of the uniformity vs. heterogeneity of mesoscale cortical computation". Here are the questions that we asked participants.
work led by Ken Hayworth
Lecture: Slides and Video
The thalamus appears to be involved in the flexible routing of information among cortical areas, yet the computational implications of such routing are only beginning to be explored. Here we create a connectionist model of how selectively gated cortico-thalamo-cortical relays could underpin both symbolic and sub-symbolic computations.
with Brad Zamft*, Konrad Kording, Daniel Schmidt, Daniel Martin-Alarcon, Keith Tyo, Ed Boyden and George Church
High-throughput recording of signals embedded within inaccessible micro-environments is a technological challenge. The ideal recording device would be a nanoscale machine capable of quantitatively transducing a wide range of variables into a molecular recording medium suitable for long-term storage and facile readout in the form of digital data. We have recently proposed such a device, in which cation concentrations modulate the misincorporation rate of a DNA polymerase (DNAP) on a known template, allowing DNA sequences to encode information about the local cation concentration. In this work we quantify the cation sensitivity of DNAP misincorporation rates, making possible the indirect readout of cation concentration by DNA sequencing.
Publication on using next-generation sequencing to measure ion-dependent DNA polymerase error rates: HTML
Publication on improved next-generation sequencing based polymerase error mapping using nucleotide imbalances: HTML
Note: this project is still at a very preliminary stage of experimentation!
news: We have published a theoretical paper (Joshua I. Glaser et al, PLoS Computational Biology) on the statistical inverse problems associated with molecular ticker-tapes, and how they constrain the synthetic biology implementation challenges.
news: We have received a Transformative Research grant from the NIH to further investigate the recording of dynamic neuronal activities into biopolymer data storage media! The full list of 2013 awardees is here.
Commentary in Nature.