Mind-blowing science: ‘Star Wars-style’ holograms to communicate with the brain

Investigators supported by the Chan Zuckerberg Biohub San Francisco are imagining the future of neuroscience

About 20 years ago, neuroscientists, recording from electrodes implanted in the medial temporal lobe, identified human brain cells that respond only to photos of Jennifer Aniston. It was a headline-grabbing development in a long arc of achievements by scientists in their efforts to map our neural circuits.

The early 2000s saw the advent of optogenetics, a groundbreaking technique that allows scientists to stimulate or inhibit neural activity with beams of light instead of invasive electrodes. That led to even more advances in neurobiology; for example, by activating certain neurons or neural circuits in mice, researchers have been able to treat stroke, or enhance or remove certain memories.

Now a trio of scientists at UC Berkeley are taking brain–machine interfaces to a new level, with a powerful platform that incorporates optogenetics and computer-generated holography in a way that shows promise as a high-throughput, high-precision technique for communication with the brain. The scientists believe it could become a powerful research tool for understanding how the brain is wired and could eventually be used to design therapies to treat brain diseases and disorders rooted in the malfunction of neural circuits.

This system draws on the collective expertise of Laura Waller, an expert in computational imaging; Rikky Muller, whose lab focuses on microelectronics for neural interfaces; and Hillel Adesnik, a professor of neurobiology who studies the neural basis of perception. Waller and Adesnik are Chan Zuckerberg Biohub San Francisco Investigators, and overlapped with Muller, who is now an alum of the Biohub Investigator Program.

Adesnik has dubbed the system a “Hubble telescope for the brain.” Just as the Hubble captures activity in space, the team’s neurotechnology would enable a view into brain activity at an unprecedented level of resolution. But unlike the Hubble, the system would do more than just track signals from neurons; it would also allow researchers to activate them, a two-way communication with the brain that promises to be a key enabling tool for neuroscience.

“We’re developing an instrument that’s going to be able to literally illuminate how diseases – and really all normal functions – form and evolve in the brain,” Muller says.

The research collaboration has already yielded several published papers, as well as a couple of major grants, including a $4 million award from the NIH BRAIN Initiative.

“This project marries the very distinct expertise that each scientist brings, where the new technology they’re trying to develop would not be possible without the fundamental physics and computational insights brought by Laura, the semiconductor design and engineering for implantable devices brought by Rikky, plus the neuroscience provided by Hillel,” says William Burkholder, director of scientific program management at CZ Biohub SF. “It’s a really deep, profound collaboration, and exactly the sort of bold, cross-disciplinary research that our Investigator Program was designed to catalyze and support.”

Challenges of scale and speed

The human brain has 89 billion neurons, and each one, on average, is connected to one thousand other neurons, making for roughly 90 trillion connections. “We want to know that connectome,” Adesnik says.

Not only is the scale daunting, but the physical density of neurons also presents a technical challenge: one cubic millimeter of the brain contains 50,000 to 100,000 neurons. Many conventional neuroscience techniques have involved stimulating the brain with electricity to see which neurons are activated, but this is a relatively crude approach, because electrical signals affect many neurons surrounding a cell of interest. What’s more, electricity can only activate neurons, not inhibit them.

“With electric charge, it’s like hitting the whole area with a hammer, causing everything in the region to respond,” Muller says. “But we can focus light on single neurons. What we’re building toward is something called all-optical neurophysiology, a neural interface where we’re doing both stimulation and recording in the optical domain.”

Bringing ‘Star Wars’ to life

Other methods have been developed for optical neural interfacing at single-cell resolution, such as by 3D scanning, but these techniques have their limitations.

“Most systems use one laser beam and scan in a grid pattern, but most of the time is wasted in the brain because the neurons are not sitting in a nice 2D plane – they’re in 3D, and there’s space between them,” Adesnik explains. “With our approach, we don’t need to waste time moving a beam. We can take a single laser and essentially diffract the beam, turning it into the right three-dimensional pattern, shaped to all the neurons exactly where they are. It’s like the 3D holographic displays in ‘Star Wars’ – that’s a real thing. And then you’re getting all that information in parallel.”

Three all-optical neural interface schemes: in (a) direct illumination with no scanning elements does not have single-cell specificity; in (b) lateral and varifocal elements can target light in 3D but reach only one cell at a time; in (c) a holographic system uses spatial light modulators to project to multiple neurons at a time with single-cell precision. (From Yalcin et al., IEEE J. Solid-State Circuits, 2022)

Muller is developing ultrafast spatial light modulators; they’re typically slow components used in computer display technologies, but she’s speeding them up using fast-switching piston motion MEMS (micro-electromechanical system) micromirrors coupled to an integrated-circuit driver. In prior work her team developed an array of 24,000 MEMS micromirrors for high-speed focusing. “They’re wired up in concentric circles, which allows us to make convex and concave shapes, effectively forming a tunable lens where we can focus or change the depth plane very, very quickly,” she says.

Waller has developed efficient computational algorithms to rapidly generate the needed 3D patterns of light to target the correct neurons. “You want to address as many neurons as possible, and you want to make sure that when you’re lighting up a particular neuron, you’re not lighting up all the ones around it. So you need specificity, and speed is also important because neural activity is happening on a scale of milliseconds,” she says.

Muller adds: “If we’re able to switch really quickly, then we can cycle through multiple holograms in a short period of time, increasing the total number of addressable neurons, and communicating at speeds that neurons naturally communicate at.”

This approach allows the team to increase the tool’s throughput dramatically compared to other systems. “What we can do in principle is scale up, by tenfold on the number of neurons, and by a hundredfold on the speed. The throughput is much higher because we don’t have to move the beam around,” Adesnik says. “No one’s been doing that.”

For example, he can stimulate one neuron and measure the response of the 999 other neurons it’s connected to. “So I can get to a matrix of a thousand by a thousand of who’s connected to whom in a functional way, and it’s the functional connections in the brain that are mediating every aspect of brain computation,” he says.

In optogenetics, fluorescent voltage indicators are introduced into neurons to “read out” when neurons are firing, and light-sensitive proteins called opsins are introduced to “write” to neurons, either triggering or inhibiting firing in response to light. Karl Deisseroth of Stanford and Ed Boyden, now at MIT, were key pioneers in developing this technology, which has revolutionized neuroscience and been used to study everything from memory, mood, and hunger to depression and schizophrenia. With the new Berkeley technology, moving between reading and writing is as simple as changing the color of the laser. The technique is not yet approved for use in humans, but Adesnik has developed a line of mice to experiment with the system.

While an application in humans is still a long way down the road, Adesnik says he can imagine this technology being useful one day for people with glaucoma, to help give them a certain level of vision by writing information to the brain from a camera, similar to how a retinal prosthesis or cochlear prosthesis sends visual or auditory signals to the brain, or for people with spinal cord injuries, or even for people with neuropsychiatric diseases, which may be caused by an imbalance in brain circuitry.

“Our goal is that by using a very-high resolution technology that really interfaces the brain in a naturalistic way, we get closer to a generalizable tool that might work in many different modalities,” he says.

The Biohub as a catalyst

Waller and Muller both went to MIT for undergrad, then both got faculty positions in UC Berkeley’s Department of Electrical Engineering and Computer Sciences. Waller got interested in neuroscience after meeting Adesnik, a professor in UC Berkeley’s Department of Molecular and Cell Biology, through a mutual friend, while Muller had previously co-founded a neurotech startup.

Waller and Muller were working in parallel on neural technologies in their Cory Hall labs on the UC Berkeley campus but did not learn about each other’s work until they attended a meeting for CZ Biohub Investigators in San Francisco in 2017. “When I saw her give a talk at Biohub, it became clear that what we were working on was exceedingly compatible,” Muller says.

They credit CZ Biohub SF not just for bringing them together, but also for providing financial backing for a risky and unproven technology. “Biohub provided the first meaningful funding for this project,” Muller says. “It’s even allowing us to expand our collaborations.”

Besides the Biohub and NIH support, the team was also awarded a grant from the Weill Neurohub, which has allowed them to hire a few joint postdoctoral fellows.

For Burkholder’s part, this collaboration provides proof-of-concept of the Biohub Investigator Program model, which not only awards unrestricted funding for scientists to pursue bold, risky ideas, but in addition seeks to catalyze the cross-pollination of ideas by holding biweekly in-person scientific meetings for award recipients.

“Biological catalysis is simply accelerating chemistry that’s already happening in nature, by bringing the reactants into close proximity and lowering the energies of activation,” Burkholder says. “The Biohub’s mission is to catalyze processes that happen in the academic world anyway, but to make things go much faster. We encourage scientists to get out of their silos and learn from each other. Who knows where it will lead. And this is a stellar example of that.”