Simulated Brains


If you've consumed any amount of sci-fi media, you've probably come into contact with the idea of mapping someone's brain patterns into a computer. This would create a digital copy of the person that exists separate from their physical self, and endures after the body dies. They could live in a computer or perhaps an artificial body.


Examples of this are found all through media. Shows and movies like Black Mirror, Pantheon, The Matrix, and Altered Carbon explore this idea, as do video games like Cyberpunk 2077 and Soma. There are salient "implanted memory" characters, like Rachael from Blade Runner and Arnim Zola, one of Captain America's villains.


While this idea seems new, it's been bouncing around sci-fi stories since the 1950s. The premise is very provocative and invites moral quandaries - what if you could live forever inside a computer? Is an exact copy of your mind really you?


The idea of simulating a human brain and filling it with memories is still science fiction, but simulating brains of simple organisms is not.


The Fly Connectome


Back in the '80s, researchers mapped the 300 neurons forming the brain of a roundworm. They took images with an electron microscope and spent almost 10 years assembling the diagram. The diagram they created is called a connectome - it details every neuron and each connection between them. While this can't tell researchers what the worm was thinking, it does give a perfectly accurate diagram of where synapses could fire.


C. elegans under a microscope. Courtesy of Wikimedia Commons
A C. elegans roundworm, like the one mapped in the '80s

Researchers have created artificial neural networks using this worm connectome, and "taught" the simulated worm to behave just like a real worm in response to stimuli. This 'digital twin' behaved like a real roundworm, using its virtual muscles in a simulated physics environment to find food.


But a roundworm isn't very exciting - it can't see, and it can hardly do anything. Over the next few decades, researchers mapped more and more complex brains until in 2024 the FlyWire consortium published the first complete wiring of an adult fruit fly brain. A fruit fly, Drosophila melanogaster, is much more interesting than a roundworm. They can fly, find food, see, walk, etc.


Drosophila melanogaster. Courtesy of Wikimedia Commons
Female fruit fly

This fruit fly's brain is roughly the size of a poppy seed, but mapping it was a colossal task. Researchers cut it into thousands of slices, took electron microscope photos of each slice, and wired the layers together. Even after using machine learning to connect the layers, it took 33 person-years of human editing and review before the connectome could be published.


The fly had 139,255 neurons and 54 million synapses. These numbers are astronomically smaller than a human brain's (86 billion neurons and hundreds of trillions of synapses), but still very large. Researchers were able to map most of the neurons and synapses to each of their functions. For example, about 8,000 neurons carry vision from the eyes to the central brain, and 1,300 neurons send signals from the brain to walk, turn, or fly.


Drosophila melanogaster brain scan. Courtesy of Wikimedia Commons
Scan of a fruit fly brain

Artificial Neural Networks and Reinforcement Learning


I work with artificial neural networks every day between my job and PhD studies. Neural networks are composed of artificial neurons, linked together. These artificial neurons perform simple math; they take a weighted sum of inputs and pass them through a simple nonlinearity, then send that information on to the next neuron. In a typical Feedforward Neural Network, information flows from inputs to outputs.


Diagram of a feedforward neural network
Diagram of a simple feedforward neural network.

However, that's not really what an organism's brain does. Real brains are recurrent, so instead we use an RNN (Recurrent Neural Network). RNNs have loops - the last thing a neuron "thought" is fed back into itself. This gives the model a kind of short-term memory.


Diagram of a recurrent neural network
Diagram of a simple recurrent neural network.

With that architecture choice done, we were ready to simulate a brain. Our RNN has one neuron for each neuron in the fly's central brain (excluding the optic lobes), about 61,000 in all. Each connection between two neurons is a weight in our network, with a starting strength based on how many synapses join them in the connectome. Since the researchers had labeled what type each neuron is, I could use the visual projection neurons as inputs, and the descending neurons (the ones that send commands to legs and wings) as outputs.


point diagram of a fruit fly brain
Map of the fly's brain.

We made a few adaptations to normal neural networks while simulating the brain - in normal neural networks, a weight can flip from positive to negative during training, but a real synapse can't switch from exciting a neuron to inhibiting it. So each weight keeps the sign of its real synapse (excitatory weights stay positive, inhibitory weights stay negative), and training only changes how strong each one is. The eyes are also simplified. Instead of simulating each optical neuron, we bypass some of them, instead we provide a simplified view of the screen. The left-eye neurons see the left half of the screen and the right-eye neurons see the right. Just like a real fly!


With a fully simulated, empty brain, I was ready to teach the fly how to do something. I decided to teach it to play one of the first video games ever, Pong.


Pong


Pong was created in 1972 by Allan Alcorn, and distributed by Atari. In the game, the user plays a simplified game of table tennis by moving a paddle. I chose Pong since it was simple, and it seems easy enough that a fly could learn it - flies are hard to catch after all, because they're really good at dodging in 3D. By that logic, I figured they could do the opposite, and hit a ball in 2D.


I made a little pong game to teach the fly. You can try it out at the bottom of this page.


Teaching the Fly


At this point, I had a simulated fly's brain, but it didn't know anything. I had to try teaching it how to play Pong.


To do this, I used a process called reinforcement learning (RL). (Read this about my last reinforcement learning project.) In RL, the agent (in this case our fly) plays the game. Whenever it wins a point, we give it a reward. Whenever it loses a point, we give it a negative reward, or a punishment. To encourage it to start hitting the ball, we also give it a small reward every time it hits the ball with its paddle. Just like in my last RL project, we use the PPO algorithm to train the fly.


At every frame of the game, the fly gets inputs, thinks for 4 steps, then takes an action, either move left, move right, or stay.


Diagram of the fly playing Pong: game state goes to eye features, then through the fly's brain to a left, stay, or right move

Because the simulated fly brain is so small, I was able to train the fly on my gaming GPU (an RTX 2080 Super with 8GB of memory) on my home PC. It played 256 games at the same time, so it trained 5 million frames in about 2 hours.


Additional Experiments


Did it work? Yes! After about 600,000 frames of practice (around 15 minutes of training), the fly was winning 90% of its points against the computer. By the end of training, it returned 98% of the balls hit at it. For comparison, a player pushing buttons at random wins about 13% of points.


But that raises an obvious question: did the fly learn Pong because of its specialized fly brain, or would any neural network of the same size have learned it just as well? To find out, I trained three other "brains" with exactly the same setup, game and amount of practice:


  • A scrambled fly. This fly had the same neurons, and the same number of connections per neuron, but the connections are wired at random instead of following the real fly's connections. If the real fly beats this one, the fly's actual wiring matters.
  • A frozen fly. The real fly's wiring, but the connection strengths are locked in place. Only the eyes, the readout, and each neuron's baseline activity get trained - about 67,000 numbers instead of 1.3 million.
  • A GRU. A standard recurrent neural network (GRU) from machine learning, with no biology in it at all, and roughly the same number of trainable weights as the fly.

Line chart of points won during training for the fly, scrambled fly, frozen fly and GRU. All four climb from about 7% to over 90%.
Points won while training. All four brains learn Pong. Interestingly, the real fly gets there a little faster.

All four learned to play. The real fly hit 90% slightly sooner than the others, but I only trained each brain once, and the gap is small enough that it's within the expected noise. Pong is pretty easy - any network with enough neurons can figure out how to move toward the ball.


The frozen fly was most surprising. Its connections never changed during training, and it still learned to win over 90% of its points - it just took about twice as long. The fly's untrained brain already carried enough information about where the ball was to learn how to play Pong. In a way, the fly was already capable of playing Pong; it just needed to learn the specifics.


This matched my initial intuition. If you've ever tried to kill a fruit fly, you know they're really good at moving and dodging objects, so they probably have the spatial reasoning needed to play Pong.


I had another question - did my fly brain use the neurons we'd expect? One way to test that was to deactivate some of the neurons and test the fly. For example, I could turn off the neurons in the "mushroom body" (the part of the brain that does learning and memory in a real fly), or turn off the input from one eye.


What I Removed Balls Returned Points Won Misses on the Left
Nothing (intact fly) 97.9% 94.0% 76%
Mushroom body (5,608 neurons) 97.7% 93.5% 71%
All inhibitory connections 18.8% 4.5% 100%
Left descending (control) neurons 62.6% 20.6% 99%
Right descending (control) neurons 66.5% 25.5% 1%
Left eye blinded 19.5% 4.5% 100%
Right eye blinded 32.0% 4.8% 0%

These results were pretty interesting. Removing the mushroom body didn't cause much of a problem at all, which suggests the simulated fly doesn't use it to play. However, blinding one eye drastically hurts the fly, and hurts it way more on the side where it can't see (not a huge shock). Interestingly, if we remove connections that would have been inhibitory in the fly (they quiet the next neuron instead of exciting it), the fly's performance plummets and it does worse than random.


Ethics


There is a deep-rooted human belief that making a convincing copy of someone steals their vitality. Crazy Horse famously wouldn't let photographers take his picture, believing that it would diminish his power, and many aboriginal Australians find it disrespectful to view a photo of a deceased person. Similarly a voodoo doll, or small effigy of a person, was thought to give you power over them if the copy was good enough.


While I personally don't believe that making an artificial neural network with the same structure as an organism's brain traps that organism's essence in a computer in some mystical way, it still feels a little off. I created a digital ghost of a fly and forced it to learn how to play Pong by giving it rewards and punishments.


If someone scanned my brain, made an exact copy in a supercomputer, and forced it to do tasks by giving it rewards and punishments, that would seem barbaric to me even though my real mind inside my body is okay. Is this okay because it's only a fly? Is it not? I'm not sure.


On the other hand, it is estimated that over 100 million animals are killed each year worldwide in laboratory experiments. If we could create digital twins of animals and run necessary scientific experiments on them, we could get the scientific benefits without killing so many creatures.


Conclusion


As I finish this post, Halloween approaches and this is definitely one of my spookiest projects. In some sense, I simulated the ghost of a fly and taught it to play a video game. That's probably the most "mad scientist" project I've ever done.


Demo


Of course, I've made an interactive demo. Here you can play the same Pong game the fly played against the computer, watch the fly play the computer, or try and beat the fly!



Citations


Fly data: FlyWire connectome v783, used under CC BY-NC 4.0.
Dorkenwald et al., "Neuronal wiring diagram of an adult brain," Nature (2024).
Schlegel et al., "Whole-brain annotation and multi-connectome cell typing of Drosophila," Nature (2024).
Eckstein et al., "Neurotransmitter classification from electron microscopy images at synaptic sites in Drosophila melanogaster," Cell (2024).
Zheng et al., "A complete electron microscopy volume of the brain of adult Drosophila melanogaster," Cell (2018).