Biological neuron 101 — dendrites, spikes, and action potentials

2:39their wetwware computer could play Doom and it was all hype. Right. I've seen it literally everywhere. Yeah, >> the tech bros are going crazy. >> Yeah, all the doomsday people are like, "Should we be doing this?" Right? >> So, you know, I naturally asked, "How much of this is real and how much of this is hype?" >> And turns out there's a lot of both. >> Okay. >> And that's what we're going to get into. >> Okay. >> Okay. So, let's do a little bit of brief background. Let's talk about the biological neuron, the one that is the substrate of our own nervous system in our human brain. A biological neuron is a cell, a biological cell. We see that on the left

3:21here. It's got a cell body. It's got dendrites which are where other neurons come in and give you input. And in the cell body, what you do is you integrate, meaning you sum up all the inputs. And if all of the incoming inputs from other neurons exceeds some kind of threshold, then you fire what's called an action potential, which is, in other words, a spike. It's effectively saying my neuron went from off to on because all of the other neurons that were talking to me, there were enough there were enough neurons that were talking to me with enough input that I decided to go from off to on. When I do that, I go downstream and the action potential goes

4:03downstream to other dendrites of other neurons and that's how we get a biological neuron. Now, in the artificial neuron, very much similar. You've got you've got incoming neurons and their activity. The GPU of Nvidia sums that up and then fires if it it fires like, you know, an output to downstream neurons. if that output ex if the input exceeds a certain threshold. Right? So we've basically taken the biological thing and we've turned it into a very simple math equation. Now obviously >> in that approximation we're not doing it complete justice because in biological

4:44neurons there's structure, there's proteins, there's all sorts of little things and it's not really just straight up integrating fire though that's the simplest kind of model that you can create. But it turns out for a lot of the AI that we see, I mean, it's doing pretty well, right? >> So even though we sort of have a simplification of this representation of biological neurons in our artificial neural network systems, >> even that simple approximation in and of itself has uh valuable uh productive outcomes. Yes. >> We don't have to replicate the whole of the brain. Yes. in order to get some

5:25benefits from the concept. >> Exactly. And the reason why is because of the neural network. The idea that we can connect a bunch of these neurons into a giant network where everything is connected to one another. Not everything is connected, but you know, you can have layers of connections from one to the other. So in the biological case again on the left, you've got one neuron that's connected to the other. So the output of one neuron is connected to the dendrites of the other through something called a syninnapse. That's the gap in between neurons that let the neurons talk to one another. And in the artificial case, you've got hidden layers of neurons. And from one layer to the next, you've got inputs going in. Again, each neuron is going to integrate, meaning sum up and then fire

6:06if it exceeds some threshold. So you've got these nonlinearities and then that's going to create the output, which is going to create the next output, which is going to create the next output. And if you have trillions of parameters and you know millions and billions of neurons then you can actually approximate any function you want including let's say thought and that's how you get large language models and