Why this AI model is interpretable rather than black-box

38:07of the day, how the forger creates the fake stuff is it starts with some kind of random noise, which is what you see there, and then and then it starts manipulating that noise to create the real thing, right? So, what you're seeing there is is act of the forger in some sense, going from noise to the real thing. But, the training part is something that's already been done by Midjourney on the back end, right? It that that's the generative part. Okay, great. So, now um Great, that's how you make a fake Picasso and a real Picasso and things like that. How are we going to use that to figure out coma? Yes. All right? This is This is where Dan comes in hot

38:48with an interpretable biophysically grounded mean field model. Okay. And that is what is cool. So, on the upper left, that's A, figure A, Yes. >> okay? On the left-hand side, we've got our DCNN, the convolutional neural network, that's our detector that we had trained previously Yes. >> in figure one, right? There's other detectors in here that that'll actually like um figure out is this real or fake data? That one is saying is this unconscious or conscious data? There's other ones that are not in this back end, you have to go to the supplement to see it, Um where it's like is this real data or fake data? Cuz first we want to just create real-looking data. Then we'll worry about, okay, am I creating conscious data or unconscious data, right? Yes. So, there's multiple steps

39:29to this process, but the key is that the forger is not a black box, >> Mhm. okay? The forger is actually a three component which within which there's multiple components, but you know that meso circuit hypothesis of like there's a cortex, there's the basal ganglia, there's the thalamus, and these guys are talking to one another. >> Yes. Each of those boxes is its own little neural network, right? And it's a biophysically realistic neural network in the sense that it's not just a bunch of artificial neurons that start at noise. What it's doing is it's saying, okay, how does a biophysical neural network work? Well, some input comes in, then there's going to be like channels, like like, you

40:11know, some neurotransmitter is going to have some characteristic time scale, other neurotransmitters are going to have other characteristic time scales. There's going to be excitatory neurons which go do positive feedback. There's going to be negative negative inhibitory neurons that do negative feedback. These I can hook up mathematically using differential equations. And I can create a biophysically realistic model. It is no longer a black box. And instead of using back propagation to change the weights between each neuron, what I'm going to do is use something called a genetic algorithm to to change the parameters of my biophysically realistic model.

40:51Meaning, how many excitatory neurons are there? How many inhibitory neurons are there? How is the connectivity? Right? What is the time scale of the interaction? Things like that. Things that are actually relevant when we think about what is a brain doing. And is in some sense the reason why it may It's like that algorithm is necessarily bounded by what the realistic value ranges are. >> Exactly. >> And that's why it's not a black box because you know that oh it's either going to be an excitatory neuron or they're going to go between X and Y. And that's a realistic normalized range. >> Exactly. That's one of them. And the other one is we can literally point to components of the model and be like oh it's the excitation in the thalamus.

41:32>> Ah yeah yeah. Like there's Yeah. >> Right? I can literally be like oh it's this part. >> Yeah. Where again it's like with with normal neural networks I don't have no idea. >> Yeah. Yeah. Like you know I I type in make me a cat on a horse. I have no idea in the trillions of parameters where it decided cat >> Yes. where it decided horse. How it figured out to put the cat on top of the horse. >> Yes. Here literally I could just be I could look into the parameters. I can see how they evolve with training and figure out what is actually going on. That's very good. >> that extrapolation from there to the brain. It's very good. That's very good I think. >> very good. >> Right? >> Yeah. And and that's one of the things I like a lot. right? This loss function,

42:14he had a loss function that sort of like figures out, right? How to how to change the parameters. And this loss function incorporates the input outputs from the networks that trained to that's trained to classify real versus synthetic data. It also has outputs from the consciousness detector that we had done earlier. It's also got a way to identify seizures. Um, and as you said, it's got these empirical constraints on the firing rates of these neurons. As you said, like, right? Right. Excitation can't be that high. Inhibitory neurons are usually higher firing rate than excitation. Things like that, right? And the interregional communication patterns. >> Right. >> Like excitation is long range. Inhibition is usually short range. This is stuff you can bake into the model.

42:54This is quite nice. Right? >> This is this is quite nice. >> the fact that it's not black box, cuz that's one of the things that I hate about >> Right. like just large neural networks is just I have no idea where anything is happening. Yeah. And it's also so different from the culture of software since the it's beginning, which has been that everything is like explicit. And you can literally point to where exactly like the the trace callback of where something is coming from. >> is the going back to that type, you know? >> which is great. Which is like like like this is like yeah, fantastic. Yeah? And so once this genetic algorithm has trained my um, you know, brain maker, it's an artificial brain, now we can ask, "Okay, um, how about we reprogram the objective

43:34now?" Okay. >> It was just making real world data. Yeah. Yeah. Now I reprogram the objective to make comatose data. So I can use that first neural network to be like, "Give me coma Right. data." >> Yes. Right? And then now that genetic algorithm is going to tweak the parameters such that I get coma type data. >> Yes. And and just I just want to re-bring this back in because we have done the training and done the validation based off of a ground truth real source of actual information. >> Yeah, we have all of those 10-second long clips. >> Right. It it that that's it like I just the it's not making this from whole cloth. Like I just that's like a really important concept. >> and there's multiple stages of training.

44:15It's like it's he's he's figured out a way to, you know, you can't just tell a biophysical model make me coma. Right. You got to first be like, "No, no, no, let's make just normal data. Give me data that can fool even me to thinking, I don't know if that's from a real patient or from my brain simulation." Right? I'm sure he did like checks just on his own where like after the whole thing, I mean, you look at he's like, "Okay, that looks pretty good." Right? And then you go,