Cortical Labs’ defense — ablations, frozen encoders, and what’s still unclear

1:03:15I get only a 7 to 8% breakthrough. So the neurons are doing something. >> Yes. Again, there's not a lot of trials. They're just spitting out these numbers. It's not, you know, I don't I don't see data. I'm just like taking their word for it that they wrote in a blog post that it's 27% rather than 8%. So, from baseline, okay, fine. Baseline is 8% 27%. Maybe I'll take your word for it. The next problem that they addressed in their frequently asked questions is that there's a you know, is it the encoder? Mhm. >> Is it is it the thing that is encoding the neuron state and then feeding it into the reinforcement learning thing because that itself is a neural network. So is that doing all the learning? Well,

1:03:57let's see. So if you look at um photo 30, this is their response. They say that >> you know well >> the brain cells in the loop they're effectively saying that assuming the BL brain cells are static. This is not correct. The policy in the cells are dynamical systems. So the biological neurons have an internal state and that is affecting the reinforcement learning. Now to me all that's saying is like well we actually made it harder for the reinforcement learning agent. >> Yeah. >> That you know it's like okay the neurons are kind of like a noisy controller that I'm like >> using to to to create action to my game. And maybe you know to me it's like okay well that just means my reinforcement

1:04:38learning agent is like >> okay with a noisy controller. The last sentence is what is um key. During testing, encoder weights were frozen and still observed improvements in reward. Which means that I >> I froze the way that I am reading my neurons and still there was improvement, which means that the neurons were maybe doing some type of learning. >> Y >> you know, but there's still the encoder in between even if it's in a frozen state. And so how do you know that it's not attributed to the some base fun some base capability in the encoder even if it's frozen? >> Yes. So that's where I'm >> that's my naive >> that's exactly right. Right. You know it's like in the YouTube video they talk

1:05:18about oh it's like >> you know if the neurons fire a certain way the game is going to turn left. If the neurons fire a certain way the game will move for the character in the game will move forward. And if they want to fire the neurons will fire a certain way. Well, really, not really. If the neurons fire one way or the other, it might just be that the reinforcement learning agent is just like also interpreting the state of the game and just like overwriting everything. So, I don't think it's a very clean test is what I'll be honest about, right? And perhaps they're working on a >> paper that is going to elucidate some of these holes. I certainly hope so, because I do want, right, >> a petri dish game,

1:05:59>> right? I sorry I do want a Petriish brain to play Doom. I think that would be hella cool. >> But I'm I'm I think I think the jury is still out because one, all of this is anecdotal. Yes. >> So there's no hard data. They're not showing actual, you know, box plots about the spread and so on and so forth. There was another thing that I read just briefly that was saying that the the the seed in my neural network actually affects how the gameplay is done, right? random seed that initializes weights and things like that. It's like, well, if it's that fragile, can you really say something about whether the >> neurons are doing anything? So, I don't know, you know, but that's my skepticism. >> And but and this is not to take away

1:06:40from the foundational uh uh discovery that Dishbrain in the Pong context, which is why we took the time to talk about how we got to the Doom version, >> which is like there is a there there. >> Yes. But the way in which they're trying to scale it up for the increased complexity of the tasks they're trying to get the wet wear to do. Yeah. It is not clear that it is purely from improvements at the >> uh >> artificial human pur potent uh stem cell level that's doing the learning. >> It could just be the infrastructure that has now been built around it. Exactly. >> That makes it kind of like an interesting thing. Exactly. But not

1:07:21brains on a chip are doing all of the heavy lifting. >> Exactly. And that's what all of the