In-memory computing and AI

1:14:10we'd like to do better. We'd like to not load the weights every single time in order to do this computation. >> And this is part of the reason why, for example, the cloud versus local with AI models and stuff. It's why if you want to do it locally, you need a very powerful machine because it has to do this. >> Yeah. It has to store all of those matrices and it's got to do all of that computation. That's why GPUs are so important because GPUs are just insanely good at matrix multiplication >> at that specific >> at that very specific task. Okay, they're not good at like doing normal emails. But it turns out Nvidia made the big

1:14:51gamble way back in the day that actually graphics, which is matrix multiplication, is going to have a use for later. And it turns out, yeah, >> in things other than video games, for those of who are gamers, we've been very familiar with GPUs for a while, and now everyone is making custom PC builds unnecessarily expensive. >> Yeah. Yeah. Exactly. So, instead of doing all this vonoman nonsense in instead we can do something called in-memory computing. In-memory computing meaning I don't have to move anything. Okay? we can use something like a resistor to do the computation without changing

1:15:31anything. So again, let's talk about how that would work. I told you about how matrix multiplication works, right? Like matrices, you multiply it by a vector and then you get a new vector. Well, now what if we had the old vector come in like wires? Okay, and this is what we see at the on the on the right. And let me let me see if I can say this correctly. Okay, so I've got a matrix and I've got an old vector and I want to make a new vector. And the way that I do that is the rows get multiplied to each of them and then I get added up, right? I can use the rules of electronics V= IR and the summing of voltages and the summing of currents to then just do that using physics. Here's what I do. My old

1:16:14vector is represented by the old by currents that are going at the very top. The old voltages, I should say. They're they're each of the wires at the very top is held at a certain voltage. Right? I've got resistors that are attached to these voltages. And then I've got a new set of wires at the bottom. Because of electronics, the new current, it's a new set of currents at the bottom. Because of electronics, right, the new currents are going to be related to the voltage at the top and the resistance in

1:16:54between. And all I have to do is add up the currents. >> I'm so mad. >> Right? >> In this case, it's not really resistance. It's one over resistance. It's the it's the conductance. And the and then the current is just voltage multiplied by conductance. But it's the same thing. It's just V equ= Iir the other way around. And all we're doing is using the physics in place, right? We don't have to move weights around. As long as the weights are the same, I can use the physics to be like voltage multiplied by conductance gives me a current. Voltage multiplied by another conductance get gives me another current. And so my new vector is going to be the sum of all of those currents. And there you go. Matrix multiplication

1:17:34using physics. So I'm just I'm so mad. That's so that's that's so good. And the part of the point here is then then you can get around the BOMAN bottleneck as a result of that because you're not having to transition between your compute layer and your memory layer. >> Yeah. And all you have to do is the weights have to be programmed into the conductance. The input is the voltage and the resulting multiplication comes out as current. I'm just using Ohm's law and Kirkoff's junction rules. The stuff that you learn in like intro physics right now. Are we capable of doing something like this? Well, this particular paper itself made a 32x32 analog resistance state and it works at 1300° F. It works at low voltage. 0.5

1:18:18volts is all you need. um 30 nanocond switching in the speed and it definitively it it it proves that you can be fully capable of storing complex multi-level analog neural weights for whatever AI interface that you're working with. It can do it in extreme environments 1300° Fahrenheit and the standard digital GPU which relies on all these transistors and the cos and all that other kind of stuff the the traditional GPU is going to get fried. But if I use this memory where the the conductances, the weights are stored using this memory, now all of a sudden I've got something that can actually work. Golly, it's so it's so

1:19:02good. And it it's this interesting combination of both things which is the um fundamental discovery around the