Backpropagation is the algorithm that lets multi-layer neural networks learn efficiently: a network's guess is compared to the ground truth, and that error is propagated backward through the layers to adjust the connection weights so the next guess is better. The hosts credit Geoffrey Hinton's paper on the method as the last major event before he left for the University of Toronto.
- 01
The hosts note that DeepSeek's training approach modifies this backpropagation concept in a somewhat different way, and point listeners to their earlier DeepSeek episode for more detail.
171 words · auto-generated from the episode video
1:22:17publishes this foundational paper on back propagation which is the mathematical engine that allows multi-layer neural networks to learn efficiently by taking the guesses and the difference between the ground truth and the guess that you have back propagating that out changing the weights of the neural network so that at the next iteration you're better at your guess. This is the last time that we're going to see Jeffrey Hinton here because um he got offered a job at the University of Toronto after this. The the Canadians saw potential and they're like, "Let's have you go on scoop him." >> No longer ours. I believe if I recall correctly, our episode where we did a deep dive on the deepseek model touched on some of these concepts that the
1:22:57deepseek model kind of did in a slightly different way. But if you're interested in kind of understanding more about this concept of back propagation, check out our deepseek episode. 1987, the birth of phentochemistry. Yes, this is Ahmed Seal