The discussion explains how a seizure-detection model already pretrained on thousands of hours of general epilepsy EEG data is fine-tuned to an individual patient using a technique called distillation. The method pushes the model to fit that patient's own seizure patterns while penalizing large deviations from what it already learned on the broader dataset, since only a few hours of patient-specific data exist. Fine-tuning in this closed loop between patient feedback and training is shown to steadily improve performance, measured by area under the precision-recall curve and F1 score, while reducing false alarms per hour. The hosts frame this as an early, narrow step toward continual learning in clinical settings, noting that the underlying models are still pretrained and not truly learning from new information on the fly.
- 01
The seizure detector and the chatbot component (described as GPT-4o-mini) are both pretrained models being scaffolded together rather than trained from scratch.
- 02
One host explicitly caveats that this is not literally continual learning, calling it only the 'foothills' of that capability because pretrained architectures cannot natively learn from new information.
565 words · auto-generated from the episode video
1:02:00Here's the idea. You've got a pre-trained model, right? Let's say let's say you have a bunch of data on epilepsy and you've got EEG recordings and you know when the epileptic seizures happened. So you've got this model that is pre-trained on thousands of hours of epileptic data. And now you want to sort of fine-tune it to this specific patient, right? because this specific patient's brain is different and the way that they experience seizures is different and so on and so forth. You can use this technique called distillation. That's going to do two things. One, it's going to try to get it right for that specific patient, but at the same time, it's going to penalize changing up what you already know,
1:02:42>> right? There's there's a twofold thing that you want to do. You want to you want to gear towards what the patient specifically is experiencing, but you don't want to forget all of the >> Yeah. >> stuff that you've learned from general epilepsy data, right? And that's this idea of model distillation. It's a penalty for changing too much on the basis of limited evidence because you have all this hours of evidence and now you're getting what maybe a few hours of evidence on this patient. You can't you can't totally upend your world model >> 100%. >> Based on everything that you've sort of realized beforehand, right? And now the the the question is are do the models
1:03:24actually do better? In fact, they do. The models do better from this closed loop arrangement between the patient and the training. Right on the left hand side what we're seeing is um from baseline the that's called the the area under the PRC curve the area under the precision recall curve. It's a it's effectively like how well is the machine learning model doing in terms of precision and recall. You can see that the baseline model is highlighted in green and the finetuning steadily increases. Mhm. >> The F1 score again another metric for how well um precision and recall is
1:04:06working is increasing. The false alarm per hour is decreasing. Y >> and that's crucially very important. >> We are on the foothills of continual learning in clinical application and I I I continual learning is a very specific thing. >> I know that this is not literally continual learning but but it I said that's why I said foothills >> because part of the challenge with all of these model architectures is they're all pre-trained. >> Yeah. >> And so they can't learn from new information is the argument that's said a lot. >> Exly. And and even in this case, right, they they're scaffolding off of pre-trained models. The the LLM is the
1:04:47chat GPT4 mini. >> That's a pre-trained model. The the seizure detector is a pre-trained model. But especially because of the complexity of biological variation and individualized medicine, you are going to kind of necessarily need this foothills of continual learning capability, which is can we allow this distillation to happen on a per patient basis in a way that takes the benefit of all of the massive amounts of data but can correctly weight >> how much to take the influence of the individual patient. in future decisionm.
1:05:27It's a very narrow thread to to to pull, but
Animal BehaviorArtificial IntelligenceNeuroscienceProtein Engineering