What Happens When AI Gets a Lab?

Playdates, brain data and robot labs

0:00Orangutang mothers arrange playdates for their infants in order to invest in their social development. You're you're turning the patient into a collaborator for training the neural network for training the machine learning model. This is so good. They have a robotic arm that in 50 steps in 30 minutes can set up the these lenses for this optical experiment which we've talked about previously. Hello internet. This is your

Hello Internet and Signal Award celebration

0:31captain speaking Lester Nar joined as always by my co-host and our resident PhD Christian Chowdery. We are back for a fun rundown episode this week with some special news about our recent award-winning show as the recipients of a bronze signal award which we just got the news of this morning. And so a big shout out to all of you in FFP Nation for watching the show, for giving us the inspiration. Uh quick question for our resident PhD. How does it feel to be recognized for the fifth annual signal

1:11award in best podcast for science? >> Um it it feels great. Um in the sixth annual signal award I would like to get gold. >> Yes. >> So yes, >> that's what I'm feeling. I feel I feel gold is on the mind and we are going to get into three science stories today for a rundown. For those of you who joined us during the Nobel Prizes, welcome. Rundowns are where we do a little bit more of a high-level overview of some science stories that we have found interesting. Starting with an an animal behavior story, the parenting techniques and sacrifices of orangutang mothers, followed by personalized medicine with AI using a chatbot to make epilepsy

1:53treatment more effective. And we're going to end with automated scientific discovery. The AISARS are telling us they're going to solve all human disease in 10 years. But what does that really mean? And there's an interesting story that actually came out earlier today that's going to help us look at the larger ecosystem of what is going to be required to have AI really impact clinical outcomes for humanity. As always, we're going to talk about the science from the ground up today because this is from first principles.

Orangutan mothers and infant playdates

2:40The headline here is irresistible. Orangutang mothers arrange playdates for their infants in order to invest in their social development. >> I mean, as soon as I saw this, I was like, we got to cover it. Okay. Because at first, I mean, it intrigues me in so many ways. As a new father, you know, I'm starting to think about the pros and cons of like putting my kid in daycare and like the social development that comes with it, but also the exposure to like, you know, just getting sick and like all sorts of like nonsense that comes with that. So, you know, um there's a lot hiding in that headline

3:21and when I was diving into the paper, there's a lot of rich science behind the methods and things like that that I thought would be perfect for this podcast. Um there's a fact that this seems to be a deliberate decision by the orangutang mothers. Like they're not just like casually meeting up with other mothers that have babies. this is a deliberate decision. Um there's a fact that they have to give up energy and time to do this. So there's a story where you know mothers are making sacrifices. It's so similar to how humans do things. Um and and the science underneath that headline is really

4:02interesting. Like the way they collected years of data, analyzed that data and then they can convincingly state that you know orangutang mothers are consciously or not consciously but deliberately trying to do this. I think it's really really cool. So >> careful with the consciousness word with all the AI stuff going on. Right. So so it's up for debate right now. >> Exactly. I mean, if you ask me personally, my personal opinion is that of course orangutang are conscious. Like just >> look at them and look at the way that they respond to their environment. They respond to each other. They respond to stress. I mean, it's it's obvious to me, but there's going to be people that like, you know, saying, "Yeah, you don't

4:43really know what consciousness is." Okay. Anyways, let's let's get into some history and background. Orangutans, they're the great apes. They're part of the great apes. That includes gorillas, chimpanzees, bonobos, and us. >> Mhm. >> The animals um that are in this study are particularly the Bornean orangutanks, specifically Pongo, Pyus, Wurumbai, another taxonomical name that I'm struggling with. But these are the guys that live in Tuanon in central Kalimantan in the Indonesian Borneo. So Borneo is a giant island and there's Indonesia has like a majority of that. And >> I'll just knowative uh for all of my Nickelodeon fans, the Wild Thorn Berries

5:24is the first time I ever heard of the location of Borneo because it's this >> family that traveled around as like wildlife videographers. Uh like that was kind of the it was like I think it's literally in the theme song uh that and and so it's it was really cool cuz they traveled the world, but that was the first time I understood where the location >> Well, they must have been looking for orangutans in Borneo because Borneo is very famous for being like sort of a native. So they the family had a a a a an ape Darwin that could speak to Eliza because Eliza could speak to >> animals. That's Oh, I see. >> That's the shtick of the show. >> And they had a you know Darwin was around >> as a good name for an ape. >> Yeah. >> And clearly was done with intent.

6:06>> Yeah. Yeah. Yeah. >> So orangutang are a bit different from other apes. Okay. Because they are not that social. Um they are solitary in the sense that adult females they occupy home regions that maybe overlap with neighbors but in a real sense like they're very solitary. Solitary here means that um the usual organization of travel and feeding like they don't they don't go in groups they don't assemble in packs you know um >> so like Americans very individualistic >> very individualistic in modern modern Americans very much so right um modern society very much so in some sense and

6:46um maybe that tells us something that we can gather from the study but we'll touch on that later on right um the idea is that rather than traveling every single day in a stable troop, they actually move as separate units. So, the mothers will be separate and if a mother has an offspring, they're going to move as separate units. Okay? They're going to have some kind of home range of like some section of the forest that they like explore, but those are going to be mostly disjoint. Now, there are good ecological reasons for that. It actually has to do with um the diet of these orangutans and like where they where they live. The diet is mostly like fruit, large fruit eaters.

7:29Now fruit is something that is kind of rare in the forest and it's sporadic. It's not like you know at a regular cadence you can be like that tree is definitely going to have fruit and then this tree is definitely going to have fruit. So group living has its advantages right because you can like share information, you can share protection. But the disadvantages are that there's a limited amount of food and you're going to be competing for that limited amount of food, right? In a space where this density of food is very low. >> So that's why orangutans I mean from a biohysical perspective there's a lot of um theories out there for why they are solitary. It's actually has to do with

8:11the the physics of the environment itself. There's a resource constraint that they are conscious not consciously but actively >> uh operating around. >> Exactly. Um this is very different from other great apes. Chimpanzees very famously they are in troops right there's a lot of resource abundance in the place that chimpanzees live right. So they can they can actually administer in that. So now in this space of solitary confinement of these orangutang individuals the adults who are solitary in the forest now let's consider the infant. Okay orangutangs usually have one baby at a time and um

8:56usually that baby happens once every 7 to 8 years. >> Oh wow. >> So they don't have siblings. >> Yeah. >> In the traditional sense, right? It's not like a a litter of kittens or puppies where like they they have a bunch of people their same age that they can play around with. These are single babies to a single to to a single mother effectively cuz the dad >> went to go get the milk. >> Yeah. Yeah. Yeah. Let's you know the dad is not involved. Let's say let's say that. Um so so these this is a single mom with a single baby no siblings and you've got to introduce some bit of social >> you know

9:36>> interaction >> interaction yes you have to introduce some bit of that you don't have group living so what do you do that's the challenge right >> that's the challenge um and that creates a developmental problem inside of the ecological problem >> that's interesting yeah there's there's layers Right. The ecological problem is well because of the forest that I'm living in and because of the constraints that I have depending on the fruit and all of the choices that I have for food, I need to stay independent. But if I have a kid, I need the kid to socialize in order to like actually grow up and be sort of functionally accurate, right?

10:17Like play is a very very important part of upbringing. So play is actually very important. It's been studied for a very long time. Play is difficult science-wise because it's easy to recognize, but it's hard to like quantify. >> Yeah. >> Like this is a a a photo of two bear cubs playing. Now, here they're probably siblings, which is why they're close to one another, right? And they're playing amongst one another. I don't think bear cubs really mess with other bear cubs because mama bears are very, very cautious of other bears when they are nursing and when they're bringing up their kids, right? So, so bears have

10:58this advantage of having siblings. Orangutang do not. Now, in orangutang research, social play means that you have non-aggressive interactions such as tagging, wrestling, chasing. It's the stuff that like humans are are pretty are pretty used to. And there's a long history behind why play evolved. I mean, one idea is that it rehearses adult skills. Mhm. >> So even if the adult is not meant to be in a social setting, play can still be important. If the adult is meant to be in a social setting, then of course play is important, right? Because you're getting used to socializing. But you could have the argument that like, well, orangutans, they're they're asocial. Why

11:41do you need to play? Well, play gives you a kind of emotional regulation and it gives you the ability to respond to something that is unexpected, right? >> Right. If you're doing selfplay, then it's kind of a self feedback. >> You know what's coming. >> You know what's coming. When you're when you're playing with someone else, you don't know what the other person is going to do. And that ability to respond to an environment or to a stimulus that is unpredictable, that can have a lasting effect. It's beneficial to your own survival ultimately because you want to be able to react to unknown stimuli in a way that is beneficial to your own survival but that is not also an overreaction. >> Yes. >> Unnecessarily.

12:22>> Exactly. Yeah. So, so, so there is there is a benefit to play even if the population itself is asocial. That was the first thing that when I was reading the paper I was like that's actually interesting to think about, right? because it's trivial that play is important for social animals because you need to be social but even for asocial animals that's an interesting >> there is a there is a component of play that is important for your development >> um any case do orangutang youngsters play well in 2024 Julia Kun who is a co-author on the current study that we're going to be talking about in depth she published this article that um showed 11,000

13:02um observation hours from 99 young orangutans at two sites, one in Borneo and one in Somatra. And she showed that social play was more frequent in the Somatan site versus the Borneo site. But actually, if you controlled for >> the amount of times that >> um play was presented, >> ah >> it was the same. >> So if they're given the opportunity to play, youngster or rangutang will play. So now let's try to hone in on the other part which is how many times are they given the opportunity to play? >> That's what this paper is about >> because there's a difference. >> Right? It's like it's like if I'm not given the opportunity then I can't do

13:42it. Right? And the new paper builds on that groundwork. This is the paper that came out in the proceedings of the National Academy of Sciences in October 2026. >> I see you've avoided the acronym >> PEAS. That's what it is. That's what everyone says. Okay. Everyone says stop saying the the acronym and that's >> no no that's a joke everyone says it um this is from Maxplank Institute of Animal Behavior and there's a few authors from the University of Zurich so the earlier work established that orangutang youngsters play >> and that the opportunities for that play varies right and here what we're saying is that the maternal association could

14:25provide for that opportunity and the moms actually make a concerted effort in providing that opportunity. Okay, that's what's so cool about this. Um, the methods are incredible, I would I would say. Okay, >> so since 2003, Van Nordwick and her team have been following mother infant pairs at Tuan Research Station on the island of Borneo since 2003. Over a decade of research, they've observed encounters between mothers that are rare, but when they do happen, the mothers are very cautious, but the kids, as soon as they

15:06see each other, >> they just like run at each other and start blast. Yeah, they're having a blast. The mothers are very cautious. They've, you know, developed the whole adult thing of being solitary. The kids,

What 15 years of orangutan observations reveal

15:18>> they're having a great time. And one of the things that the first author Van Nordwick noticed is that these infant encounters, >> they seem to come at a cost to the mothers. >> Oh, okay. >> Okay. The cost being it's a really long and hard day for them >> when they have to arrange these playdates. >> Are they even arranging these playdates? And if they arrange these playdates, >> are they doing it at an expense to their own benefit? Right? Cuz they could be doing other things. >> Resource gathering. >> Yeah. They could be eating. They could be resting. Instead, they're doing this, right? So, is it a concerted effort? After 15 years, they've amassed 32,000 hours of observation from 31 mother

16:00infant pairs. That's actually incredible. First of all, right? >> Like 15 years of work, 31,000 hours. >> That's like on the order of like a year a year of just like continuous observing orangutans. It's incredible. The basic technique is something called a focal follow. This is a very famous photo of Jane Goodall >> doing a focal follow on chimpanzees. >> The idea is that you have researchers that choose an identified individual and then you basically follow its behavior from sundown to sunset or from sunrise to sunset. >> Um you monitor every 2 minutes

16:41where the person is. So you have like a GPS monitor and you're like, "Okay, I'm going to monitor where the orangutang is right now. I'm going to monitor what the animal is doing at that moment. So is the mother feeding? Is the youngster playing with whom? And roughly every 30 minutes you log a GPS location, right? So you you get a kind of GPS of of where this person where where this animal is going and you get uh every two minutes some kind of log of the activity. Mhm. >> over the entire day. >> That's a quite that's a a rich data set. >> Yeah. Incredible, right? And and it requires a human being just straight up following this orangutang or this mother

17:24infant pair for the entire day. >> Mhm. >> Pretty incredible. >> For days and days and days and it's the same individual that you have to follow for days and days because you want to have some type of longitudinal data on the same individual. >> Yeah. >> Now, the data set here includes 3,000 plus follow days, 2,000 of which were full nest to nest days, meaning like like there's like mothers and all. Um, only 6.3% So only 6% >> did they record play. >> That's that's incredible. What we're saying is over the course of all this observation, a less than 10% proportion of it has this peer play concept, has

18:04the the thing we're looking for. And that's why you need such a large data set because it's so infrequent. >> Yeah. And you need a large data set because if you have infrequent events >> in order to do good statistics, you need a large data set, right? >> Reminds me of the ice cube nutrino detector. >> 100%. It's the same thing. It's the same exact paradigm, right? Cuz you need a lot of events in order to hone down on a very rare like per chance. >> Mhm. >> So now let's get into the figures. They've collected all this data. What do they actually see? Let's start actually with figure C here. This is figure 1. In C on the x-axis, you have the age

18:45difference between two offspring that are meeting up for a play date. And meeting up meaning, you know, they're playing with one another. This is obvious from a human that's like looking up and seeing them. >> Um, on the Y ais is their play rate, which is, you know, some normalized sense of how often do these two individuals get together. >> Wait. Oh, so we're looking at C. >> Okay. Yes. >> Right. We're looking at this this part right here. >> And there you can see there's a negative correlation meaning >> that as the age difference gets bigger, the play rate diminishes. >> Yeah, that makes sense. >> Kind of makes sense, right? >> My little brother and I have a 13-year age gap. >> Yeah. >> So, our play rate was much lower than siblings who were much closer together.

19:26>> Exactly. The other cool thing is on the y-axis, it's actually exponential. That's a log scale cuz it goes like 10 -1, 10 - 2, 10us 3. So this is exponential decay. It's a very strong effect that you're seeing which is like it seems that like the the closer you are to age the longer you're going to play like right okay on A and B what we're actually seeing is the effect size based on different markers that the the human beings that are on the ground are actually like marking. For example, you can track the relatedness of two individuals and you can track the fruit availability in the in the forest

20:08because you could imagine, right, fruit availability is a compound. If there's a lot of fruit, then of course the mothers are going to be foraging a lot and so they're going to meet up a lot, >> right? You want to disentangle those and see what's the actual effect. Well, actually fruit availability is not that is not that high. In effect, the correlation is not that big. On the other hand, >> related relatedness, like whether you're related to the other kid, >> that's high for occurrence. But if you control for occurrence, the amount of play that you do per occurrence is not that high. >> That if you if if you're family, I'm not about that life.

20:48>> Yeah. Like like we'll meet up a lot. >> Yeah. Yeah. But >> but I would I would play with you just as much as I would play with a friend. >> You see what I'm saying? >> Yeah. Yep. Yep. >> Like there's a difference here. Like maybe the occurrence is higher with family because like the moms know each other. >> But just because you're family doesn't mean I play with you more. >> Yeah. Yeah. I play with you the same amount that I would with like a stranger's kid, right? Um the two big anti-correlations are again age difference. That's the very top. Mhm. >> It's like if the age difference is very high, then the amount of play rate is very low and the occurrence is very low. So, it seems to be that the mothers have honed in on who has the offspring that

21:28is about the same age as my kid. >> Fascinating. >> Isn't that crazy? >> That's so good. >> Like, I mean, they're keeping track. >> Yeah. Yeah. Yeah. >> Of like of like who has the the right age group >> of kid. Now, the the other cool thing, the ranges overlap for moms because I told you, right? The the the moms have a um they're solitary, which means that the range of forest that they forage might not overlap all that much, but overlap is higher for moms that have similar age group, kids. >> Oh, that's really good. >> Right. >> That's really good. >> Because you have this data over 15

22:08years, you've been collecting GPS data. Yeah, >> you can actually tell where the moms forage and the overlap is higher. It's stronger for two moms that have similar aged kids, >> which is pointing to this idea of some intentionality of going to that place of overlap because I know the p the other mom that comes around here, >> their little their little Bobby and my little Timmy are about the same age. >> About the same age, right? So, they can they can like play around. This this entire paper is going into that intentionality, right? Can we prove intentionality in orangutang mothers? That's the point. >> And the way they've structured these questions and the data points they're

22:50tracking in order to reach this this >> the conclusions about intentionality >> are very clever in the areas that they're trying to cover to remove other confounding variables. >> Exactly. Yeah. And that's it. I've always find this fascinating because like especially with anal behavior, it's very hard to think of all the compounds, but these guys are doing a pretty good job. I think >> that's quite nice. >> Um the next figure is figure two. Here we talk about um what it costs for the mothers, right? Um here the researchers are comparing three different categories. There's days when the mother was alone with her offspring. There's days when another orangutang is nearby,

23:31but there's no play. And then there's days when there is infant play. Okay, those are the three sort of categories. Um the middle category is really important because here we're simply disentangling between like okay if another orangutang is nearby does that have an effect versus another orangutang is nearby and my infant is playing right there's there's two different things that are happening and here we're seeing an actual difference right on the left hand side this is the daily amount of path that the mother takes when the kid is playing the path length that the other traversed in the forest is a lot longer. >> Yeah.

24:11>> Almost 20% longer. >> Yeah. >> Okay. Which means that she's she's exerting more effort to actually go from one place to another. >> On the right hand side, figure 2C, that is the proportion of time feeding. >> This is what's most important to me and most salient to me. The amount of time that the mother spent feeding herself was lower on days when her infant played with another friend. There there's there's more caloric deficit from the amount of movement >> while having a lower amount of caloric intake at that stage when you're having this higher caloric deficit. >> Yeah. Yeah. Yeah. It's it's a twof it's

24:53a twofold >> Yeah. >> effect. >> Yeah. And and you know as human beings part of this is going to feel like be like oh obviously like we talk about child care and >> all these things and you are experiencing this personally right now and and going through that. >> Yes. Okay. With human beings for sure. >> Yeah. >> But we're trying to empirically show this

Testing the playdate hypothesis

25:16in a different species that does not have all of the things that we have. And so we have to ask what may seem like basic questions. >> Yeah. because we don't we don't know unless we have the data, >> right? And and that's that I think is what's key about this paper, right? Is like with human beings, I can just ask mothers and they'll tell me, "Yeah, it's hard, right?" But here, it's it's I think it's really cool that using data from behavior over 15 years, we can empirically show that orangutang mothers are kind of feeling the same stresses that human mothers feel. Right? The last the last figures that I'm going to talk about I think are really really cool. The next step is really asking whether

25:58mother's movements change before and after play. >> Mhm. >> Okay. Like is it intentional? You know when we talk about like intentionality? >> Yeah. >> Well, here >> what we do is first we define each mother's reference core using the average of her GPS location from the preceding year 365 days. So we've got a range, right? cuz we need to be like where was this mother before the before the foraging like what is her control and we're like okay this particular mother um Mindy goes in the green region of the forest and another mother Januk goes in

26:38another region of the forest and there's a little bit of overlap because they have similar aged infants okay we've already covered that part right now let's talk about the day when the the infant play happened. >> Okay, let's look at the day before and the day after. The day before, you can see both mothers actually venturing into each other's zones, >> which means there's planning involved, >> right? >> They're scoping the scene. >> They're scoping. Yeah. They're like They're like, "Let's, you know, let's try let's try to find the other woman who has the kid >> that's the similar age as mine and then let's let's try to meet up." That's the yellow, right? You see you see you see

27:20Mindy like going into Januk's zone. >> And then the day of the play, they actually play around and you see where the play starts and where the play ends. >> Yes. >> And and they're they're moving around in their territory. >> And the day after. >> Incredible. >> The day after Mindy's like, you know what? >> That was enough. >> We're good. >> We're good. We're going to we're going to hang out entirely in my own zone. This is incredibly like it's so visceral and visual like because you can see it visually >> you know and again the pre during and post this is why anytime you do data capture like you want the >> pres of the event you're tracking for >> and it's the distinction between the day

28:01before and the day after is so in your face. >> Yeah. >> Um >> and they have statistics to prove this because this is obviously an anecdotal thing right? >> Sure. One of the statistics is this directional bias. I mean these are the statistics on figure 4E. They're actually showing the difference between like the correlation day before day after. There is a huge difference in the behavior. >> Yes. >> The directional difference is what's really cool. >> Um in the day before >> they're moving towards >> Mhm. >> the other territory. That's the yellow compared to the control on the left. >> Yes. And the day after they're moving away.

28:42>> Yeah. Yeah. Yeah. And it's, >> you know, I I one of the things that as we've gone through so many episodes over the last year almost and a half now is really getting a strong understanding of the questions that are asked, the types of data that are gathered, and the type of analysis that's done on that data. and understanding how those three component parts connect to >> conclusions you want to draw. And with animal behavior especially, we've done a lot of these stories. >> It's a very complex all biology is complicated. >> Uh now we're adding behavior on top of just the biological substrate.

29:25>> Um and when you see something like that, it's it's very >> I don't want to say the word convincing. Um but it is present. Yeah. I mean, to me, it's like it's it's um it's confirming a notion that I already had, which is that like of course these animals are like >> are like >> doing all these things, right? But it's just it's pretty awesome that like with 15 years of data and all of this effort, we can like quantify these things. I think it's I think it's absolutely amazing. This makes me think of the whales have culture uh story we did >> and we were having this very emotive

30:06reaction about it because we were showing something that might feel obvious. >> Yeah. >> If you just like of course whales have >> Yeah. >> culture and things like that, but like how do we know? >> Yeah. I I think that's kind of cool >> and I think it's cool that we can like say this is how we know. >> Yeah. >> Um like if you were to say oh the orangutang mothers are solitary but they >> the the headline like we just looked at. Oh, how do you know that orangutang mothers are setting up playdates for their kids? Well, actually, here's the >> Yeah. >> 1 2 3 4. >> Here's how we removed other elements from it. >> Um, and >> it's the well constructed. The methods are well constructed. >> Yeah. Yeah. I I really quite enjoyed

30:46this. Yeah. The evidence here supports um active maternal facil facilitation. >> Yeah. >> And is consistent with planning as we were as we were talking about just now. Now the broader scientific contribution here concerns paternal investment because in evolutionary biology we think about like investment meaning effort or resources devoted to an offspring that have a cost or trade-off for the parent, right? It doesn't require some kind of conscious calculation. We usually think about it in terms of nursing, caring and protection. Those are the obvious examples. In this case, the paper is making a case for including access to a social environment,

31:28>> right? As something that we need to talk about when we talk about paternal paternal parental investment, right? Um the the fact is that like a mother can shape development by changing which other animals her youngster has a chance to meet. Um it's part of a growing movement. There's been a growing movement in the science of behavioral analysis these days where the biologists are now starting to study solitary mammals and they're saying actually like play and socialization is important even if these solitary mammals

32:08are kind of doing their own thing. like you can't count out social boundaries and social interactions just because you know by tradition these these animals are are solitary. Now there is a second implication >> okay >> which I found quite intriguing. It was in the discussion of in their in their paper. Um as an Indian I found it quite intriguing. Here's why. So group living means that you know young animals they meet potential partners almost automatically. If you live in a group,

32:50you get introduced to um potential partners pretty regularly. If you're like an orangutang and you're solitary and your mom's solitary, so which means that you are solitary because you're tied to your mom, you're not going to meet a lot of the opposite sex on an organic basis, right? Just from Brownie in motion of like the orangutans. So perhaps in a dispersed society um someone has to bear the cost of making these encounters possible, >> right? >> Kind of like an arranged marriage system, >> right? And and and this is in the discussion of their paper. They're

33:30saying look like you know in a solitary society this is an evolutionary advantage. >> Yeah. Right? Because because the moms like, okay, if we if we get real nitty-gritty down to it, right? The moms care about their genes being passed on. This is about evolutionary fitness, Darwinian fitness, so on and so forth, right? That's going to require their kid in a solitary society to like kind of ingratiate themselves with individuals of the similar age, right? What if what if that's the motive? >> Yeah. Yeah. Yeah. Right. Right. Look, look, that's the question. Now we can ask that question >> now. now and and the more data we have we can ask that question and that's

34:11actually what's next right is whether the investment actually ultimately pays off >> in an increase >> in a in an increased like fertility of the offspring >> right we don't have the answer to that question yet but a as an Indian son that like kind of reminded me a little bit about about some of the struggles that I've had >> I this is so it's I love the connection >> you know it's Yes. Yes. I I will make and this is a big spoiler alert for anyone who's not watched the uh new Netflix TV show below. So, spoiler alert. >> Mhm. >> Fast forward like a minute or two. Um,

34:52what's funny about what you just said is in this show uh below, which is about this sort of local town in Newf Finland that gets moved around because they some rich entities want access to resources where they live. >> Um, the main character >> got moved when he was a kid and now he's an adult in an island next to the island he grew up on um in this bay area. And one of the plot lines in the show is that they thought where they were moved from was abandoned, but there was actually a family, a a family and a few other individuals that had stayed there that were solitary. And there was a daughter that the dad was struggling to

35:35find a partner for. And there was this sort of scheme where the main character's son got whisked away to this solitary island to be the potential partner. And this idea of the the parent having this intent around this evolutionary advant like wanting their it was it was and it was a core tension point in the show that had anyway but like it >> I'm just bringing a popular culture reference to something that >> you're talking about here because it's it's literally narrative story lines in like the number two show on Netflix right now. >> Yeah. Yeah. And and that's what the authors are proposing here right in

36:16their discussion. They're saying like as a solitary society the moms are also thinking look I kind of want grandkids >> and even these orangutang moms are thinking the same thing. They're like look I want orangutang grandkids which means that you're going to need to meet people of similar age. It kind of explains the negative correlation between you know age difference and like meetup. >> Very good point. you know, like it kind of explains like the intentionality behind all of this. >> I I think so. So, if if I like the theory, you know, and of course, like there's going to be a lot more data that's going to be involved because what you're going to have to do as

36:57scientists, you're going to have to now follow the offspring. >> Yeah. Right. It's >> and see what their reproductive success is over several decades. Right. >> This is a long longitudinal study. >> Yeah. A long longitudinal study. >> Exactly. But uh but I'm willing to bet that I think that's what's happening. I the the data suggests. Yeah. >> Um very good. Very good. I just want to know because usually in the cont context of below the mother was dead which is why the dad was doing the job. And so that's part of the gender difference there. But like obviously >> here it's like all moms. It's the moms. >> Here it's generally speaking. Generally speaking >> generally speaking look we I we know how it is. Okay. Even in humans it's it's

37:38usually the moms. That's how it goes. >> Uh, fantastic first story. Uh, the animal behavior stories are always so interesting because it's kind of nonobvious for layman to like understand like how would you even go about asking the question and they're so clever at it. >> We are going to transition into our next story which is going to be about AI chatbots and epilepsy. But before we do so, I'm going to do a quick commercial break here for some housekeeping. Welcome for those of you who are joining us. This is from First Principles, the best science show on the planet. I have one very amazing announcement about how you can watch the show. If you're an

38:19audio listener or have been an audio listener, we do a lot of overlays and visuals in our show to kind of talk through the fundamental science, the methods. It really helps you get an understanding of why these stories are so important. And I'm very proud to announce that finally after weeks of promo that the podcast is now available in video on Apple Podcasts. So for those of you who have iPhones or or listen previously on Apple, you can now watch the show. It's also available in video on Spotify and YouTube. On social we have so many clips that are a way that so many of you listen to us. If you've come from an Instagram clip and are now

39:00watching the full podcast, leave a comment below letting us know. You've come from Tik Tok, Facebook, etc. We'd love to know uh that you've made the transition to the full show. It's super helpful. You can always get all of the research papers that we talk about at our website ffpod.com. And we are now going to move into our second story of the rundown. AI, chatbots, epilepsy. How do they all connect?

BrainChat and patient-guided learning

39:30Well, we've covered a lot of AI doom and gloom. >> Yes. Because well, there's a lot of AI doom and gloom. >> Yes. But this one I think is genuinely uplifting because it's the promise of AI that got me excited about it in the first place all those years ago when we got the first chat GPT moment and everyone was like oh this is going to change the world the future you know it's going to be the future um this isn't the apocalypse this is actually great now it seems like ah I don't know um in any case this particular story it's about personalized medicine this time it's for epilepsy patients the story comes out of the University of Pennsylvania. The headline reads, "AI

40:11platform allows real time conversation between patients and their medical devices." I think it's really, really cool. >> This is interesting. >> Now, in this case, when I say personalized medicine, I'm talking about really a personalized machine for medicine. Okay, we've covered things like this in the past. We have pacemakers that regulate your heart. We have colear implants that digitize sound. And we've got deep brain stimulators that suppress the tremors of Parkinson's disease and epilepsy. Now, these are marvels of modern engineering, but they're all entirely silent to the patient that is housing

40:52>> these machines, right? Like if you get a medical implant today, the device records your physiological data and then it stores it in some black box effectively, right? Some kind of USB drive or something. And then months go by, weeks go by, depending on what the treatment is. You go to the clinic, the physician downloads whatever the data is, and then only the physician really understands what's going on, and then he or she or they are going to be going through and understanding, and you're essentially locked out of that loop. >> That's actually really good. I never thought about that. Yeah, it's really cool. >> Like the patient who is creating the medical data >> is out of the loop, right? the physician

41:33kind of just like uses the patient for the data and then tries to understand the patient from the data. But there's no real feedback that's happening, right? Nowhere is this disconnect even more critical than with the treatment of epilepsy. Globally, around 70 million people live with epilepsy. Epilepsy, a third of them get seizures that simply do not respond to medication. A seizure is basically a temporary episode of abnormal excessive activity in the brain. It's like synchronized neural networks. It can sometimes make a person experience convolutions, brief loss of awareness, unusual sensations,

42:15disturbance in memory. There's a myriad of different things that a seizure can do. And a single seizure does not automatically mean epilepsy. But if you're prone to this recurrence over time, then it's it's reason to diagnose with epilepsy. Now, sometimes drugs work, but then there are drugresistant patients. As with almost every brain disease, there are drugresistant patients. In this case, um, one of the ways that we try to combat it is with implantable neurom modulation devices. This is where we stick an electrode into the brain that is that that has the ability to

42:58read the electrical signals and then pump electricity in. So what this thing is going to do is read brain waves and in principle it can detect seizures using the brain wave profile and then it can also provide current into the brain to mess with the neural circuits and then perhaps stop the seizure. Right? And the history of this of these types of devices has a rich rich um history. You've got the earliest FDA approved devices in 1997. Something like the Vegas nerve stimulation. This is purely open loop. This is something where you actually

43:39stimulate the vagus nerve that's like down near your neck, >> not actually in the brain. And what you're actually doing, you can think of it as like a lawn mower. Not a lawn mower, a lawn sprinkler. >> Okay. >> Okay. That just like turns on and then it like sprinkles. >> It's doing that but with electricity. It's this is what we would call open loop. >> Okay. because there's no feedback coming from the brain and there there's no it's just it's on a timer >> and it's just like going it's just stimulating the the vagus nerve at regular intervals. >> It doesn't know if the grass is growing. It doesn't know what patches of grass are growing more than others. Any of

44:19that kind of stuff. >> Any of that kind of stuff. It's just blindly implementing. >> I know it needs water. >> Yeah. And and and it it had its effects. >> Sure. And it was great. 2013 there was a massive leap into closed loop systems. So this is the RNS system which is the responsive neuro stimulation. Meaning I've got some kind of read mechanism that is going to maybe identify that a seizure is happening and then only stimulate when the seizure is happening. Right? So this is where the responsive thing comes in.

Putting patients in the feedback loop

44:57Now instead of open loop, we've got a closed loop kind of thing, right? The device knows when the she seizure is coming, but at the same time it's not closed in the sense that the patient is aware of what is going on, right? You can't you can't tell the patient maybe you need to sit down, maybe you need to pull the car over, >> alert, alert, >> like yeah, alert you're about to have a seizure, >> right? Which would be obviously great. >> That would be great. And that's where the breakthrough currently comes in handy. >> This is this the patient was never involved in any of the the the communication cycle of this. Obviously, we weren't there yet. >> Yeah. >> But because it required like a human

45:37being with a trained experience to understand all of these things. Now, with the advent of machine learning, you can put the patient in the loop >> of his or her own medical >> destiny. >> Yeah. >> It's pretty incredible. It's a this is a really big deal. Um because I think it also, you know, there's this whole been this whole era of like quantified self and you're going to get all this medical information from all these devices we have around us. It's going to help you improve your well-being. >> But part of and this is that's a different thing than medical and clinical treatments and solutions. I'm not saying they're the same thing, but there's been this big promise with the rise of technology as bringing us closer

46:18to understanding our own bodies >> in real time. M >> um and this is a perfect example of how that can have a hugely important impact for uh epilepsy patients. >> Yeah. And I think I think it's not only limited to epilepsy. This particular case >> is taking epilepsy as kind of like this microcosm because it's it's such a rich data set in terms of like you know you get >> in intraraanial stimulation. There's a lot of data that's going on. It requires a lot of expertise that perhaps a machine learning model can elucidate, but um it's it's absolutely an

46:59incredible paper. It's a team out of the University of Pennsylvania. And what they've done is fundamentally re-imagine the patient device relationship by bridging this EEG that comes with the the patients of epilepsy with agentic large language models. Right? That's the key. You're tying in the brain waves from inside the brain of these epileptic patients with now agentic models that use large language. >> This models this is the the promise that has been again large language models are not just a tool in the context of a chatbot. >> Yes. >> Is kind of the point here.

47:40>> Yeah. They've built a kind of interface that allows the patient to maybe interface with the chatbot, but the chatbot now has access to all of the data from within the patient's brain itself. It's incredible, dude. >> Now, now I actually like you know from a personal standpoint um my PhD had to do with electrophysiology, right? where you know you're taking electrodes, you're sticking it inside um the rodent brain while the rodent is freely behaving inside mazes inside you know behavioral tasks. One of the in one of the bane the bane

48:21of my existence during my PhD was deciphering between real neural signal and all of the artifacts that come with behavior. >> Because if the rodent is chewing, that's going to cause electrical signal, right? If the rodent is um going into like a micronap, that's going to cause electrical signal. >> This problem is real even for epilepsy patients, right? Because if you've got electrodes that are going into the brain, there's a giant signal to noise problem because brain waves are incredibly chaotic and highly variable. Here we're seeing brain waves where

49:03chewing artifacts come in. All of those all of those jagged things, that's when the person is chewing. >> That's crazy, >> right? If you have a really terrible machine learning classifier that just thinks you're having a seizure every time you chew. >> Yeah. Not great. >> That's not going to be great for two reasons. There's this thing called alarm fatigue, which is basically the boy who cried wolf, >> right? If if every time >> Yeah. If every time the thing just says, "Hey, you're going to have a seizure," and you're not having a seizure. There's so many false positives, well, then you're going to just disregard the thing. Right. Right. And two, if you're kind of someone who has seizures and has anxiety, that's

49:44going to add to your anxiety and it's going to like be detrimental to your well-being at the end of the day. Like the whole point is to get better and like having these alerts is making it worse right? >> You're constantly put in this state of a heightened >> Exactly. >> You know, u defense mode anyway. >> Right. And and the the reason why this is so hard is because historically tuning these algorithms requires an expert neurologist to manually go through all of the data from your brain and be like, "Yep, that was you chewing. That was you not chewing. That was an actual seizure right there." Right? You got to have some MD be like, "That was an actual seizure. We got to tag that,

50:25not tag the other thing." Now, >> I can't believe it. >> You flip the script entirely, right? You're you're turning the patient into a collaborator for training the neural network, for training the machine learning model. >> This is so good. >> It's I dude I think I think it's so cool. I mean, there are things that are involved here that they were very careful about. You've got to be HIPPA compliant. And >> um these guys at UPUPEN, they made a system that features a HIPPA compliant instance of the GPT40 mini. >> So this is one of those mini models that OpenAI has, right? which means that you can like like put it on a local thing, >> right? It doesn't need to go to the cloud. >> It doesn't. Yeah. Yeah. So now patient

51:06data is not going to the cloud and like you know the the the Levant El Pog and Buckmaster >> uh nightmare where right where it's like they if if you guys don't know the Navier Stokes episode right Buckmaster was talking to OpenAI codeex about his solution for Navier Stokes and now behold >> lo and behold a week later >> yeah OpenAI came out with Navier Stokes right like I don't know I don't know how that happened Well, in this case, they've got a GPT40 mini that is local, right? So, there's no way that HIPPA that the data from the patient is going into the cloud. It's it's and it it's

51:47non-trivial to be able to execute something like this on on one of these smaller mini models. Like that's been part of the challenge generally is is you want the capability as you go to smaller parameter based smaller parameter models, you lose some of the capability you're looking for in an LLM. So, to be able to uh work within that constraint and still get the solution is I think an interesting part of the the the success here. Exactly. And one thing that I want to note here is the LLM is not actually doing the medical diagnosing. Okay. They've got a specific model, a neural network that they've trained for earlier purposes that is doing the medical diagnosis. Like there there's a

52:29separate model that is like taking in the brain waves and being like, "Hey, is this epilepsy? is this not blah blah blah. The LLM is being used to interface with the patient because that's the whole point of the language, right? It's like as a patient, I want to use natural language to interact with whatever data modality. >> Crazy, >> right? >> So, it's really agent-based. Like there's the agent that's the LLM, >> then there's the agent that's like has nothing to do with English, but just trying to classify brain waves. mate. Yeah. I I I think I think I think it's really cool. So So you can imagine, right? Like the deep learning model

53:09detects anomalous brain synchrony. The AI then texts the patient, "Hey, did you just experience a seizure?" >> This is so crazy. >> And and then the patient is like, "Uh, no, that was just me like jumping up and down." Or, "Actually, that felt weird. That might have been a seizure." And now it's again you're taking the the cycle of training goes from patient collects data over x period of time. It goes to the physician. The physic physician spends x period of time with some assistant or someone else that's tagging manually and then >> the physician doesn't even know the personal experience of the patient. So it's like experiencing the seizure. So they're just looking at it and based on

53:50our past things this is what normally and so it actually it has both the combination of bringing the patient into their own >> uh giving them a sense of control in their own experience but it also has a very functional benefit to improving the detection >> because the the part of the challenge is always patient translating their experience to the physician and then like that back and forth which there's it's a very uh noisy the signal to noise there is also not right right >> I mean it's absolutely incredible I think I think we're standing at a new dawn of medical technology right we're moving away from passive hardware that

54:30treats patients as biological substrates and towards semiautonomous conversational partners >> that help us navigate our own biology like imagine you have your you have like agency over your own kind of outcome in some sense And and I want to make a small note here. I there's a I know there's a lot of u people have a lot of feelings about specific providers of foundational models. >> Yeah. >> Uh I think part of the point here is >> this could be done on an open source mini model. It's not it's not open AI mini specific. It happens to be what they used in this case. >> The large language model is doing a very

55:11small amount. Right. Honestly, >> right. Right. And but I just wanted to make that distinction that you you know the >> now that we understand it can be done. >> This can be extrapolated to whatever provider you prefer. >> Yes, exactly. >> Um >> yeah. Um it's incredible. I want I want to take a brief note about how they actually did it because you know even though this is a rundown episode, I want to get into like some of the >> We got to hit two hours. You know how it is. >> Yeah. Yeah. Yeah. I'm sorry guys, but you know these pen researchers, they recruited 13 adults >> in the epilepsy monitoring unit. The setting here matters because it's an epilepsy monitoring unit which is one of these things where you go in high quality recording there's clinical

55:52supervision there's detailed event review and also sometimes they even take reduced medication to induce more and more epilepsy >> and more and more seizures to actually get more and more data. So that matters a little bit, right? Um the platform actually there's three parts. There's a stream of EEG and clinical information and then there's software that extracts whatever the useful measurements are and then that goes into a phone interface that then connects to the patients and the patients can be like just like in the epilepsy monitoring unit with their phone talking to this agent that is also talking to like the models that are interpreting data from his brain. This

56:34is so good, >> dude. It's It's absolutely incredible. >> It's clever, too. Yeah. It's so obvious. Like, it's so obvious in retrospect. >> Yeah. And And there's so many multiple modalities here. Like, there's a system that estimates sleep stages so that that is not confounding the algorithm. There's a system that med that estimates medication because there's medication information in the patient's notes, right? That'll tell you, okay, the patient took the medication at this time. And then there's a model that tells you how the medication is deteriorating with time. >> And that's going to inform this ensemble of machine learning

57:14models. >> We have all these inputs into the model that gets to decide, do I set the alarm on or off? And you also now have the feedback of the patient, which is arguably one of the most important inputs. Yeah. Into that >> uh let's call it detection system. Um, and to be able to have them all connected and streaming >> Mhm. >> real time. >> Yeah, it's absolutely very very cool. And the result is a structured collection of measurements, right? You can you can have these measurements be queried, plotted, and discussed. And the chatbot and the seizure detector are different models that are doing different jobs, but they're all coming together for this seamless sort of loop. The patient has a brain interface. It

57:57goes through all of the stuff, but the patient only only interacts with it using natural language, which means that, you know, it's kind of a democratized system. The patient doesn't have to understand what the machine learning nonsense is happening. Right. Right. >> The chatbot is going to go ahead and ask it questions and get the feedback. This is something we talked about at the end of our Navier Stokes episode, which is when we talk about these concepts of AI, we have to think about them as systems, not as individual component parts. Because this is a perfect example of why that framework of how to think about these things matters because they can be individual modules with discrete functions that can work in collaboration

58:38towards an end goal and they're also not just LLMs. There's a whole variety has a whole ecosystem of meaning and capability. Yeah. >> Uh and this is an example of combining two things in machine learning and the LLM. >> Exactly. >> Uh to for an outcome that has clinical relevance. >> Yeah. And and actually to piggyback back off of that, um you you can ask like what exactly is this m this there's a bunch of agents, right? What exactly is happening? Well, let's suppose a patient asks how much sleep they got last night and how good it was. Well, they've published kind of a flowchart of how things happen. Yeah. >> You've you received the patient message.

59:18Now, if the message is unsafe, then you don't do anything. you you maybe you know alert the physician who's in charge and things like that but if it's safe then you have a message router. >> Yes. >> And the router asks okay is this a general chat or do I need to actually do a data query? >> Yep. >> Right. If it's a data query >> then it actually instantiates a SQL query. Yeah, >> like there there's there's an agent that goes, "Okay, I'm going to go into the SQL database of all of the data on this patient and like instantiate get that bit of EEG or whatever brain wave and then now instantiate another thing that

59:59asks maybe a clarifying question that asks what are the tools that I need to understand this data and then you go back and send the AI message through the chatbot to the phone, >> right? it's it's able to actually look at the raw data, analyze the raw data, and kind of understand what to do next. >> And and from this flowchart, I'm seeing there's actually also two safety uh gates. One at the front end to moderate the patient me patient message if it's unsafe. And then once all of the processing is done, like you just mentioned, it also moderates the AI response. Yes. If it's unsafe prior to getting to the patient. So again there, you know, they've thought about both the

1:00:39front and back end from a a safety perspective. Um, and I, you know, the medical researchers, clinical researchers are some of the most uh anal retentive about details like this. >> Yeah. >> Uh, in ways that maybe uh people wish the Frontier Labs would be more conscientious. >> 100% dude. 100%. Um and I I one last thing that I want to talk about is the fact that the the system can ask questions back. So it's not just that the patient asks questions. Um for example, suppose it has detected a possible seizure. It sends the patient an alert asking hey did you consciously

1:01:21experience a seizure? Sometimes you have these kinds of mini seizures where you don't really experience anything and sometimes you do. Um, in any case, the the patient answers yes, no, or uncertain. And in machine learning, that means that you've established a label >> to the EEG. >> Yeah. >> That the the model had triggered as something that possibly could use a label. >> Now, usually you only want highquality labels, but actually even in the case of these lowquality weak labels, you can do a bit of learning. There's a technique in machine learning called distillation.

Personalizing seizure detection

1:02:00Here's the idea. You've got a pre-trained model, right? Let's say let's say you have a bunch of data on epilepsy and you've got EEG recordings and you know when the epileptic seizures happened. So you've got this model that is pre-trained on thousands of hours of epileptic data. And now you want to sort of fine-tune it to this specific patient, right? because this specific patient's brain is different and the way that they experience seizures is different and so on and so forth. You can use this technique called distillation. That's going to do two things. One, it's going to try to get it right for that specific patient, but at the same time, it's going to penalize changing up what you already know,

1:02:42>> right? There's there's a twofold thing that you want to do. You want to you want to gear towards what the patient specifically is experiencing, but you don't want to forget all of the >> Yeah. >> stuff that you've learned from general epilepsy data, right? And that's this idea of model distillation. It's a penalty for changing too much on the basis of limited evidence because you have all this hours of evidence and now you're getting what maybe a few hours of evidence on this patient. You can't you can't totally upend your world model >> 100%. >> Based on everything that you've sort of realized beforehand, right? And now the the the question is are do the models

1:03:24actually do better? In fact, they do. The models do better from this closed loop arrangement between the patient and the training. Right on the left hand side what we're seeing is um from baseline the that's called the the area under the PRC curve the area under the precision recall curve. It's a it's effectively like how well is the machine learning model doing in terms of precision and recall. You can see that the baseline model is highlighted in green and the finetuning steadily increases. Mhm. >> The F1 score again another metric for how well um precision and recall is

1:04:06working is increasing. The false alarm per hour is decreasing. Y >> and that's crucially very important. >> We are on the foothills of continual learning in clinical application and I I I continual learning is a very specific thing. >> I know that this is not literally continual learning but but it I said that's why I said foothills >> because part of the challenge with all of these model architectures is they're all pre-trained. >> Yeah. >> And so they can't learn from new information is the argument that's said a lot. >> Exly. And and even in this case, right, they they're scaffolding off of pre-trained models. The the LLM is the

1:04:47chat GPT4 mini. >> That's a pre-trained model. The the seizure detector is a pre-trained model. But especially because of the complexity of biological variation and individualized medicine, you are going to kind of necessarily need this foothills of continual learning capability, which is can we allow this distillation to happen on a per patient basis in a way that takes the benefit of all of the massive amounts of data but can correctly weight >> how much to take the influence of the individual patient. in future decisionm.

1:05:27It's a very narrow thread to to to pull, but

What the BrainChat study does not yet establish

1:05:33the effect efficacy that we've seen in this study in particular is very encouraging in again being able to now kind of take something out of the box and make it evolutionary in tandem with a patient. >> Yeah, I I think it's really >> a really really cool >> It's really cool. um study it's it's a small study it's short and it was conducted at only one center right um the participants they need sufficient English cognitive capacity and smartphone ability I guess um so you could think of that as a kind of limitation but I think a convincing next study would be to measure the outcomes

1:06:14that matter to patients for example like the seizure burden the quality of life anxiety the amount of emergency visits the clinical workload. >> Yep. >> Um I I think I think the fact that we're seeing this though is very encouraging to me because you know >> we we we hear a lot of AI doom and gloom. You know this is some good news. This is this is what was promised to me. >> Okay. When when when AI came in, they were like we're going to make medicine better. We're going to make your health better. That's what they said. >> This is what was promised. >> This is what was promised. >> Right. And so we're we're starting to see it slowly come out and it's not just

1:06:56the frontier lab companies that are using these technologies. Um, and a fascinating second story. And not to burden you with another AI related story, but unfortunately uh when we were uh in our 20s, the phrase was software is eating the world. And now in our 30s, AI is eating the world. And so there are just a multitude of areas, especially in science research, where it it comes up. And we are going to now move into our last story which is going to be a little bit of a zoom in but also a zoom out of what we mean by this whole idea of automated

1:07:38science research or AI going end to end from hypothesis to clinical outcome. And I think this will be an interesting there's a story that came out today that I think will give us a nice on-ramp to discuss what is the current state of play in what is a pipe dream at this point around AI scientific discovery. So

Anthropic scorpion-toxin binders

1:07:59earlier today we had planned this episode and then this story dropped and I was like oh this will be interesting >> and so this is a last minute edition uh about the other frontier lab that we did not talk about in our last story which is open AI. This is an anthropic story about scorpion binders and the path from AI to real world science. And so what are we talking about? We've seen all these stories like we mentioned every week about AI is going to change science and make all these new discoveries. And today Anthropic published this research article how claw designed protein

1:08:40binders against scorpion neurotoxins. And it was a collaboration with uh uh Citrini research and the laboratories adapt bio and twist bioscience. >> The idea is they're trying to design proteins for these dangerous venoms that are in scorpion venom potentially leading to a treatment. So interestingly there are there are some antivenenoms that exist uh but they're very specific to different species. They're species specific and even with any individual species, there's also some regional things and so it's a very complex as all the biology we talk about it's very

1:09:21complicated and so one of the holy grails is this idea of discovering a universal antivenenom. >> Uh obviously >> Yeah. Yeah. Yeah. >> Cuz then you currently see the antivenenoms that have been created are not in underserved areas or areas with less of this clinical and medical infrastructure. Y >> um and so you know the challenge becomes when you don't have these antivenenoms for for scorpion uh stings the the current clinical solution is this idea of aggressive supportive stabilization right you're basically waiting for the body to naturally metabolize but you

1:10:02have to keep the person alive while that's >> like happening right >> that's the stabilization part just like keep them alive >> right and so that can be done by supporting for example uh artificial uh air like if they can't breathe and things like that and they also have this idea of utilizing secondary drugs to counteract the specific damage that the venom is causing which in some cases can work and it impacts the nervous system directly. So the this story was immediately all over the place, okay? >> Because again, marketing, right? >> And so the headline read, "They tested 211 designs in the lab and 17 met their binding criteria for four toxins." So

1:10:43like, what is I'm reading this, I'm like, what is this? >> What does this actually mean? Um, crucially, they've not shown that this can actually neutralize the toxin, protect an animal, or actually help a human patient. Um, and they explicitly made that clear that they did not create an an antivenenom, but this is sort of this incremental stepping stone process that we've talked about previously. Another small detail point is none of the uh candidates also could bind to multiple toxins. So in the journey for a universal antivenenom >> that's not >> it's still not there but it was still an

1:11:25interesting design challenge where they were able to find the ability for the protein to bind. Um which is still interesting and impressive for the system to have hypothesized and designed on its own. So, the first thing I want to look at here is like what did this experiment actually establish? Um, and so they act they have this video overview we'll walk through really quick. So, this was the scorpion from a a Mexican scorpion or scorpion from Mexico that the CH2 is the the toxin. And so, we've talked about sodium ion channels uh that allow things to transition. And so this toxin comes in

1:12:05and activates so that it causes the ion channels to become overactive. >> Okay? >> Right? And so that's obviously a problem. So there are current antibodies, this LR antibodies, actually a lab engineered antibbody that we know can bind to it to prevent uh this from happening. So Claude designed this CN2BI protein that is is able to bind to the venom but it conflicts what it looks like conflicts with where the antibbody is able to connect but not all antibodies this RU1 antibbody uh the anthropic designed one does not have a confliction with and what's interesting is what they were basically trying to

1:12:46identify is with LR the initial one we saw can we bind to the same location for a known antibbody body that we know uh is able to combat the venom cell. So we're going to repeat this one more time and I want to come back to this really quick. So they use two antibodies. >> Yeah. >> The LR and the RU1. They bind to different locations on the on the this phenom itself. >> This binding test was trying to figure out can we design a protein that can bind to the same location as a known working antibbody. And so it was a competition. uh yeah >> uh experiment here. So now again we see the claw designed protein is able to

1:13:28connect. When the LR antibody is there it's saying it's in conflict. >> Yeah. It's in conflict. It's not able. >> So it's it's kind of in the same location. >> But when RU1 is there it's like it's a little bit but it's only at 16%. Right. >> Yeah. Yeah. Yeah. Yeah. Yeah. And so the idea, what they're trying to say here is that look, we can kind of have a binding in a similar location to something we know that works, but that does not mean it neutralizes the toxin. We're just saying that it can bind, >> right? Am I doing okay so far? >> Yeah. Yeah, I think that makes sense. >> So LR came from an earlier antivenenom research from UNAM in Mexico. They engineered a human derived antibbody fragment that showed that it protected in mice.

1:14:09>> Um, and it's a laboratory reference. A lot of times with these antivenenoms, what they end up doing is they'll inject a horse with the venom, then take the antibodies from the horse and then do some voodoo management and then give it to the human. >> Fair enough. >> Um, and then the >> the RU1 is a similar that just attaches and binds to a different location. So they wanted to see how it worked in in both cases. The idea here is like you know neutralization is the idea of capping the ability of the venom to create that overactivation. Yeah. Um and ultimately like that's the solution here. So the the headlines and the analysis are always really So what we're

1:14:50saying that they've been able to do is they designed a bunch of uh options for what could potentially be successful at binding to a similar location as the LR antibbody. Uh 211 of them. Uh they found as a result of that that they were able to get 17 to meet the binding criteria. >> Okay. >> Across four different toxin types. >> Mh. >> Uh so that's interesting. >> Yeah. It's it's four it's like generalized across four different toxins. That's pretty good. >> It's interesting. Okay. And so but this is still not helping a patient.

1:15:30>> Mhm. >> Right. because we've not even gotten to the point of being able to actually confirm that it neutralizes the toxin. So I wanted to take a step back and say what are the steps between this binding mechanism >> which was the result of this and a medicine to a patient. >> Yeah. >> Um and uh

Why a toxin binder is not an antivenom

1:15:49>> Catrini actually put this road map out that we can look at. That's kind of how they sort of define it. So step one, right, is does it neutralize the toxin? We haven't even gotten to step one yet. We've found a binding mechanism and so now step one is making sure that it can actually neutralize the tox because it could bind but not necessarily neutralize it like the actual >> Oh, interesting. You know what I'm saying? I see. Yeah. >> And so there's a difference between those two things. We know it can connect but we don't know that it makes uh that it protects the actual nerve function itself. >> So then number two, does it cover the relevant venoms that we're trying to address? >> Number three, does it work in a body? uh in this case in animal testing. Uh then

1:16:30we get to number stage number four which is is it ready from a safety perspective for human testing >> and then five does it actually help patients once it gets through that developmental process. They estimate that whole pipeline for this specific use case about four to seven years plus. Um and all we're talking about is not even yet having solved step one. >> Step one. Yeah. uh but this is the pipeline that all these AI is going to cure X Y and Z. This is the pipeline they kind of have to go through. And they've identified a few things in their chart about where they think AI is going to be the most impactful. So in proving the neutralization of the venom itself,

1:17:12they're like, I think we think it'll have moderate impact >> in being able to then cover once they know it can neutralize, they think it has the highest impact of being able to then generalize it across uh all these different classes of venom, this universal antivenenom. They also think that in that first stage of animal testing that it can have high impact as it relates to animal testing. But when you get to human preparing for human trials and the trials and acceptance process, it goes to moderate to limited which is obviously very understandable. I think the point here that is sort of being brought up is >> there is there's a real world aspect to this stuff that is just naturally going to be slower unless and until some of

1:17:52the processes the regulatory environment and the way in which we do those downstream processes fundamentally changes. So that's kind of part one of what this OpenAI announcement >> was about. And it's interesting because it dovtales with the wet lab story we talked about a few weeks ago where everyone's like, "Oh, Anthropic has a wet lab." >> Yeah. >> You know, but it was where the AI was designing and the human scientists were in the lab actually conducting the entire process. >> Yeah. >> Um itself. So I want to sort of briefly talk about the timing

Fast AI designs, slower laboratory testing

1:18:31because the promise here is that this is going to accelerate and it's going to happen very quickly. >> So this next chart kind of looks at how long did it take them to create the design candidates and how long did it take them to test those candidates in a laboratory with the two partners we mentioned earlier. They were able to get the first 180 designs in 64 hours which is quite quick. >> That's quite quick. It's quite fast. The easiest part of the process maybe, but it's still quite fast. >> Yeah. Two and a half days though is quite fast for 180 designs. >> It's quite quick. >> Yeah. >> The first binding results arrived after 39 days, which is from the beginning of

1:19:12when they started the design process, not after they got the design process. So about five and a half, five five and a half weeks. um to get to this point where we have the designs done, we've tested them in the lab and we have these um candidates in between there. There's synthesis, there's protein production, there's getting the reagents, scheduling the work, actually measuring what happens. There's quite a bit. >> Yeah, there's quite a bit that hasn't been optimized for

Connecting AI to the physical lab

1:19:40an AI lab ecosystem. Let's say that that's exactly right. like like this this is there's so many parts of this that are are still sort of not even optimized I would say just like catering to human time scales like PhD student time scales >> right right and and I think part of what we're trying to talk about in this story is looking at each of the stages of this process >> and where is AI but also the combination of automation and robotics ultimately interact with the real world. >> That's right. >> And so this is where the AI and robotics

1:20:22fit in that process. So I I wanted to again I was trying to >> help myself understand what are we looking at here. And so I created this map of how AI is going to like meet the physical lab and where the current state of play is and some of the sort of this is not a uh complete map of every player in the space but it's just to give a kind of representative example. >> So it kind of breaks down into these three stages. There's the design and prioritize. So we just talked about that in the centropic story with the scorpion antivenenom. So the AI is going to design and prioritize >> correct >> in this model. >> We we may still define the question,

1:21:04>> right? >> Scorpion antivenenom, >> right? Be like, give me an antivenenom to all of scorpions, >> right? >> And I think the original prompt for this was about 35 pages that they fed. They had I think 300 GPUs. It costs a total of about $40 to $60,000 to run this design and prioritize process, >> which is not crazy. >> No, but it's a little expensive, but it's, you know, not crazy. >> No, it's it's not the usual stuff that I ask Claude and Chad GPT to do, right? >> My prompts are not 35 pages, >> right? And if one of those ultimately became something that was able to get to the end of process, that would be

1:21:44fantastic. Okay. So, a variety of the frontier labs are doing that that that design and prioritize, but we're talking about this anthropic story in combination with Catrini. So, then sort of stage two is this idea of physical automation, not necessarily AI, anything. Uh we have a couple of players in here, MIT, uh Emerald, Cloud Lab, Stratios, and GKO. They're trying to now take the physical process that humans do in a lab >> and have it be run by a robotic automation. Obviously, it's a connective tissue because you can take part one and then send it to part two. >> Now, there are some companies that are trying to do the full

1:22:26closed loop cycle going back to our previous story of the learn and repeat. This is Lila science madiraa where it is the combination of both the AI designing prioritizing the robotic lab executing getting the measurements beating that back directly into the AI to then decide what to change and then retry right human oversight but yep >> kind of a closed loop cycle these processes are also using a variety of shared infrastructure for example Carnegie Melon is providing some of this shared infrastructure as well as Slack, the uh national lab, the uh Stanford linear accelerator lab. Yep. Is is

1:23:08giving some of the infrastructure around these pieces. So that's that's where it's sort of where we're trying to go is where Llaya Sciences is effectively saying we are right now. And so the

Robotic optics and automated laboratories

1:23:22first step that the frontier labs think they can take is this design and prioritize because it does not require this connection to the downstream robotics and automation piece which is not their bread and butter. So now let's transition and this is something we've talked about a lot is this with the methods is this equipment and measurement. We talk a lot about tools >> on the pod. We talk a lot about being precise in our measurement in our methods and MIT actually gives us a great example of this automating step. This is not necessarily new as in new in the last couple of years. It's been happening for some time but understanding how this now connects to

1:24:03these AI workflows is a different way to start to think about these steps. So researchers at MIT working in collaboration with Nokia Bell Labs and Arizona State built a robotic system that can assemble and align uh an optical experiment and this was actually highlighted uh recently in September. uh this research always talk about the funders supported by National Science Foundation, Army Research Office and some other funders, private funders including Shell and this was one of their demonstration videos uh where they have a robotic arm that in 50 steps in 30 minutes can set up the these

1:24:45lenses for this optical experiment which we've talked about previously. And not only can it set up the lenses if any it's because it's constantly taking measurements, it can also make a change to fix something when there becomes when a misalignment arises as well. So it's both things. It's a >> this is crazy >> human designed setup. >> Yeah. >> But it can set up and iterate when there's a misalignment. I mean I mean so already this is giving me flashbacks to my PhD days. Like for example, the table that they're on. You see all those holes? There's little screws that you can put in um and you can fasten like

1:25:26mirrors and lenses. um any type of tabletop experiment, meaning atomic, molecular, optical, like if you want to do molecular traps, a lot of quantum computing experiments are literally just this, which is establishing lenses everywhere and then um you know, aligning them to the point where you can suspend atoms in a laser or you can, you know, isolate a single atom to to do some type of spect spectroscopy measurement. >> This reminds me of the MIT double slit experiment episode. Yeah, this this is all that is like a lot of these

1:26:07experiments are literally just this >> and just so I you know I want to ask you briefly like the comp the sensitivity and complexity cuz someone who's watching this might be like oh this looks >> pretty No, this is incredible. like if if they're able to align a laser and like make sure that the transition frequency is exactly aligned to whatever you know system that they're trying to control like that's that's pretty incredible. Like if they're able to do that just generally like that's >> that's like half the work of these PhD students like my roommates you know like

1:26:48uh Pratik Puri >> I I don't mean to call you out but this is half your job like you know like so >> um Elliot Bora another friend of the pod like this is something that he does a lot I mean obviously there's a lot more that's in his head right there's a lot more stuff that they could do perhaps if they didn't have to spend their time >> doing that >> aligning lenses and lasers. And this is this gets to this is this was a demonstration going back to our three stages of mainly this idea of execution and measurement because AI can design all these things but we still have to be able to execute and measure

1:27:28>> and this is for optical experiments not biological experiments so different category but I'm just trying to give the listeners a sense of where these component parts are that people are ultimately trying to connect in order to get this closed loop endto-end uh AI based research cycle um you know but that step might close the equipment control loop >> as one component part of that now the next video that this is sort of a promo video so there there's some

Cloud laboratories and remote experiments

1:28:04>> history about how we've done these things for quite some time right >> so this is from Emerald Cloud Lab and Stratios and they kind of address a slightly different part of this execution which is making physical laboratories accessible through software through the cloud remotely. So I'm in one location, I need to do X, Y or Z in another location and the entirety of the process that I need to be done is remotely executed by automation and robotics. But I have a software interface that I can interact with in real time to execute. So like a lot of folks might go go to the doctor

1:28:45and then they need to get, you know, a blood test and then they send it to Quest Diagnostics, right? So this idea that you have one person doing the question, one person doing the execution and then it comes back to your physician to give you the result. The difference here again with still some elements of human involvement for some of the component parts many of the components are now directly cloud connectable and fully automated. >> This is a I think a a big area. >> I think this is very big. >> No, this is crazy. This is absolutely and you get how long it takes to run each. You get the real-time data back in the format. It's

1:29:26a It's a It's a wet lab that you can access through the cloud. >> This is >> Isn't that crazy? >> I mean, no, they're doing mass spec, I mean, or something, you know? This seems like >> this seems like stuff that would require specialized equipment, but there you go. Like Right. >> Um, >> no, I mean, that's actually incredible, right? Because for the longest time, we've seen machine learning researchers do this. you just access GPUs from the cloud AWS and then and then you know you're doing machine learning experiments. Well, now you're actually doing like real biological experiments. You've got pipets that are automated. You've got >> um I think I saw a centrifuge that was

1:30:08automated, right? And they did it correctly unlike the Ryan Gosling movie. >> Um I >> look it was it was one of the only flaws I brought up. I was wondering if I was gonna get if you were gonna say or miss it. But >> but like they did it right. Like the the AI knows to to balance the centrifuge >> unlike Ryan Gosling. >> So but no, this is incredible. >> And what again this is not a promo for any particular entity. There are variety of companies that are trying to do something. >> No, I'm just saying this is a new era in uh in scientific. >> Correct. >> Yeah. It's like so I mean one one thing

1:30:48I I want to understand is like is Emerald Cloud Lab going to be a co-author on I I wonder like as a as a former researcher in like a PhD or a postoc setting it's like I'm always worried about like who's going to be co-author on the paper if I were to use this data right do they have some kind of thing in their contract being like we got to be acknowledgements are fine co-author it's a bit you know like okay Maybe a maybe a footnote. Yeah, m may maybe a footnote. And one of the things that this brings up that I was kind of thinking about that's not necessarily as obvious cuz like the idea is like oh you know a lot

1:31:30of the thoughts around these AI implementations is it's going to allow the rich to get richer which is true in some >> in some in a lot of context that's true but what it does also like what type of research teams that might not have access to these types of resources who are in underserved environments. What kind of questions can they ask >> now that they and it's the idea of like AWS create and all this cloud computing allowed people to build websites and do video and we can do this show because of >> cloud computing and distributed networking and you know this wouldn't exist without that. So it does bring up the question like what kind of research

1:32:11questions and what access that teams that otherwise would not have been able to access these kind of highfidelity resources particularly in my family who's worked in you know global health and neglected diseases. um it becomes a very interesting democratization of the access to the types of tooling when you have this cloud accessibility and that that was a non-obvious conclusion uh after kind of digging into this is like it it is a different paradigm that allows us to ask some different questions. One last sort of note I want to bring up going to the third stage because we talked about design and how to prioritize what questions to ask. We

1:32:51then looked at automation at two different levels. right? One where the humans are still involved in the lab, one that's cloud accessible. Um, Llaya Sciences is trying

Closing the loop with Lila Sciences

1:33:02to connect the full loop, right? They had this catalyst study. Um, the AI helped select compositions, automated the equipment, made the test materials, and the measurements informed subsequent choices. So, it was a very narrow question. >> Yeah. But it's closed loop. >> But it's closed loop in some sense. Um they evaluated almost 3,000 compositions related to hydrogen production. Um and this diagram here kind of shows how they look at that cycle. Um you know and what they're ultimately trying to do in terms of this closed loop catalyst discovery. And so the the the goal here was trying to give a baseline lay of the land. Um there are

1:33:46more than just the frontier companies doing this. people are attacking different component parts of the process. Some people are trying to do the whole thing. Some people are just trying to do a module of the thing. Um and there is sort of this shared infrastructure. So Carnegie Melon has AI science foundry which is trying to connect these cloud labs and scientific infrastructure to like a shared this sort of shared distributed research infrastructure to make sure it's >> evenly distributed and accessible for many people. Slack has this new Spire project which is trying to connect their robotic sample handling, beam control and real-time analysis. And we actually have uh a reference video here. This is

1:34:27from from Slack's Spire project. Now, it's it's just part of the execution and measurement infrastructure. It's not the completed future system. But again, I think that there are multiple national labs, universities, private companies that are all moving in this direction. And when we go back to one of the reasons we started the show, the thesis was we're going to be in a Cambrian explosion of a whole variety of breakthroughs. Yeah. and we want to be here having a front seat and bringing you along with us for what will happen. Again, there are very complex social, economic, political, and other issues as

1:35:09it relates to quote AI. Um, but on this show, we talk about the researchers who are using this in very specific ways that are ultimately looking to create opportunities to benefit us collectively. And it's important not to just say AI is bad in its entirety um when there are really interesting and valuable use cases that are uh coming up. Now I will just end on this note of all the doom scenarios uh expect there to be automated wet labs. >> Yeah. >> And then an accidental release. >> Yes. Obviously, none of the things we we discussed are related to pathogens that

1:35:52are extremely deadly, but there are certainly a lot of consequences as it relates to the progress we've already made with >> cloud wet labs. >> Yeah. >> And dude, I mean, it's it's uh it's the best of times, it is the worst of times, as Charles Dickens said, right? Um, and it really feels that way. There's if we if we do nothing, it's going to be both, right? And um, the scales only need to tip one way or the other. It's kind of like the chemistry Nobel Prize, right? You only need a slight asymmetry and then all of a sudden it's very much

1:36:33not a great thing, right? Or all of a sudden it's a very good thing, right? So um >> we we uh we want our homo kyirality to be positive >> positive in our direction >> in our direction not in the AI's direction. Um >> uh it's so I I I thought you'd find this because we kind of touch on aspects of this but >> as we continue in future episodes this is going to continue to come up. >> Um and uh I successfully talked through a live molecular diagram. >> Yeah, that was pretty good. >> It was not bad. Yeah, >> I'll get better. I'm learning, guys. I'm trying to get better. Um but uh >> I I thought that was that was pretty cool. I mean, um the

1:37:17the laser stuff >> is was crazy to me because I've been in those labs, right? Um in the physics department at UCLA, like we those those tables are the bread and butter of every tabletop experiment. Um, so that was crazy that like there was just a robot that could just hang out and do that, which was like half the job of the PhD. >> 30 minutes, 50 moose, >> right? That's crazy. Yeah, >> it's not bad. >> Yeah. And no breaks. No, you can just

Wrap-up

1:37:49work overnight kind of thing, you know. Yeah, >> it can adjust when there's an issue. >> Yeah. Yeah, >> it's not bad. That's pretty good. >> That's not bad. We are going to continue to bring you the frontier of science research globally uh not just here in the US but from all around the world. My name is Lester Nar joined as always by my co-host and our resident PhD and I also just want to remind everybody Signal award-winning podcast >> bronze. We're going for gold. We're going for gold, but it'll it'll go it'll go like my point. There it is. Krishna got the

1:38:32point. And for those of you watching on Apple Podcast on video for the first time, welcome. We know it's been a long time coming. We will see you all next week. Heat. Heat.