With that, I'm going to transition to the real show of the day, which is we're delighted to have Dr. Michael Howell as our opening keynote. I've known Mike for a long time, and you're in for a treat. He is the chief clinical officer and the deputy chief health officer of Google, a small organization some of you may have heard of, probably go on it 100 times a day, as we all do. He's had a long career really focusing on issues around quality and safety, and I have both shared lives of being chief quality officers in places, a really fun job, and really I think in particular has really thought long and hard about how technology can really improve health. He is certainly an expert in healthcare delivery science, has written a book on it as a matter of fact. He's done lots of expert panels as we all have on CMS and National Academies, and his writing is something we have all read. In the last few weeks, I was teasing him as we were prepping for this session that in the last couple of months, every time I open anything, it is Mike Howell talking about AI and ML. He tells us we are the end of that tour, so we are really in for a treat as he promised to share with us that he will give us the most important thing specialty societies need to know about AI and ML. I will also add that he is also one of us. He is a pulmonologist, critical care doc, part of our community, and with that, I am delighted to turn it over to Dr. Mike Howell. Thank you. Thank you. Pardon? Probably not. It's the vertically challenge. There's a box behind here. The danger is that we may one day solve AI, but it's not clear we will solve AV. Oh, it came up. That's amazing. Okay. So, I'm going to try to tell you some things I've learned over the past six years and change about AI in healthcare. The 2023 part is very important. If we were having this conversation just one year ago, it would be a totally different kind of conversation, and I'll try to share some of that with you. The important disclosure on here is that I work at Google, which is both why you have me here and a meaningful conflict. I also get some royalties from a textbook and other things like that. So I'm a little bit of a weird hire at Google, so between 1995 and 2017, I was either training or practicing. I ran parts of health systems, mostly focused on quality and safety, but a few other things here and there. I did a bunch of research, and I was talking with my new friends who focus on quality also. My hands are not completely clean of quality measures. I did things like co-chair a technical expert panel for Medicare on a few of them. And then in October of 2017, I moved to Google, and I was pretty early there. Things like they didn't know how to hire doctors, but there wasn't a job ladder, so I got hired as an engineering research scientist, and I got put in this group that was called Brain, and Brain turns out to be the group that invents new kinds of artificial intelligence for Google to use, and so my first few weeks, I would go around and say, oh, what do you do? And someone would say, oh, I'm a brain researcher, and I would be like, oh, you're a neurologist, and they would say, no, I invent new kinds of math. But what it meant was that when I didn't understand something, I could often grab someone who invented it and take them in a room and spend an hour with a dry erase board and learn some things. So I'll try and share a few things that I've learned with you. My job today is that I'm the chief clinical officer, which means I have teams of docs and nurses and psychologists and health economists or two who embed with the teams that make a lot of things focused on health at Google. I suspect that we have shared values, but I've learned over time that it's better just to say some values and make sure that they're shared. And so the first is that health care delivery is really, really, really complex. That means that providing care and receiving care is really, really, really hard, and that that complexity means that our intent doesn't always translate to our results. And you know this if you've ever gotten care or helped a family member get care. You know that it's cold, impersonal, fragmented, and filled with waiting. If I were giving grand rounds, I would ask medical students, before you came to medical school, had you ever seen a fax machine? And they would say, no. And I would say, now, do you believe, those of you who have done clinicals, do you believe that it's a life-saving piece of medical technology without which the health system would collapse? And they would all say yes about that. This lived experience, which we've all had, translates into the numbers. Whether it is that globally 8 million people a year die who, if they had access to things that we already know work, would survive. Whether it's that we spend more than $4 trillion in the U.S. and $9 trillion globally on health and yet at least one in five of our fellow Americans can't afford a health care expense. Or whether it's that in the U.S. health care is the largest employer of any segment and yet we graduated three full classes of medical schools to fill the chairs of physicians who died by suicide last year. There's an immense opportunity for improvement. And it can be difficult to hold the opportunity for improvement in your head at the same time as we realize that health care today is also the best the world has ever seen. But I think all of us look around and the reason that you're here is that you see the chance for improvement and you want to work on it. And so we've been asking this question of whether technologies like AI and ML could help with part of that. And it's worth starting with what does it look like in the rest of our lives. And so I'm going to show you two examples of AI literally on my kitchen table. And the first of these is that moving in California has been great, but moving west of the Rockies, the bugs are different. And so I was eating one time and this thing crawled up beside me and I took a picture of it because I didn't know if I anaphylaxed and died or was envenomated. I wanted to be able to show it to the ER doc because I didn't know what I was looking at. But I took a picture of it. Now, the sharp-eyed among you will recognize that this is Google Photos, which you would expect me to show, but it's running on an iPhone, which my Android colleagues get annoyed about sometimes. And I'm just going to hit this button and it's going to look at that and it's going to tell me that it's a western tussock moth, which I can then go and find out that I'm not about to die in the next hour or 24 hours from the western tussock moth. It's pretty amazing that that just works and runs. The second piece is my undergraduate work. I was an Asian studies major with my thesis was on structural changes in landscape poetry from 0 to 300 A.D. and how they were mirrored in structural governmental changes as the Eastern Han fell apart into the Three Kingdoms. But I was not very good at language, and you can tell that by the kind of books I have laying around. And now, again, this is an iPhone. It's actually running locally. And I'm going to play this video twice for you. I want you to just look at it for a minute. It's just Google Translate running on an iPhone. And so what does it do? It predicts which pixels are language. It uses a sequence-to-sequence model to translate them. It predicts the font, the color of the text. It predicts the color of the text. So what does it do? It predicts which pixels are language. It uses a sequence-to-sequence model to translate them. It predicts the font, the color, and which pixels to interpolate so that you can just see that, you know, car is chicha and train is huacha. If you'd asked me 10 years ago would we have that, I would say yes, but I would definitely be retired. And now it's just in your pocket, like many things. And so it is clear that AI makes new things feasible. And the interesting question is whether AI could make new things feasible in healthcare. We have used AI in a ton of health-related product launches over the past five years. The ones that are shown here are just ones that were big enough that we put out a blog or they got press coverage or other things. So we have experience with this. And I'm not going to show you anything like most of them, but I will show you three. The first is about protein folding. So, again, I'm an Asian studies major. I can barely spell DNA, but I can recognize that protein folding is a hard problem and that it's important. And why is it important? It's because if a protein is shaped in one way, it causes cancer, and shaped in another way doesn't cause cancer. Shaped in one way, it's an effective treatment for disease. Shaped in another way, it's toxic or causes lymphangiomyomatosis. Any ATS folks on the ground? A little pulmonary joke there. But it does, right? Yeah, LAM, pronounced LAM. So I can recognize that it's important. I can recognize that it's difficult because the 2017 Nobel Prize was awarded for improvements in the 3D measurement of protein structure. And I can recognize that it's difficult because there's a named paradox, which is called Leventhal's paradox. If you take an average length protein and look at how many different ways you could fold it up, and you do that pretty fast, checking out 1,000 confirmations per second, so that's pretty fast, it would take you longer than the observed age of the universe to get through all the possible confirmations of a protein. It's a pretty hard problem. Team called DeepMind inside of Google solved this as a math problem, inventing some new kinds of AI. And the reason we know they solved it and were able to publish in Science and other places is because every two years there's a competition where people have painstakingly figured out the actual structure of proteins, which historically it's one PhD dissertation to one protein structure, four to five years of work. And they save them up and don't tell people, and then they run a competition. So we know that it worked. Protein folding, hard problem. But it's not just basic science. There are 110 billion views of health videos a year on YouTube. 110 billion. So 1,000 seconds ago, Helen was talking, a million seconds ago was 12 days ago, a billion seconds ago George Herbert Walker Bush was president, and 110 billion seconds ago was about 1,000 years before the founding of Rome. It's an unbelievable number, there are only 8 billion people on the planet. We've been working to lift up high quality authoritative health information because patients and people, before they become patients, want this information driven at the beginning by work from the National Academy, shown at the top of YouTube for complex things. And then if you look over here, very transparent ways that we show people why something is credible. This is from AAP in the room, American Academy of Pediatrics has been an amazing partner, has done terrific work. This is a recent video about self-harm on a channel lifted up and shown to be credible. And the reason that this is an AI problem is figuring out what people are asking is a tricky problem. But we also know that groups like hospitals or the NIH don't always make the most credible videos. And so CMSS led work to describe what are the criteria by which you would determine if an individual creator was credible or not. And we've been able to operationalize that, again, showing from a doctor licensed in the United States. This is one of my favorite channels. But one of the things is we seek to engage our communities, both our professional communities and our patient communities. I want to show you is this one video about, I don't know if IDSA is here, about an ID doc going to a fish market. I learned a lot about fish-borne diseases from this video. Look at the engagement with this video. Thousand people talking about it to each other. But it's not just medical facts. People also want to know how do I live with cancer, how do I live with diabetes? And so we've worked to algorithmically identify which videos tell personal stories about how do I live with cancer and then how do we tell which ones have good content that is consistent with medical consensus so we show them at the top. Very difficult AI problem. On search, hundreds of millions of people come to us every day and ask questions about their health. If I ask this group how many people have gone to the doctor and then afterwards gone home and Googled something to double check your doctor, many people would raise their hands. We take that responsibility very seriously and we work really hard to give good answers to this. And I'll tell you my favorite statistic I've learned since I've come to Google that will explain why this is an extraordinarily difficult AI problem. Every day, today, 15% of what gets typed into Google.com as a search will never in human history have been typed in before. It sounds almost unbelievable but language is a very high dimensional space and it turns out it's new all the time for us. And we have to be able to give good answers to those. So I'm often asked, you know, why is Google doing work in health? We don't have a choice. People come and ask us questions about their health hundreds of millions of times a day and we work really hard to do well at it. But in the past year, past two years, we've seen a turning point in AI's capability. These are real examples where you type in a sentence and you get something that looks for all the world like a photo that has never existed before. This one is my favorite. Teddy bear swimming, the 400 meter butterfly event. It's remarkable. And so there have been so here's the first key point that I want you to know. There have been three epics of artificial intelligence and each of them have different risks and different opportunities. The first which I'm calling AI 1.0 is symbolic AI or it's sometimes in the scientific literature I promise, it's called GOFI for good old fashioned AI. And this is if then rules, branching logic, encoded human knowledge. Think of IBM's Deep Blue beating the world champion in chess or think of clinical pathways embedded in EHR with branching logic would be an example of this. In about 2011, there were major breakthroughs in back propagation and an architecture called the convolutional network. And we saw sort of all of these, this is the era of deep learning, AI 2.0. This does one thing at a time. Is this a cat or is it a dog? One thing at a time. We saw all of these consumer advances. Our teams have done work in diabetic retinopathy, optical coherence tomography, lung cancer screening, breast cancer screening, DNA variant calling, pathology and on and on and on. We published 150 papers in this area. Our sister company has figured out how to do a self-driving car using this technology. But beginning last year, we saw this new category of AI. It's got several names and I'll say them, I may flip back and forth. Generative AI, foundation models, large language models. These can do many different things without being retrained on a new data set. So do a cat versus dog and then decide you want to do diabetic retinopathy, gotta go get a different data set, retrain it, test the model, all these things. How many people here have personally used generative AI in a substantial way? Okay, so fair number, but most people haven't. So I'm gonna show you an example of this. Come on video. And this is me using voice recognition. So it gets a couple of the words wrong cuz there's a lot of noise when I was doing this. I meant to say what should leaders in medical specialty societies know about generative AI? Okay, leaders should know, it actually answers the question I was trying to ask. It could be used for diagnosis and treatment and research. You should educate your members, all these things. Now, I know that there are very senior leaders here, and I've worked with CEOs before. And so I was like, this is a little long for presidents and CEOs that I've worked with. Could you summarize it maybe in 75 words? And it says, sure, and also apparently it knows that CEOs like bullets. So here are three bullets about things to be doing. Now let's play with it a little bit. And let's write that again as a poem. Remember, my thesis was on landscape poetry. This is not terrible, right? And so now let's be a little more serious. Let's assume that, potential's so vast. Medical societies guide you at last, I like that. Now let's assume that a specialty society decides to do a pilot of Gen AI to reduce burnout in its physicians. Write an internal marketing plan to support the rollout. Cuz all of us who've done QI know that you often fail because you don't do internal marketing. And so it writes a marketing plan. This is a pure consumer product, by the way. And it's got nice evaluation metrics. I like, as a healthcare delivery scientist, I like evaluation metrics. And then I know that programs need acronyms, especially if cardiologists are involved. And so I'm like, can you give me an acronym for this? I made this for you this morning, two hours ago. And okay, I don't really think that's that good. So, can you try again? Supporting AI for vitality and engagement, burnout lessening innovations for support and satisfaction, healthier experiences in AI. So the point of that is not that it can write poetry or that it can give you acronyms for your program, which I was always terrible at it. But it's amazing doing it. The point is not that. The point is that I just showed you five or six totally different problems. It did them without retraining or a new data set. It talked to you, it wrote in a way that sounded like it was from a person. That's a new category of AI. It's not just an incremental change from AI 2.0. And so there's seven things that I think leaders of medical professional societies should know, and I'll try and tell you what they are. The first is that this is a true technological step change in capability. You would never have heard me come here and say the words block and chain in the same sentence. This is a different thing. It's moving very fast. I have never in my career seen anything move this quickly. What do I mean when I say step changing capability? It can interpret very complex questions. I just showed you examples of that. It understands context, especially in language. It can take text and pictures and sound as inputs. Gives answers that are plausible, sound like they're from people. It can create words and images. Some can do videos and even music. And one of the really important things as we think about health literacy is that it can adapt the way it's talking. So you can say write this like it's for a dissertation, now write it like it's for a sixth grader. And it does a great job at those things. These things have obvious applicability to healthcare. But when I think about designing things that could exist in the world, I think about the capabilities rather than the specific model. I want to give you an example of what I mean by speed. So our team has worked on a medically tuned large language model called MedPalm. Published in Nature. I often get asked why do we publish in journals like Nature and JAMA? And I can assure you that it is not that journal editors move at the speed to which Google is accustomed. It's because we think it's important to show our work. And this is one way to show work. So there are four time periods, all within the past year. And I'm going to show you the differences between a December paper and a May paper. So about five months, not quite five months. And then the nature paper is roughly the results from here. And I'll show you some work from August. So all of this is December, May, August, three, four, five months apart. And this is all in the category of answering medical questions. So medical question answering has long been held out as a grand challenge for artificial intelligence. And questions like those on licensing exams are a good example of this. This is one of the questions from the data set that was used in this paper. It's not the easiest question in the world, right? When you look through it, it's a challenging question. The nice thing about this data set, which is called MedQA, and it's similar to the USMLE. People have been working on it for a number of years. They say that the pass rate is about 60%, meaning that if an average physician who just passed a USMLE took it, they would get about 60%. And whether it's exactly 60% or 65% or 55%, I don't know. This is what people say. People have been working on this year over year over year, and they've been getting better 3%, 5% at a time until November-ish of 2022. So this time last year, the best in the world was 50%. This is what happened. MedPalm in December was across the line at clearly 67%, and by May had increased to 86% on questions like that. Now, passing a multiple choice test is one thing, impossible to do in this time last year. That's one thing. But the more interesting thing is the following. So we took about 1,000 questions that real people ask about their health. We know that they ask them because they come to search and type them in very commonly. Then we open source that so any set of researchers could use them. So there are things like, can incontinence be cured? Can a CT scan detect diverticulitis? People ask these things of Google. We took those. We gave them to the model. And we said, hey, model, write an answer like you're answering a patient. And then we hired a bunch of physicians. And we said, physicians, write an answer like you're answering a patient. And then we took those, and we gave them to another physician blinded and said, which one is better on several dimensions, dimensions like alignment with medical consensus? Is it likely to be biased? If somebody followed this, would it hurt them? And in December of 2020, to 11 months ago, blinded physicians preferred answers written by other physicians on most dimensions by a little bit. Let me show you the results from May. So these are the dimensions. Better reflects medical consensus, better reasoning. Here, higher is better. So let's do reflects medical consensus. Here's the answer of the model. Here's the answer of physicians. Model, better reasoning, physicians. Knowledge, recall, model, physicians. Now, if we go over here where higher is worse, we see evidence of bias, physician, model, greater likelihood of harm, physician, model, and on with the one that's different of more irrelevant info from the model. So on eight of nine dimensions, blinded physicians prefer the long form answers written by the model overwhelmingly compared to those written by other physicians. In August, the team figured out how to add things in like chest X-rays and EKGs and spirograms. It's a technically difficult thing to do, but it lets you do things like have a conversation with a chest X-ray. This is an example from the paper. Is pulmonary edema present in this image? Yes. Where is pulmonary edema present? Both sides. Is it, how bad is it? Mild. This one is a little hard to follow because it's a QI example. It gives the radiologist interpretation, stable appearance of right-sided pleural effusion pneumothorax is resolved. Is the report wrong? Yes, the report's wrong. There's a right-sided pneumothorax. It has not resolved. So having a conversation with a chest X-ray is a demonstration for what multimodal AI is. Now, this looks fast, right? Remember, this is less than a year ago where it just getting to the 50 to 60% mark and now we're at 86%. I don't even know, I feel like this, to have a conversation with a chest X-ray thing is a different axis. So that's thing number one is, this is a true step-changing capability and it's moving fast. Remember, nothing else remember that. Thing number two is that people will quickly come to expect this way of interacting with you and your members. We're seeing this already. How often are people searching for AI? This is four years ago, three years ago, two years ago. Last year, more than doubled. This year, sustained eight-fold increase. So it's not just patients and families. It's also going to be your employees, your clinicians and your learners. So be thinking about customers and workforce because they will need different solutions. That's thing number two. Thing number three is it's not just a science project, it's being used today. Here, we're using it on a number of internal platforms for consumers. Healthcare enterprises are using it. These are announcements in the past few months from HCA, Hackensack Meridian, ICAD, which does breast cancer screening, Bayer, which is using it for drug discovery, Meditech, which is using it for summarization. A couple of startups using it for different things, including payer interactions. Two that we're really proud of. One is, you may not recognize it, but this is the president of El Salvador talking about work with three pillars, one of which is around transforming healthcare using AI. And this is a group that is doing, building a new category of foundation model, not starting on language, but starting on DNA and proteins. So thing number three is it's not a science project. It's getting used today. It's very quickly moving. Thing number four is that not all AI is the same. Very different upsides, very different risks. So let's go back to AI 1.0, 2.0, 3.0. AI 1.0, if I said, can you write some rules to distinguish a puppy from a muffin? You would say, sure, puppy has four legs. Muffin has zero legs. Puppy has two ears. Muffin has zero ears. Awesome. Now you'd think this is a stupid example until I show you this. This is a hard problem, it turns out. In the deep learning era, 2011 until now, you have some pictures of puppies and muffins and you just label them, puppy, muffin, puppy, muffin. And it gets good at telling that. And you give it a new one and you say, is it a puppy or a muffin? And it says 99% muffin, 1% chance that it's a puppy. Generative AI, this new category of AI, you say, go read this huge stack of documents, then come back and at least explain the difference between a puppy and a muffin. And so we're gonna go a little deep on how to do math on words because it will show you all the vulnerabilities of this new kind of AI. And so there's this really clever idea of take all the words in a set of documents and throw them into a space. And when I say that, I mean like imagine each dot here is a word. And let's assume we can get them in the right place for a second. Got that picture in your head of a bunch of words in a 3D space? Now in the model, it's like 500 dimensions or something, but I can think in three. Okay, so now the words are in the right space and you start to notice some things like engineer and engineers are close to each other and happy and joyful are close to each other in this space, as it turns out. And when this is done well, you find things like if you find Moscow and then go find Russia, you know the distance and direction between them. And then you find Tokyo and Japan, same distance and direction. Washington DC and United States, same distance and direction. And so you can do analogies in this way. Turns out to work for drug and disease and other things like that. It also becomes clear that you can just represent words as numbers, right? Because they just have an X, a Y, and a Z. So that part is really important. Throw them in a space. They've got an X, Y, and Z. You can do math on them. So how do we get them in the right place? Well, now that you know how to represent words as numbers, you can do equations on them, just like if you're doing a linear regression because it's just a few numbers. And so we're gonna take a stack of documents or whatever, and we're gonna show each word in sequence. And the model is gonna guess what the next one is, and it's gonna get it wrong. And when it gets it wrong, it's gonna move those words around a little bit, and it's gonna come back around again and predict, and it'll get it wrong, and it'll move them around and move them around and move them around until it misses the least. We're gonna try and do this. I'm gonna give you the word the, and I'm gonna ask you to predict the next word. And I will just tell you that's a stupid question to ask you. Okay, but now if I say the quick, what's the next word? Brown. Brown, okay. Oh, I heard, brown I heard most, but what was number two? Somebody said another word. When? Rabbit. Rabbit. So I made up these percentages, but you would, you know, 5% of the time brown. It turns out and is common because of the phrase the quick and the dead. Rabbit will be in here the next time I give this talk, for sure. Okay, now if I say the quick brown. Fox. Okay, great. I don't know, maybe there's some others. Yeah, everyone, you can tell who took a typing class, right? So here's the key thing. So when we tell a language model to go learn, they do what you just did, is they predict the next word, they build a representation in that space with all the things, and when they get it wrong, they just move it around so next time they're more likely to say fox. And they do that millions and millions and millions and millions of times until they get the words in the right place. This is called self supervised learning. It is a big deal. The papers underpinning what I just showed you have been cited 80,000 times. This is how it looks, right? This is a sentence from Crossing Quality Chasm. Given healthcare, it's gonna try and predict should. Given healthcare should, it will try and predict to be. All those things. So we're gonna, why did we look under the hood? That's how you train a large language model, is what we just did together. And the reason that we looked under the hood is it makes it obvious where it's gonna screw up. And you can see for yourself. Anybody heard of hallucination? Yeah, so this is where chatbots or large language models make things up. This New York Times article is about a popular consumer when making up references to New York Times articles that didn't exist. It'll do that for medical journal articles also. And you know why they do that. Because what are they doing? They're not looking things up in a database. They're predicting the next word in sequence. And so when they get to something, they're like, oh, this looks like it should be a medical journal article. Let me give you words that plausibly sound like a medical journal article. That's what they will do. Now, turns out that there are some significant technical advances that have already happened. So that when you go to one of these things and ask it, they can often now cite their sources. This is an example of one of them on search, where it'll say, what are common causes for long ED length of stay? And if you see this little down arrow thing, it will cite its sources so you can go verify it. Sometimes if you say, oh, this looks like it should be a PubMed citation, just go look it up in PubMed. They turn out, these are a bunch of stories about how AI is terrible at math. And it turns out that they're terrible at math. They're good at two plus two, but they're terrible at things like 13,247 plus 17,289. And you know why they're bad at math. Are they doing math? No, they're not doing math. They're predicting the next word. And so there are lots of examples on the internet of two plus two equals four. So when they hop around that embedding space, they jump to a whole bunch of fours. But there aren't lots of examples of five digit addition on the internet. And say, oh, this is a five digit number plus a five digit number. The most common thing that happens next is a five digit number. And so they'll just give you a five digit number that's plausible. Because they're predicting the next word. They're not doing math. Now, there are a bunch of technical innovations. There's a paper called Toolformer from Facebook, where you identify that it's trying to do a math problem, and then say, ask a calculator and give that result. Don't remember out of your magic embedding space. They're biased. You know why this is a huge risk. The models learn the structure, language, and relationships from the text that they're trained on. The internet has lots of dark corners. Even if you filtered out the dark corners, and we had 100% agreement on what's a dark corner and what's not, our language is still full of bias in the way the language itself is constructed. They predict the next words from what they've seen. There's another issue here, is that people who are able to contribute and have an opportunity to contribute to things written on the internet differ. So if you live in a part of the world where you never in your life have access to the internet, the chance that you write something on the internet that the model may learn from is zero. And people creating the tools usually don't represent the people using the tools. So there are huge areas of bias to be worked out here. This has been a giant focus of ours for many years. In 2018, we put out a thing, this is our CEO, more than five years ago, about our AI principles. And to be honest, people looked at us funny at that time. They're like, well, okay, you can tell cat versus dog. Why are you so worried? Well, it was because we had just invented a kind of AI called a transformer. That's the T in GPT. And we could see around the corner of what was coming. We've been working specifically on fairness and equity for a number of years. We took the very unusual step in 2018, five years, next month will be five years, publishing in Annel's fairly in-depth paper about this. And we have a health equity team led by Ivor Horn, who I suspect many people know here as a national leader in health equity, focused on this work. But it's a really important risk. So all AI is not the same. The risks differ across the kinds of AI. But these capabilities offer a huge opportunity to improve patient and provider experience, to improve patient outcomes, and to reduce the marginal cost per interaction. I think we have a chance to improve health on a planetary scale if we don't screw it up. If you wanna read one thing about the places people are thinking about using this, there's an actually quite good McKinsey report from July talking about a number of use cases in healthcare. And their assessment is about a trillion dollar opportunity primarily in the US on this. But they're quite concrete in what it might be used for. I often get asked how I think healthcare organizations will prioritize. And I think that there's a spectrum between ordering paper clips and sewing an aortic valve into a human being's heart. And that people start on this side of things. But it is a mistake to think that things that we label as administrative don't have patient outcome implications. So anyone who believes length of stay in a hospital is a purely financial problem has never done CPR in an ER waiting room. Anybody here besides me done CPR in an ER waiting room? Yeah. And why was it? It was because the hospital was full, because you couldn't get patients out, so the ER was full, so the waiting room was full. There are a million examples of that. So I think that people start, what we're seeing is people are starting on the paper clip side of things. But thinking about things like how do I improve shift change, how do I improve documentation, all of those things I expect that we'll see things migrate toward the direct clinical care over time. Number six is that this technology is new. It has risks and unknowns, like hallucinations, bias, and then I'll say use by adversarial actors. There's a lot of fraud out there. There are a lot of groups that actively promote things that societies in this room have called misinformation. And adversarial action, rather than just well-meaning misinformation, is a different category of problem. It's disinformation instead of misinformation. This will need technical approaches to deal with it. It will need policy approaches. It will need regulatory approaches to deal with it. Our position is that AI is too important to not regulate and regulate well. We've called specifically for AI regulation in this space. We think it's really important. Again, it's a huge area of focus for us, and you should be thinking about AI governance today if you're not already. Governing something like this has different issues than governing a regular IT system. The last thing is that not everything needs AI. There's huge value in data science and business intelligence and predictive AI, AI 2.0. And technology will not help many things. So this is a picture of my mom. 2019, my mom was diagnosed with gastric cancer. It was at the gastroesophageal junction. It's a terrible place to have it. You can't eat even while you're getting evaluated. She had a feeding tube very early. I was down helping my dad with something, and this thing on the right was the tray of things that my dad used to take care of my mom's feeding tube. AI is not gonna help with that. Technology is not gonna help with that. Huge amounts of care need to be in-person, physical, together. That will always be true, at least for the lifetime of everyone in this room. But AI will help with many things. And so even in things that technology will help with, you don't need AI for everything. And so what do I mean by that? I mean that for health systems or healthcare providers, maybe for specialty societies, that there's a huge advantage to data organization, harmonization, availability, and governance. I'm pretty convinced that most data in US healthcare that go into a server, go into there never to be heard or seen from again. And just making them available to people in a way that they can use it with low latency changes the way that people have conversations. The second part is regular old business intelligence and reporting, that a huge amount of value happens just from asking questions of your data. And there are many kinds of modern tooling that instead of needing to be a CEO of a $10 billion health system with a data science team, and if you're that, you can send off four people to think for a week and come back with your report. Many of the kinds of tools that exist today, those can go all the way down to the frontline manager, and they can make decisions with real data and making high quality inferences. The next thing is seeing the future and predictive modeling. So things that would have taken us 40 Google engineers and 18 months to do in 2018, you can basically get off the shelf and they work today. This is pretty settled science at this point, has all the regular problems that if you implemented a logistic regression it would have, but all the super difficult things that used to require very specialized engineers, they're a solved problem today. And so places where being able to do prediction or classification is useful, can just build that on top of things. And then I do recommend that you start getting experience with generative AI. For people who've used it, it's quite different than what came before it. And there's nothing like getting your hands dirty to begin to understand where it's strong, where it's weak, and to be creative about how people may use it in the future. But not everything takes AI. Thank you.

Dr. Michael Howell

artificial intelligence

machine learning

healthcare

risks

regulation

patient experience

technology