Hi, everybody. I'm Helen Burstyn, the CEO of the Council of Medical Specialist Societies, and thank you again for joining us, another one of our webinar series on registry science and research. We're really excited to have this webinar today, specifically focused on sustainability and member engagement in clinical registries, their evolution, as well as areas around clinical research. We've got an extraordinary panel, as you will hear. I'm really excited to kick it off to our two panel leaders, Greg Martin, who is the past president of the Society of Critical Care Medicine and a professor of pulmonary critical care, allergy, and sleep medicine at Emory University, and Krista Singleton, the CMO of the Office of Policy Analytics and Population Health at the Centers for Disease Control and Prevention. So Greg and Krista will introduce the panel, and looking forward to a great discussion. Thank you. Thanks, Helen. So thank you for the introduction, and yes, I'm Greg Martin. So I'm a physician at Emory University in Atlanta, and I'll start off by offering a grand welcome to everyone. So thank you for joining us today for what will be a great program as part of the CMSS series on registry science. Before we get to our first speakers, let me just briefly set the stage for today's webinar. We have three outstanding speakers with complementary perspectives on clinical registries, each with different experiences in launching and sustaining those registries that have served important member needs. And so you're all probably familiar with clinical data registries that can be vitally important, and we have seen them be quite important in times of urgency, as they were in gathering evidence and guiding treatments during COVID-19, and also for long-term assessments and longitudinal quality improvement activities. And most often, perhaps most importantly, they start with the clinical need, a knowledge gap, a need, or an opportunity to better understand diseases in patient care in a way that often informs optimal clinical practice. So today, with that in mind, we'll hear several different aspects of clinical registries, everything from creating registries for quality and practice improvement, and creating a virtuous feedback loop, or a virtuous cycle that involves member engagement and scientific discovery outcomes and practice improvement, through the things like the why, what, and how of registries, and how they're created, run, sustained over time, meeting member needs, and then also developing registries as a key tool for gathering real-world evidence, and particularly spanning both federal partners and things like the coordinated research networks, or registry networks, that can assess the state of evidence and help guide learning opportunities along the way. So with that as a prelude, let me hand it over to Krista Singleton, my co-host from the CDC. Hi, great, great. Thank you, Greg, and welcome to everyone. We're excited that you are here with us today, and just extremely excited to hear from these panelists. I am representing the Centers for Disease Control and Prevention for this engagement, and public health departments have been connected with registries for quite a long time, and we have had space in things such as birth defects and cancer care, antibiotic stewardships, and working with critical access hospitals, and as time has evolved with different data elements, data standards, CDC is leaning forward in that space, and one of the things that we are excited to partner with CMSS to better learn how, Greg mentioned about how clinical data registries can gather patient data, but one of the fascinating components that we have leaned into recently is looking at both the health and cost implications for work, and one of our speakers is going to touch on, and I'd love to hear more about, the role of registries, how to inform some of the potential payment models for clinicians, and how they can support that effort, and we are going to also hear opportunities for sustainability of these registries, and I love looking forward to the dialogue between our panelists to help us think this through. So, with that, our first speaker will be Dr. Flora Lum, who is currently the Vice President for Quality and Data Science at the American College of Ophthalmology, and she is going to give us a conversation around sustainability from that particular specialty society's perspective. So, Dr. Lum, over to you. Thank you, Dr. Singleton, for the introduction, and thank you to Dr. Burstyn and CMSS for this opportunity to present. I do not have any financial disclosures except for being an employee of the American Academy of Ophthalmology. Next slide, please. The IRIS Registry, which stands for Intelligent Research Insight, is the nation's first comprehensive eye disease clinical database. The number one goal was to help ophthalmologists improve care delivery and patient outcomes. It does this by providing individual feedback on performance and comparison to national benchmarks, and it helps practices meet the merit-based incentive payment system requirements. Next slide, please. As of July 2022, a total of 18,020 physicians, which are the ophthalmologists and the eligible clinicians in their practices, from 4,147 practices participate either manually or via their EHR. Of those, 15,799 physicians from 3,002 practices are contracted for EHR integration, and since the start in 2014, the total number of patient visits is 454 million, representing 75.4 million unique patients in the U.S. Next slide, please. In the literature, it has been shown that quality does improve with actionable feedback and targeted education. We also demonstrated this improvement across a broad set of quality measures, including a measure for achieving 2040 visual acuity within 90 days after uncomplicated cataract surgery over three years' time in this publication. Next slide, please. One of the initial goals of the IRIS Registry is quality improvement, and we do that via a dashboard, which is refreshed on a monthly basis. This provides a way for physicians to evaluate their own data across as many as 50 quality measures, with the majority being outcome measures. It also provides two benchmarks, one being the prior year CMS benchmark that's for everyone who reports on the measure, which could include many different physician specialties, and the other is a current year benchmark that includes all ophthalmology practices in the IRIS Registry. The dashboard provides an easy way to help physicians focus on which measures require the most attention, because it's difficult to work on all 50 measures at the same time. Green means that they exceed the IRIS Registry current year benchmark. Orange means that they're below but close to the IRIS Registry benchmark, and red means that they are a little bit significantly below the IRIS Registry benchmark. Next slide, please. The dashboard views also let the practices look across the entire practice by practice site or location and by individual physician, because quality and care processes can vary, and thus these differences and outcomes can be measured and then managed by the practice. Physicians can also look at patient listing of everyone who doesn't meet the success criteria for a quality measure and be able to look back in the medical record to see why the patient outcome or follow-up was not optimum, and then go through a discovery process to see if there's changes in processes or techniques that can be made going forward to improve care. This is the actionable feedback, the ability to drill down and see which patients did or did not meet the measure criteria for success, and with enough time over the reporting year period to enhance performance and correct, take corrective actions. Next slide, please. We also provide targeted education. Here in our magazine, next, thank you, here in our magazine, we gathered tips from high-performing practices, oh, sorry, back to the slide before, sorry, we gathered tips from high-performing practices about monitoring performance, working as a team from the office staff to the physician, and adding checklists and reminders for patient follow-up to be able to share with all other practices. Next slide, please. On our educational website, we provide targeted learning plans that are built upon the IRIS Registry quality measures, and these refresh physicians on subjects like preventing complications, assessing risk of disease progression, and optimizing use of medications. Next slide, please. Another major approach to engage our clinicians is having them participate in something bigger than themselves, the potential to advance scientific discovery. Dr. David Park, our former CEO, put forward a vision from the start that all ophthalmologists could participate in, shortening the timeline for dissemination of important clinical knowledge, research, and results of drug and device surveillance. Dr. Stephen McLeod, our current CEO, has an even greater vision for expanding the application of real-world data and evidence, and building clinical trials on top of registry platforms. Next slide, please. So, to date, we have had 59 peer-reviewed publications about the IRIS Registry, big data analyses in various journals. Next slide, please. And we have had 174 presentations about the IRIS Registry findings in meetings in the U.S., as well as around the world. Next slide, please. And in October 2022, we also announced a research network that IRIS Registry practices can join to use the data that's already collected to identify patients that meet inclusion and exclusion criteria of clinical trials, and match them to the appropriate clinical trial sponsors. This reduces the work that patients have to do to find patients, which is the top issue for adding to the timeline and cost of conducting clinical trials. Next slide, please. And last but not least, the IRIS Registry has been the main MIPS reporting tool for ophthalmologists during this time period. This has yielded ophthalmologists a higher average score than the average MIPS participant, and the majority of ophthalmologists have earned an exceptional performance bonus, which actually hasn't really amounted to a lot of money, but it's in the range of about $1.5 million. So, this is a very high-yielding tool for ophthalmologists, about $1,608 to $28,764 per ophthalmologist over this time period. Next slide, please. Altogether, the IRIS Registry has permitted all the ophthalmologists reporting through it to avoid a cumulative $1.2 billion in penalties that otherwise could have amounted, which averages to about $118,962 per ophthalmologist. So, when physicians put in the time to review and enhance their quality performance, there is a tangible return on investment. Next slide, please. In conclusion, as we look at factors contributing to clinician engagement in registries, the number one reason is to improve quality of care and evaluate practice patterns. Benchmarks allow physicians to evaluate patient care processes and outcomes against most other ophthalmic practices in the United States. Feedback. Physicians receive actionable, timely information that allow them to make changes to enhance patient follow-up screening, education, and outcomes. Targeted education. We provide access to learning plans that address how to improve diagnosis and treatment of patients. Another important reason for participation is that they can do their part to contribute to scientific advancement. Big data analyses add to what we know about real-world practice patterns and outcomes, and answer important clinical questions that might be too costly or too time-intensive to launch clinical trials. Clinical trials use data that's already been collected in the IRES registry to facilitate patient recruitment and collect patient information both faster and cheaper than traditional means. And the quality payment program registries, our registry has saved ophthalmologists over $1 billion in avoided penalties and helped them to attain an additional tens of millions of dollars in bonuses over the five-year time of NIPS. Thank you so much. Thank you very much, Flora. That was really fantastic and a great overview of the clinician side of registries and where the value may lie. So just to make sure everyone's aware, we do have a period at the end where we'll be able to do some Q&A and some discussion. So if you have questions along the way, please go ahead and put them into the Q&A function in Zoom there at the bottom. You should see that. You can put them there and we'll get to them where you can let us know when we get to the end. Our next speaker is going to talk about it a bit more from the society perspective. So Vishakha Kumar is the Director for Research and Quality at the Society of Critical Care Medicine, and she's going to talk about sustainability, particularly the why, what, and how of clinical registries. Vishakha, thank you very much for being here. Good afternoon, everyone, and thank you, Dr. Burstyn, CMS's staff, and the CMS's Registry Science and Research Initiative for this opportunity, and thank you to our session leaders, Dr. Martin and Singleton, and my co-panelists today. Next. I'm Vishakha Kumar. I'm the Director for Research and Quality at the Society of Critical Care Medicine. My disclosures today related to this presentation is that I am a co-principal investigator for the Virus Registry, which in the past has been funded by the Gordon and Betty Moore Foundation, Janssen RND, LLC, and the ASPE FDA. Next. Thank you. My presentation today is focused on the why, what, and the how of registry sustainability and member engagement, and this is from a society perspective. Next. A little background about SCCM. The Society of Critical Care Medicine is the largest nonprofit medical association dedicated to promoting excellence and consistency in the practice of critical care. We have more than 17,000 members in 100 plus countries. Our mission is to secure the highest quality care for all critically ill and injured patients, and we envision a future where multi-professional teams use knowledge, technology, and compassion to provide timely, effective, safe, efficient, and equitable patient-centered care. Next. So, let's start with the why. So, in 2017, SCCM started Discovery, its critical care research network, to establish the infrastructure to support investigator-initiated research studies and create a network for research collaborations. We received early programmatic successes. However, we realized that significant resources are required with data collection for research studies. Specifically, during the pandemic, we learned a few lessons along the way that helped us better strategize to support investigator and members with sustained and continuous engagement. Next. So, I'm going to use the virus registry as the case study for the presentation today. The Discovery Vital Infections and Respiratory Illness Universal Study is a cross-sectional observational study and a registry of all eligible adult and pediatric patients who were admitted to a hospital since January of 2020. It was one of the early registries that were deployed for COVID-19 during the pandemic. In the first two months of the registry, we had more than 1,500 in-hospital subjects entered globally. But we quickly realized that the manual data entry was a very resource-heavy ask for participating sites. So, from funding support from the Gordon Betty Moore Foundation, we initiated a pilot of 25 sites to locally automate the data to be uploaded to the registry. Now, this effort was called a data automation practical EMR export pathways. Later on, we received additional support from Janssen where we were able to expand this local automation effort as well as provide some support to the manual data entry sites as well. Today, the registry stands at about 90,000 in-patient subjects enrolled for data collected. That accounts to at least 350 in-patient hospital days for more than 200 data variables on these hospital days. The registry is still ongoing until April of 2023, beyond which we plan to expand it to other diseases as well. Next. So, what did the registry do for the members? Next. So, right from the beginning, it helped engage a core group of investigators to drive adult and pediatric scientific content from the registry. They helped lead discussions on the operational, the clinical questions from the registry, developing and leading grant proposals, and so on. Next. Globally, we had investigators from more than 28 countries involved. They helped us create a scientific community of more than 3,000 members and non-members volunteers globally. Now, these individuals helped really drive the utilization of the registry data for the scientific and the academic purposes. Next. Thanks to our automation work, we were able to create some local data automation leaders. Next. And lastly, collaborations. This has played a very big role in the overall success of the registry. So, over the last two years, SCCM collaborated with many organizations, including academic medical centers such as Mayo Clinic, Boston Universities, societies such as the American College of Radiologies, other registries such as AHA, Cardio COVID-19 Registry, EHR folks like informatics and technology groups, and also looking beyond hospitals with healthcare software providers, helping support long-term and acute care facilities such as PointClickCare. Next. And lastly, the publication of the projects. So, for the investigator group, this is also one of the ways that they measured success with the registry. Next. So, we started with some early publications important to the scientific community. Now, during the pandemic, SCCM had twofold focus here. One was to get the scientific community to get the scientific content out to the clinical and the research community in a timely manner, and secondly was to leverage the engagement of members to help build the bench for future clinical investigators. Now, many of these manuscripts published from the virus registry during the pandemic included a mix of early, mid, and late career investigators, and also tried to focus on multiple disciplines to move the publication in various different clinical areas as well. Next. Now, many of these early publications provided timely insights to clinicians about ICU and hospital care patterns and trends in COVID-19 patients. It also helped us to engage the education SCCM was able to provide through its COVID-19 Rapid Resource Center. Next. One of our main commitments here was to maintain the investigator engagement, was making sure to provide collaborative authorship opportunities to all those who were involved. So, we developed a publication guidance and policies and also processes for all manuscripts that were published using this registry. Next. We created a number of member-facing tools such as dashboards, data explorers, educational contents. We created an online platform for investigators to submit their research ideas based on registry data. Over a period of about 18 months, we had more than 130 research proposal ideas submitted out of which we had about 68 ancillary studies that were approved. Now, we also created data-sharing infrastructure which focused on the legal, the policy, authorship guidance, and also data access and the analytic support needed for these studies. Now, apart from this, we also developed some ancillary projects. Two of them that we've highlighted here. One is a quality improvement project called Astop Virus that assess the effectiveness of virtual coaching for bedside clinicians. And a multicenter qualitative study on facilitators and barriers to implementing new critical care practices during COVID-19. Next. So the result of all of this effort, we had more than 30 publications in less than 24 months, comprising of journals focusing on various clinical and non-clinical areas with more than a thousand collaborative co-authors on many of these manuscripts. Next. So from here, we thought, how do we align and sustain such an effort? So we thought about what our core challenges were here. Next. Number one, this was a volunteer-driven effort. So that makes it very difficult for us to sustain such an effort over a long period of time. Next. With data entry, this is a very resource-heavy ask at many institutions. So that was one of our biggest sort of concerns that were brought forward by many of our sites. Next. At times, we had limitation on institutional data that could be shared. Next. And we all are aware of this burnout, right? It has impacted at all levels at so many organizations. At all levels at so many organizations globally. Next. And lastly, keeping the registry up to date is so relevant when it comes to scientific output. Next. So our goal here was not to reinvent the wheel here. However, we want to leverage the collaborations that we have built. So for example, here within the virus registry, we collaborated with the American College of Radiology who helps support the collection of imaging data for the virus sites through their triad system for the clinical data that we were collecting on our registry. Next. We provided a mentorship platform through our investigator network. This gave opportunities to many to collaborate on studies, grants, new initiatives, educational and academic activities. Next. So for SCCM, this effort gave us an opportunity to expand our own infrastructure. And we developed a data science campaign focusing on data automation for research purposes and also on standardizing and harmonizing critical care data elements. So SCCM rolled out a data science campaign in April of 2022. And this is actually built on the infrastructure that we've created from this registry and the discovery network. Next. We leveraged this collaboration with the virus sites to create the pool for future collaborations. One such example is the SCCM's collaboration with the FDA CureID CPAT and its collaborative partners for focusing on aggregating and analyzing COVID-19 treatments from EHRs and registries globally. Now for us, this program is an extension of the virus registry, really helped tap the pool of members and sites that were already engaged. To be involved in a more global data automation project using a standardized data model. Next. And lastly, we really wanted to build on this registry. So to build on the registry and data infrastructure, our goal is eventually to help support the adaptive platform clinical trials in the future. Next. So in summary, for continuous engagement, for continuous engagement and sustainability of registries, we've learned that the why, what and how are really driven by what the organizational and programmatic strategies that are implemented. And for deploying such strategies, it is very important to have a value proposition to those that are involved at all levels, whether it's within your organization or outside of your organization. And there should be a clear guidance around the key performance metrics and that you're open to more broader collaboration. Next. That is it. Thank you everyone for your time. And we'll take questions at the end. Off to you, Krista. Great, thank you so much. That's quite fascinating. I, hopefully someone will have questions, but I definitely have some great ones and at least some ideas or thoughts after this one. So following the next speaker will be Dr. Gregory Pappas, who is currently serving as the Associate Director for National Surveillance at the FDA Center for Biologics Evaluation and Research. So turn it over to Greg Pappas. Thanks so much, Krista. It's a pleasure to be here today and thank you all for this opportunity. Next slide, please. I am going to speak broadly. My comments do not represent FDA policy or policies of any FDA centers. I also have no financial disclosures. Thank you. Next slide. I'm going to start with some policy context and the development of Coordinated Registry Network, CRN, which is a term of art that I'm going to be focusing on, but I'm going to really work with you and be focusing on, but I'm going to really whiz through this. You've heard all of this from my colleague, Dr. Marina Dobich, who is online, may be able to answer some of the questions as we go forward. The slides are as a reference. There are a lot of embedded links here for the listeners, but what I'm really going to focus on is the research and development agenda for Coordinated Registry Networks, how to determine value of real-world evidence and how to build a national infrastructure. Next slide, please. There has been a 10-year, over 10-year development of this notion of Coordinated Registry Networks in the medical device space. Next slide. And receiving lots of federal support from a variety of federal agencies. Next slide, please. I'm going to go quickly through a lot of this again because you've seen it. While the effort to look at real-world evidence started early in device space, 21st century cures gave it a good push for medicines, for pharmaceuticals and for biologics. 21st century cures gave the FDA authority and funding to evaluate real-world evidence. Next slide, please. And I'm going to spend a little time here because sometimes I think this gets lost. The real understanding of real-world evidence sometimes I think this gets lost. The real underlying purpose of a lot of 21st century cures from the FDA. FDA's perspective, has been moving towards a new pre-post-market balance. We've been, next slide please, we've been using this effort through a number of techniques, through expanded access pathways through risk-benefit analysis to do less on the pre-market side and get things to market faster and evaluate on the post-market side more strongly. Next slide please. What real-world evidence really potentially enables us to do is to do even less on the pre-market side and really look to the post-market as our source of a strong evaluation. But of course it necessitates having high-quality, robust post-market data sources in place and that's what the CRN, the Coordinated Registry Network, has represented in the device space. Next slide please. The Coordinated Registry Network is the, as I say, the term of art and you've heard about these as resources that have developed in multiple surgical subspecialty areas, bringing together a wide variety of clinical data, electronic data, and coordinated by a learning community. We have about 300 publications documenting this development. Next slide please. The business model for the CRN and really something I'm going to focus quite a bit on is collect, it's real simple, hard to do but real simple, collect once, use many times. And most of you who are working in registries understand this implicitly. There's been a lot of important work on quality assurance, benchmarking, support for training, research development. Most recently, support for using the registries for regulatory decision-making, post-approval studies, label expansions, compliance studies, signal detection for the FDA, and national coverage decisions for CMS have been important. Of course MIPS, I should have put on this also. Next slide. And this business model makes this a more sustainable resource because you've got funds coming in from multiple sources. Next slide. Let me just go through it. This will be very familiar to most of you. Using the Vascular Quality Initiative, a registry that includes about a third of all vascular surgeons, and they collect information typically, the typical stuff, a cohort of patients and exposure. Next slide. And then you link those to claims data and research literature on this. And voila, you've got an epidemiological study, exposure, a cohort exposure, and an outcome. Next slide. And the CRN, a mature CRN, then that's kind of the core of the CRN. Then add to that patient report outcomes, EHR data, data from wearables, data that come directly out of devices, mortality statistics. You've got a very rich data set. Next slide. And this exists in a mature CRN, and the one I'm going to highlight here is the vision of the Vascular Quality Initiative. And you can read more about that at your leisure. Next slide. There are about 15 device-focused registries currently in operation and three international. Next slide. And the, what has emerged is a maturity model and these 15 or so activities can be placed along a maturity model and have helped the through this community practice lessons learned to be shared and guiding the development of, of this, of these coordinated registry networks. This paper is not out yet any day now. This should be available electronically at BMJ. Next slide please. So this is all stuff you've heard before up to this point I'm going to shift a little bit away from the device and talk about biologics. There are a number of biological products in the white in the, in the, the pipeline that are going to be used by the surgeons that are using plastic and metal now vascular surgeons cardiology surgeons, and they're going to be recording the use of these, these new products in their, in their registries, creating a really potentially a very, very potentially useful resource for biological evaluation. Next slide please. Dr. Marks our director of the center is highly supportive of this way. We also have a new registry of being stood up by the American Society of hematology focusing first on multiple myeloma myeloma and sickle cell anemia, this is both a platform trial and a post market registry that includes the 30 academic centers that have human hematology departments next slide. Very exciting activity. It's really quite mature already. And I'm going to shift to what I think is the, the, the speculative or my perspective on this, my reading of 21st century cures, is that FDA is, you know, should evaluate RWE using the criteria of better, faster, cheaper. If, if this isn't why we're met, we're interested in RWE I don't know why, and then we need to measure this, this is, this is the work that I have been focused on for the last couple years next slide. And we have three papers of that have been completed, three case studies, the first one on vascular devices second TBT vast. I'm sorry, cardiac devices the TBT transplant transcatheter valve therapy then vascular surgery was published. We got one in press on hernia surgery. Next slide please. The results of some of that study, the return on investment of using the registry. This is for industry so we did a case study of the TBT transcatheter valve therapy, and three companies three device manufacturers, use the registry to produce studies 23 decisions by the FDA supported a 23 decisions at the FDA, and the pink bar there is what the registries, what the registry received that what the companies paid for those services for those studies, and the blue bar is what it would have cost. We had done a traditional study you see substantial saving 500% over $100 million saving for those 23 studies 500 over 500% return on investment. Next slide. We also looked at time saved and interesting from the, from the industry perspective it's probably more important I didn't present the results here, but years and savings, and this is a under underlined the important of time to time to market is, is a critical important maybe more important than what it costs to produce the actual study. Next slide please. So it's the better in the mantra of better faster cheaper that I as a public health physician of most most interested in, I think, first of all the post approval studies. It's important to understand that those studies tend to be rather small, even in the drug and biological space they could be as little as 250 people, and you really can't get a clear understanding of the effect of the heterogeneity of effect, how the smaller race ethnic age sex subgroups experience these, these interventions, of course when you have registries with thousands hundreds of thousands of cases, we're really able to answer some of these questions in a much more effective way including looking at operators in context of of sites, which all can be very important as you all know very well. Registries also provide an understanding of the full complexity of the patient the multiple conditions and treatment that they are undergoing something that RTC systematically avoid. So, it's better on that ground. Finally, it's better because you actually get the data we FDA, all the product areas have halls of shame where studies, never get done. Some of the studies the post market studies are not requirements of things that we encourage them, studies, companies to do. And the fact that these studies actually get done is a better criteria. Next slide please. This is the real pivotal slide in the presentation. I see and my colleagues, see that after a decade of exploration of real world evidence of the various centers biologics devices drugs have all taken a slightly different approach, but we're all beginning to converge around the reality that is the registries the coordinated registries that really are have been successful. We've tried a lot of things, and these have been successful device space I've mentioned, there's a documentation of 90 decisions that have been used to support regulatory decisions and 70% of them are are coordinated registry networks in the cyber and the biologic space I've mentioned some of that work in oncological center for excellence and cancer therapy, the eyes by platform trial is paving way for very similar sort of insights. And finally, in drugs. The paper that really rebooted real evidence after the pandemic we went through well with the pandemic and other priorities but concata and great. Corgan Craig published a paper about what's working and they highlighted registries also point to the guidance on registries at the use of registries is real world evidence and the subcommittee. Next slide. That a sub subcommittee that addresses the issue. So if these are active areas of inquiry. I think that this convergence leads to a research agenda that supports all registries and there's a research agenda in two areas that I'd like to touch on briefly versus the determination of value and second is building this infrastructure. Next slide. And the, the research that I've done that I presented on the determinative determination of value focuses on value for medical device sponsors that manufacturers. However, all the stakeholders gain value the patients clinicians their society, the regulatory agencies and public health and and what we need is a Is a framework to study all this value. Next slide. So working on a paper. I've got a group that's guiding guiding me and and some investigators at MD EpiNet and we're organizing the value propositions of the registry by stakeholders and you see that that done here. Next slide. We're looking at the various buckets of Key performance indicators, the metrics and the case studies as better, faster, cheaper. Next slide. And then arranging under those categories, the The metrics and the case studies and I've got them organized here. You can look at these. I don't want to take up too much of your time on the details of this. Next slide. I want to say something about the Next slide. Do we have the next slide coming I want to say something about the research agenda. Now, to increase the value and the use of the CRN, they need to mature. But we need research and development for this community and all of the very impressive work we've heard today to mature. There's a research agenda, I think, that will support all of the activities that we've heard about today. Next slide. Looking across the board, I see a need for knowledge management, the improvement of curation methods, linkages for both mortality statistics and other sorts of data. We need the provision of all pairs claims data. Over 65 represents all claims. We can effectively use Medicare data to link patients over 65, but we don't have anything equivalent for under 65. There is some legislation and research and funding for development, although this is a major activity that needs a lot of support, and then, of course, methods. Next slide. I'm going to drill down on that second category, improving curation methods because of the focus of this discussion today. The financial sustainability of the registries depends, in part, on decreasing the burden for collecting data and curating data by the clinicians. There are a lot of research efforts going on, diverse ones. I don't think any one of them is going to solve the problem, but combinations of these may address particular registries. We've got efforts to upstream harmonization of standards to address the messy data issue, the semantic interoperability issue. We've got semi-automated curation tools. The whole e-source movement is at play here. There are efforts to redesign clinical workflows to make the data collection seamless and more accurate. Finally, natural language processing is also being used. Next slide. These are some activities that are going on on that one bucket, but as I say, we could do a whole conference on a research agenda. I'm going to finish by saying that anybody can aggregate data, but I think that the specialty societies are the right strategic choice for investment because they're part of the learning healthcare system. It's not just about the FDA. It's about a learning healthcare system, improving patient care through evidence-based medicine, through the implementation of clinical decision support, improving efficiencies of larger systems, and contributing to the larger scientific questions that we are faced with. Next slide. So with that, I want to thank everybody and looking forward to the discussion. Thank you for indulging me in my speculation. Thanks, Greg. That was a great presentation. I guess we are now moving on to the Q&A portion, and we have two questions in the chat. Looks like the first one is for Dr. Lum, and they are asking, the requestor is asking, how are the QI efforts associated with the IRIS registry funded? Do clinicians pay to participate, or is that included in what they pay for the registry? So the academy does not charge any fees for participation. That's one reason I think that we do have such a high rate of participation. It's included, or it's part of their member benefits. But of course, they do have to be, we do ask that all ophthalmologists in the practice have to be academy members and good standing to participate. So our original efforts have been all funded by the academy's foundation. So we have a nonprofit foundation that funded all the creation, initiation, and sustains our registry going forward in terms of the quality improvement efforts. And I have a follow-up question with that one. How has your society, I guess, compared, or have you looked at how other societies have funded theirs? I saw a report looking, I think, at IDSA, the Infectious Disease Group, is looking at the pros and cons of funding them. And they looked at the American College of Rheumatology. They looked at the Allergy Society, I guess, plastic surgeons using member dues. I was just curious of your all's perspective compared to other specialty societies for funding. Sure. So when we launched this in 2014, I think we did look at different models. And of course, yeah, different societies do have different approaches for revenue and support. And I think we did have the benefit of having a foundation with sufficient resources to launch it. And we thought it was important to amass a large proportion of our membership quickly and to reach that goal. Within two years, I think we already had about 6,000 of our ophthalmologists. And now we have a majority, like 2 thirds, on EHRs contributing to our database. So we just thought that was an important imperative for the success of our registry. So we did put that forward. And our entire board of trustees approved it as coming from our internal funding, just to get that on a faster track. But I know that every society has different considerations and limitations as well. And I think this question is also for you. What have your members said is the biggest value for participating in the registry? So I do think the MIPS, obviously, is a very tangible ROI. I mean, we estimate, yeah, every year it's about $28,000 in penalties, because ophthalmologists do participate and are heavily involved, eligible. The majority are eligible for MIPS. So that is a tangible ROI. But I do think they are excited about contributing. Obviously, like Specialty Society, I think they trust the Academy. So they trust the Academy with the data. They trust the Academy to do the right thing with the data. And I think advancing science is in everybody's interest. And so to the collective interest, I think they do feel some participation and some reward in contributing to that advancement. Great. Thank you. And I wonder, to me, a little bit of what you're describing is what I was thinking of as sort of this virtuous cycle that you're creating something that has benefits, and people are engaged in doing it, and it sort of is sustained because they want to be involved and they get something out of it. And I wonder maybe, Vishakha, those two questions I can also see being applied to the registry work that SCCM has done as well. So I wonder if you might give us your insights about funding of registries and sustenance and those things as well. Thank you, Greg. I think, again, it's twofold in a way. I think sustainability, when you're talking about the financial sustainability, it really is SCCM is driven by our own infrastructure and SCCM's own funding to that. However, we are always open to looking at funding through various funding mechanisms as well to help support that extension beyond whatever is that core that we've developed from a registry perspective. And the second piece of it is also from a membership perspective. Right now, we are looking at the value of what are the scientific contributions that a registry is making. And I think that's what drives a lot of our effort from a sustainability perspective. However, I see that changing as we are going beyond this initial two-year period of COVID. And we have a focus on what is easier for members to participate and what is a faster mechanism that someone wants to be able to participate, whether it's a research study or whether it's a registry. So I think we see a trend that's changing beyond that. How is this going to be sustained? Is it just the infrastructure that an organization provides or also its collaborations and beyond that as well with members and other organizations and so forth? And I see another question in the chat. And I saw also one in the chat maybe for Greg. But let me ask the Q&A question first. It's for Dr. Lum. So the question is about funding the development and integration into EHR systems. And what are the challenges there? And broadening the use of EHRs as incoming sources for the registry versus, for example, manual data submission and extraction? Sure. So actually, in the late 90s, we launched our first EHR registry. And I always think we learned from that because it failed. And it failed miserably because we were relying on practices manually entering data. So that was kind of one of our mantras was we had to make the data entry barrier low. We had to make it automated. The practices don't have to do anything. We extract their record and we put it into a common data model. We extract information from over 60 different EHRs. And so all that work has been done through our IRIS registry vendors. So that's what we've undertaken. So we do have manual people just to give them a way to participate in MIPS. But obviously, their data doesn't contribute at all really into our database that's used for big data analyses and for clinical trial identification. That is interesting. And it seems like that's a common theme for many registries now is that you'd certainly like to have electronic tools and make it as the barrier low, but really almost to make it completely opaque to the person on the other end so that they're not contributing any major energy to making that work where it becomes a significant barrier. So Greg Pappas, there's a question that you talked about in the chat a little bit about the 21st Century Cures Act and working with ONC and CMS to think about future versions of the core data interoperability piece. I don't know if you want to talk about that. And you're muted. Yeah, sure. Good question. Thanks very much. There are a number of things other than the US core data that need to be addressed. I mean, data is very messy. The amount of curation takes a long time. And there are a lot of things that need to be addressed in the US core data. Curation takes a long time and very, very expensive. Even with the semi-automated excellent activity that we've been hearing about. FDA is working with ONC and CMS on a number of issues on addressing this. And we need your support on that. FDA can't do it alone. And we need to collaborate very broadly with the ecosystem on these problems. Lab is one thing that's a huge problem. I mean, you'd think after 40 years of having lab digitalized and standards well, lots of standards, we've got ample standards, mature standards, and CLIA, you'd think that lab data would be interoperable. It's the worst. It's a major mess. And there's a major initiative at the FDA working with ONC and other partners to begin to do that. The American College of Pathology, by the way, taking on that issue. Greg, I can add a couple of things about devices. I'm here just to support Greg Pappas. But just to give an example of what FDA is doing specifically to US CDI related to the capture of the device related sort of data classes and elements. The goal is really to expand the scope of devices beyond implantable medical devices that are subject to device identification using the unique device identifier. As you can imagine, it's really very difficult to study device specific adverse events or performance or any type of evidence generation in the context of medical devices without having the UDI embedded in a number of data sources. So we have been working very closely with the Office of the National Coordinators to expand the scope of the devices to focus on expanding device data elements beyond unique device identifier aligned with also other initiatives to drive the adoption of device information in clinical health information systems. And obviously, including all the other work that we are doing in the context of public private partnerships specifically funded by the Assistant Secretary of Planning and Evaluation at the FDA to the core trust fund effort. For example, some of the things that new US CDI device elements that we certainly are supporting or promoting would be the device type and classification, the device setting, the device route, site and place, body site and anatomical locations, which have not been in the previous version, device observation, measurements, and things like that. Just to give you a sense that it's really, I would say, a lot of details that are involved with this. And at the same time, as this US CDI version is moving forward with the comments incorporated from other stakeholders, I also would like to say that under the research agenda and research and development, we have a lot of really very specific collaborations with ONC staff and leadership as part of a coordinated registry network, the core trust funded effort, in which both ONC and the National Library of Medicine have been partners in this. Just to, again, highlight what Greg had mentioned before, that FDA really cannot be doing this alone. And we are just one link in this chain of the larger ecosystem. And ONC's role is absolutely instrumental in moving this forward. Thank you for that. Great. So we have another question that has come through, wondering if all of the panelists can provide some insights for how registries can change use cases. For example, how can registries shift from a primary focus on MIPS to include broader uses, including research? How can you get your members to embrace a research use case? So Krista, that was me. So in particular, I'd love to hear. I think a lot of our registries really started out being lots of different starting points. But for many of them, it was really built around MIPS, as Flora talked about. Love to understand as you try to make a transition or think about adding new use cases to what you've already done, how do you get your members engaged? And specifically, how do you get your members engaged in the research piece? I mean, Vishaka and Greg know how much work clinicians put into putting in their virus data. How much of that was because of the pandemic and how much of that is because they recognize this as a real opportunity for your society to engage in research? Well, I actually have another one to add on to that. One of my thoughts around using the registries to better help us inform how different social determinants of health can be identified. Greg talked about getting that aggregation of elements by race, ethnicity, data, rather than having to go patient by patient, hospital by hospital. Now you've got this collective piece here that you could tease out. So I'm very interested in how the panel thinks about a research use case, and maybe even moving in the social determinants elements. That's a great one. Flora or Vishaka, just from your, I'm sorry, just because they've got specific ones that I think made that journey. Can we start with them, Greg? So Flora, thanks. Sure. So for the IRIS registry, we have always kind of made it about quality improvement. So actually, I have this conversation with practices very frequently. They don't have to report myths because they're part of an advanced payment model or an accountable care organization. But what we do is we stress the quality improvement angle because they have no other tool to be able to compare their providers. And they can use that performance to other payers, even if they don't have to participate in MIPS. They can use it for maintenance of certification. So we've worked with our American Board of Ophthalmology to make participation in IRIS registry count for quality improvement in the maintenance of certification part four. And as well as the research, I do think they do feel some pride in the IRIS registry and being able to see publications come out, see changes in practice patterns, or hopefully, FDA devices and drugs being able to expand the label or have other regulatory decision-making based on faster, as we said, faster, better, cheaper real-world data. Yeah. Vishakha or Greg, perspectives from virus or what you learned? I think, I mean, everybody knows the virus registry really started as a research registry to get the content out there. When we were very early in the pandemic, we saw that a lot of involvement revolve around contributing to the scientific community. But as we moved along the pandemic and more recently, we are seeing that shift in focus now towards moving into more of an automation area where you don't have to be participating on a daily basis in order to participate. However, it would be like setting up nodes around the country to be activated when you are in another pandemic. So with our data automation work now, we are utilizing collaboration with Johns Hopkins and with Emory to focus on implementing a freely available tool into a width academy and community hospitals to engage them in delivering that data on a more automated basis and trying to deliver it as you come through projects. So it doesn't really have to be just COVID-19. If you want to do research, if you want to do sepsis study, you could utilize that mechanism to study sepsis. You can do it for ARDS, and it could go beyond that. So I think now we see a trend where investigators, researchers, members are looking more to get involved where participation is easier and less time-consuming than what it used to be in the past. So our focus very much on platform trials and so forth as the new or emerging interest, I should say. And I'll just say that, I'll just underscore that by saying that there's a potential virtuous cycle if we could take the money that's being generated by research and plug back into the process so, again, as Dr. Kumar was saying, creating ways that make it just easier. Yeah, yeah. Greg M., did you have any insights to share? Yeah, I mean, not a lot because Vishak led this. But I think when you look back at the pandemic, it did open some doors or windows for doing things differently, certainly more quickly because of the urgency and the need, and often a bit of altruism. Everyone wanted to understand what they were doing and what they should be doing. And clearly, that was key to the virus registry because it very quickly stood up and allowed people to share data and begin learning from it because it was transparent and online data sharing and so forth. I tend to think that there's some equation out there that helps people determine when they should decide to participate in something. And Flora gave us an example where there's MIPS dollars, right? There's real dollars that might come back to them, at least in some way, right? You can say, here's what you saved or what you received that you wouldn't have otherwise gotten. That's probably the most concrete and dollars tend to motivate people. But COVID obviously showed us that people will do it for other reasons. And I think probably this activation energy or the barrier has got to be one of the pieces in that equation too, that the lower you make it, then people can begin to equate and say, okay, well, I'm going to get this out of it, or I'm going to learn this. And this is how it's going to help me or my practice or my patients. And all those things together have to come into it. And so the environment and the cost and the barriers and what they're getting from it. And you'd like to think that it's not all dollars, but clearly dollars is one of them. And Flora's registry is a great example, but there are many others that real do generate real returns for people that help drive patient care and clinical practices. Yeah, that makes sense. I really like the concept of an activation barrier because you're right, but you get past that. I know that's been a lot of the work the registries have done so far. Crystal, I also really liked your question about SDOH. Have any of you built sort of abilities to look at SDOH and equity within the context of your registries? I know, Flora, you guys have thought a lot about natural history, for example, of some eye diseases by race and ethnicity. How difficult or easy has that been? Sure. So what we find across the EHRs is about 80% of patients have race and ethnicity. So we are really looking, I mean, we have that limitation that it's not totally complete. And of course, it's not really validated as well as whatever the practice or patient reports in the EHR. But we are doing, I think a majority of our studies now are looking at social determinants of health and disparities in care, access to care or treatment outcomes. And we have a specific emphasis. We have a few grant programs to interest our participants in research. And we do emphasize both from the researcher angle, the investigators that we will give some consideration to those of underrepresented minorities, as well as looking at topics of disparities in care. That's great. Vishvaka or Greg, other thoughts about that? So I'll just share that, you know, it was really not a focus for us when we started it, given that, you know, we were just getting into a pandemic. We learned very quickly that it should have been one of the factors that we've looked at. So we've made, you know, attempts to address that going forward, especially with our data automation work to make it more, more comprehensive to see how those factors could be used for future studies and so forth. Because I think it's not only the collecting, collection of race, ethnicity is the go-to where people are going now, which has been underrepresented and definitely needs a place. I would also offer, you know, looking at insurance status, access status, these are elements that because of the more we can automate it, I guess the key point, those who are entering the system, therefore there is an access issue or access point there, but just kind of better unpacking that piece. I think that's an exciting component of the registries as they get kind of clumped together. Greg, do you have anything else to add about the social determinants of health piece and gathering it and incorporating it into the registries? Oh, years ago, I published a paper on zip code and it's a very, very useful, robust, it's commonly in these data sets and it can be used as a proxy for lots of different, different social determinants of health. So it's something in this, in this arena might, might be worth looking at again. Yeah. And it's, I mean, geocoding generally is something that's really been helpful, I think, as a strategy and it's, it's getting better, right, with electronic tools and more granular data. So I think that is one way to think about that. We have another question in the Q and A about financial sustainability and maybe for all three of our panelists, the question is really about whether this, whether the registries you have are truly sustained financially, and if so, how long did it take to get there? And if not, do you have a plan to get there? So Flora, do you want to go first? It sounds like you've, you've had yours for quite a while and it sounds like there is, there is sustainability there. Sure. So as I said, we internally funded the development and then we did have a commercial partner. So we, we had actually originally done some commercial studies to be able to offset some of the costs of the IRIS registry, but we also have a commercial partner who does work with that. And so they do support some of our expenses, but it is an on, I would say it is a ongoing, expensive process and something that we feel is important to support. I mean, we hope to expand our research agenda. So we are getting grants from outside, but also non-commercial grants to help support it. But we do think that it's an important priority that the society will be supporting it to some extent going forward. So unfortunately I don't have an answer to when we will be totally self-sustained. Yeah. Vishaka? So I'll just start by saying we are not self-sustained at this time because we really, I think it's really what your, what your priority strategically is, right? Ours was very much research and, you know, scientific oriented. So for us really, it was the first step in trying to do something much bigger than what we have been in the area of research. So it, although it was supported from an infrastructure point of view by the society, we did actively look for grant funding on you know, to help support and expand the registry. We continue to do so even now with, especially with the data automation work. However, I think there needs to be more of a comprehensive vision around how it could meet with the organizational goals to really see if this could, this is something that would later be sustained by an organization or we can, or we need to continue to look at external fundings and so forth as well. So I don't think I have a direct answer, but this is where we are at this time. It is one of the challenges. I would turn the question around. These, in my perspective, I think these are sustainable activities. People pay for it. I mean, it's voluntary, you know, no one's forcing it. Obviously the people that pay for membership and some registries have very substantial membership fees because the hospitals want this information. They want the benchmarking. So I think that that's got to be the perspective. What value are you producing for who? And everyone pays a little. Unless we move to a European system, I guess in the international perspective, our European colleagues, the government pays for it, pays for the registries in Australia and Israel and other countries. And of course, in the context of the registry of the pandemic, we see, you know, that's where the data came from, post-market surveillance of vaccines. We didn't have one in the United States. We didn't have a source in the United States that was like Israel or Australia or UK or Germany, where most of the post-market surveillance of vaccines came from. So anyway, an international perspective also is useful here. And it sort of comes back to the purpose that my guess is people are, maybe sustainability is easier, but people are also probably more willing to pay for it if there's a clear return on investment, whether it's financial or not, but clearly something that's giving them something that they're getting a benefit from. Research, maybe I would guess, may be among the most challenging, right? Because you're generating education and maybe generalizable knowledge, but maybe you're not benefiting the individual practitioner as much. It changes a bit when you think about it through the health system lens. When I was head of quality measurement a very long time ago at the Brigham, we used to actually get a lot of benefit from the fact that we could report our incredibly low cardiac surgery rates. That's cardiac surgery mortality rates. So we actually were able to use some of those data, like from the STS registry, in a way that promoted our hospital, proudly hung my, you know, top 10 U.S. News and World Report sign out my window. And, you know, those are ways I think also, as we think more and more about what's happening in 21st century cures, and I saw Dan had a question about that, who moved towards this more interoperability. There's real opportunities I hear to make a closer alliance, I think, between what the health systems need and what the registries and research needs that I think could be interesting. I'd love people's thoughts about that. I'll just say that it's very, just from a society perspective, it's really driven by who your audience is, right? Because at this time, our audience is those clinicians and members who are out there. It's not necessarily the hospital or the health system that we are focusing on. I think when you shift your audience to involve the hospital and health system, that's when you will get some leverage from a sustainability perspective to get that sort of response to, you know, keep continuing to be engaged in such an effort. Yeah, I would tend to agree with Vishakha about that one, as I think as individual clinicians become more employed entities, and that health system will then shift to the audience. Because I was curious, I know at CDC, the section where I work, we work a lot with our colleagues over in infectious disease and antibiotic stewardship. And so they have been able to show that kind of audit feedback that you talked about, and that when you show the clinicians their data, and their performance, that we as physicians are very competitive, and you see where you lack, and then, oh, that's a change. I was curious, the feedback now you have from that document, when you could show the clinicians, here's my data, here's your piece, do they push back? Do they, just how they react to it? Yeah, I would agree that the physicians are, they want to do their best. They're, you know, we have physicians that always want to do better, and so, yeah, you give them the data, that's something that they can act upon. If they don't have the data, they can't, right, they can't manage it. And I was going to second Helen's position, too, is that, yeah, we, people in the IRIS registry, they can take their reports as, you know, you only have to do it once, as Greg said, you only have to collect the data once, you can give it to your hospital to show your quality performance to other payers, and, you know, that helps them save money, and they're, again, CMS-approved quality measures, so, you know, that gives another layer of validation. Great. I like this, right? Danica, just for Greg, to add one more thing from the devices space, in terms of the long-term performance for the participating patients in the registry, that can add immense value to the interest of stakeholders to get periodic updates of it. I can just give an example from BQI, Vascular Quality Initiative, who, for years, starting from 2012 to when it was established, all the way toward recently, provided updates to the participating sites and physicians on the short-term follow-up that was captured in the registry, but just last year, they start first after the linkage had been achieved with the CMS data, completely on all the patients in BQI, they started providing benchmarking and updates on the performance of the health technology on the long-term side, which actually generated immense interest and value and sort of additionally contributed to the sustainability of the registry itself. So, that might be another, you know, way for the registries to look at, which obviously is aligned with what Greg had presented very nicely in his introductory speech. Greg, I see your hand up if you want to add to that. I guess I'll just throw in something that maybe many people know. The Kaiser Registry is not self-sustaining. They make money. You know, it's a profit-generating activity. It's so valuable for the health system. So, I think, and I talked to other big registries that use hospital-based, you know, registries that have been looking at this, and while they haven't done the calculation, they find this the same thing. I mean, if you've got a health system that needs to be accountable, registries are the way to go. Thank you. That is helpful. I wonder, I mean, for each of our panelists, again, maybe we talked a little bit about registries serving multiple purposes and particularly getting to research as an example, but I wonder how often registries pivot from one purpose to another, either changing purpose, maybe starting with one purpose and either intentionally or by scope creep taking on another, and is that, you know, is there a foundation that people begin with and then it grows to be something else? And, like, I don't know, Flora, if your registry was always intended to be what it is now, or did it start somewhere else and then become that? And the same thing for Vishaka, if you're thinking about your registry, is there another plan that might layer on top of that or other things you might grow beyond that? Thanks, yeah. So, I always think of that we grew the IRIS registry kind of with a perfect storm. So, the perfect storm was the advent of MIPS, or prior to MIPS, needing more quality measures and having, you know, kind of that penalty becoming more significant. So, that definitely contributed to the introduction of our registry, as well as the advent of, you know, more EHR adoption. So, it's more EHR adoption, of course, for them we could make it more automated. So, definitely that was the foundation, and the scientific had to come after it because we didn't have enough data, obviously. We had to grow our database and then have the majority of ophthalmologists to be able to have, you know, a strong enough, I think, evidence base. So, it has evolved. And then now, I think clinical trials, you know, that also has real ROI because participants can gain payments from being part of it, and they can give their patients, you know, another avenue of potential treatment. Yeah. Plus, I would offer that the fact that if you could then get the patients in the, to get them in the trial, sometimes one of the barriers is, well, what types of patients look like me? So, now by sorting the register when you've got this larger, larger number of patients in there, that will actually, X, Y, Z has been used or implemented in people that are 55 and over and whatever. So, now that, oh, okay, I'm more confident because you now have data on me, because that's what we, some of the things we saw with COVID-19, particularly communities of color were like, where am I in this database? Oh, you don't have any data on me? So, I'll just wait. So, now with the registry, you've got that a lot faster. That's what I really like. I appreciate about Greg's, one of his slides showing the time to do the study shortens when you've got the registry. Yeah. I'll just add that there's definitely scope creep and it happens at all levels, right from the start to, you know, wherever you go. However, I think internally for us, we did create a very nice blueprint that we actually tested with the virus registry to help create ancillary projects instead of expanding, you know, your original scope of the registry that you outlined. So, one thing that, you know, one example I'll give is with the virus registry, there was an interest by a group to look at quality improvement within the registry. And we were able to create a cohort of subsites to get involved in the quality improvement initiative to expand and look at that specific area that the group was interested in. So, creating subprojects really helped us, you know, have a number of activities that align with the registry without really trying to, you know, expand the original scope. However, now we are at a point where we want to expand just beyond COVID-19. So, you know, the other conditions within critical care and how we are able to build on that registry, I think that is our, you know, next focus now. So, scope creep is always there. Sometimes it really works out well and helps you create that process or that plan for your future. Okay. Thank you for that. It's always good to learn those sort of aspects of it as well. I think we have one last question or coming to the end here. Could each of the panelists, we have a question. Are you able to share the approximate annual cost to support your registries? Whoever wants to jump in on that one. I would just say this is very much in tunnel to SCCM yet. However, you know, we certainly can, you know, share our details on an individual basis. If anyone's requested, we can certainly share how you're trying to manage and sustain this activity. I think for us, it's about 16,000 participants with the EHR. We also support manual for another 2,000. So, it is in the neighborhood of several million a year, just in operational costs and staffing costs. Perfect. And we all sort of anticipate that it would not be inexpensive, probably depends on exactly the scope and what the role or the purpose is. But it is one of the key questions, I suppose, to try and sustain it over time. So, I think I've really enjoyed this. This has been fantastic. And I want to thank all the panelists, but maybe Krista and I will hand it back to Helen and let her summarize what we've learned and get us to the next steps. And thank you to all of the panelists. Yeah. Thank you to certainly Krista and Greg for leading that discussion. Our panelists, we just learned so much about all the potential ways registries can really be used. And I think what's been great, just want to thank Heidi as well for her work in putting this whole series together. I think these have all been building on each other, looking at how we can use standards, looking at how you might be thinking about different approaches in terms of looking at devices or some of the use cases we talked about today. So, just really excited to see this just continue to build on it. And in fact, at our upcoming meeting next week, I'll ask Julia or Heidi to put, thank you, information on our meeting next week in DC on November 10th and 11th, we'll actually have several registry specific sessions. Of note, we'll have a session on sort of next generation of clinical registries led by Kathleen Hewitt, who many of us know have been part of this work group along with Krista and Greg. We'll also have a session on what's next around quality measurement. How do we get to a level of quality measures that are really useful for a wider variety of use cases for whatever these emergent registries are. And that'll include Cliff Coe from Bergen College of Surgeons, as well as Matt Nielsen from the American Urological Association and yours truly. So that'll be a great session. And then finally, we're going to actually present on some work that Greg's also been part of with us of a project that we've been working on for PCORI of looking at how patient led research can be integrated into our clinical registries and research and the collaborative effort we've been doing with the patient led research collaborative on long COVID. They're going to share some of the scorecards and ideas they have for how us and more in the research community can really think about how patients get engaged in these research questions from the immediate start, rather than sort of getting involved, asked to just be sitting on a panel, how they can actually help come up with the hypotheses and really be there from the very start of the research. So I think those sessions and many others really welcome you to join us. Feel free to take a look at the link. It's all open and we'll continue to do the work we've done, continuing the work on data standards going forward, and we'll have several more sessions until we hit the new year. So with that, thank you all. Hope some of you can join us next week. I know I think Flora's coming and Vishaka, and we're really looking forward to seeing you. And thanks again. Really excited to see this work continue. Lots to do to reimagine registries and how clinical research fits into the future. Thanks again. Bye everybody.

Member Engagement

Sustainability

clinical registries

COVID-19 pandemic

evidence gathering

clinical research

IRIS Registry

quality improvement

real-world evidence

Coordinated Registry Networks

data collection

strategic collaborations