I'm Kathleen Hewitt. I am with the American Society for Hematology, and I oversee the ASH Research Collaborative Data Hub Program and also our learning communities. And I'm excited about our session today. Joining me are three other individuals who run fairly well-established registries within their organization. And all four of us, and there are many other associations who do the same thing, but all four of us have worked together in different capacities and have shared the successes, the challenges, the pains, the cuss words, all of that above. So our goal here today is to talk a little bit about our programs and some of the things that seem to be going really well, some of the challenges, and what do we see in the future as we start to strategize going forward. I will just mention that with each of our programs, you will see that they all kind of structurally are intending to accomplish the same sort of thing, but they actually, in the way we conduct our programs, are somewhat different. So kind of take note and compare and contrast as we go through. So I'll start off and tell you a little bit about the ASH Research Collaborative. So the ASH organization decided to create the ASH Research Collaborative. So it's actually a separate 501c3 nonprofit organization, but the way the bylaws are written is that the function of ASHRC is to support ASH in its mission. So while it is a separate business entity, we really are structurally the same. We use the same executive committee. Of course, they have to close the books and open it back up. We also share all of the basic operation resources, such as HR, IT, policy, communications, marketing, and so forth. But we do have our own ASH Research Collaborative team, and you can see here on this slide that we're focused on collaborating, again, all things that are considered in toxicology, which is blood disease conditions. There are three main components to the ASH Research Collaborative. One is the Data Hub Program, same thing as a registry, we just have a different name. We also have a, for sickle cell disease, we have a clinical trials network and a learning community. The clinical trials network is a consortium of centers that are research-ready, and they have a lot of work. They actually, we fund them to have mechanisms in place to be able to conduct clinical trials. And it is about a total of 20 consortia, which has a clinical trial unit, a hub, with multiple spokes. So the spoke sites would be able to support research as well. And then the other is quality improvement, and that is focused in sickle cell disease only at this time, but it's like a national quality program for those of you that have those yourself. So the way in which we get data, for us, we are capturing real-world data, and we're using electronic health record data. And you're probably familiar with, you know, real-world data does represent many of these items right here. Again, we're focusing on the top one right now. But the concept is as that data comes in and we're working with a broader community and figuring out how to best analyze it, then it's used for research and quality improvement. So it's the same thing as enhancing clinical care, and you can see a few different examples of that. The other thing I wanted to share with you, again, and this is a part where you can compare and contrast across the presentations today, is for us, as we're focusing on electronic health record data, that's working fairly well for sickle cell disease. But for multiple myeloma, which is our other program, EHR data is not sufficient to address the important clinical concepts that are needed for meaningful research. So in that instance, with the sites that we're in the process of enrolling right now, we are pulling in local registry data and harmonizing it all together. The point of this slide here is that we've learned we have to be flexible. If our focus is pulling in real-world data to generate real-world evidence, we have to recognize where the data is and be ready to work with that. The other element to this is, as all of this data is coming in, which essentially is at the point-of-care data, so this is not redundant manual data entry, but it's entered, like the folks who are taking care of patients aren't thinking, oh, wait, let me document this because I know it's going to be used for a study, or let me document it because I know it's going to be used for quality improvement. So that's something that is part of the continuum at some point in time, is thinking about structured documentation. But for us, as that data is coming in, it goes into our data hub, and then we build an ECRF, a data collection tool, electronic case report form, whatever you want to call it, but it gets pre-populated based on the EHR data, and I'm going to share a little bit about how that works. That pre-populated data, then the site can sit there and look at it and validate the information. Does it accurately represent their patient? And if not, they fix it, or if there's something that's missing, they complete it. And whatever they do in the ECRF, that data becomes the source of truth. So when they submit data in the future, it does not override anything that was modified in this ECRF, because the ECRF is the source of truth. I'm bringing this up because CMSS just hosted a session with some colleagues from ONC talking about the U.S. CDI, this is United States Core Data Interoperability Data Elements, which have evolved a lot. These are the categories. What this means is that EHRs that are certified by the Office of National Core Data, EPIC, Cerner, all of the large majority of them, they have to structure and be able to organize these categories of data in the same way. So to create interoperability. This is critical. We didn't have this years ago. And U.S. CDI Version 1, Version 1.1, Version 2, Version 3, they are being updated just like anything else, and it does take time to roll them out. There are mechanisms of where you can modify and contribute into what those future core elements are going to be. So if you're thinking about your society, like for us, we're focused on hematology. These are categories, so there's a lot of information that falls within each. But what happens is that if we have another category or more details, you can actually get an add-on to it, but you have to work with a federal agency to do it right now. So make friends with your federal colleagues, because there might be an opportunity to work together to expand U.S. CDI. So I'm not going to go into details, okay, but I do think since we're all registry leaders, many of us are registry leaders, just how does it work? So you have this core data that's being captured and organized in a standardized way in the electronic health records. That data then, for us, is coming into our data hub using FHIR, which is a way to transact data, or OMOP, which is a way to organize it. Kathleen Hewitt's way of saying what OMOP is, is like when you go to the grocery store and you buy all your groceries and you come home and you put them out on the counter, you got to figure out where you're going to go put them, in your pantry or your cabinets. You know your cans go in this cabinet and your flour goes in this cabinet. That's what OMOP is. It's a way to organize one more time all of your data. And then it goes into the data hub. For FHIR, it's the same thing. It's that USCDI data, and within each of those categories are a ton of EHR codes, which are just a bunch of numbers that have titles to them, right? So all those codes transact through FHIR and can come to us. We ingest data in both mechanisms. And when that data gets into the data hub, then what do you do with it? Because it's just a bunch of numbers, right? So ICD-9, ICD-10, SNOMED, which are more like clinical attributes, LOINC, which is looking at labs, RXNORM data numbers are drugs. So as that data comes in, we have teams that work together and they say, okay, here are the 80 different codes across SNOMED, LOINC, et cetera, that equal this patient had heart failure. And we go through every single clinical concept that is important for us in our hematology patients, and we create those technical definitions that equal heart failure, that equal an EHR – I'm sorry, an ED visit, and so on. So there's a lot – this is like informatics and medicine are like peanut butter and chocolate. They come together and you need it all to figure this out. And then machine learning is something – we're not doing this just yet, but we are positioning ourselves to be able to use our data to figure things out in a way that eventually some of this could be automated. But that's a bit away. So the whole – here's an example. So here you can see pulmonary embolism, you know, what's the numerator, what's the denominator? This is like the clinical definition. But the actual phenotype – and we – computable phenotype, phenotype – there's so many different names. It's all the same thing. But the phenotype is, what are the characteristics that make up pulmonary embolism? And in this case, you can see here just a sample of the different types of codes, the numbers, and what basically it means, if a patient in your database has any one of these codes, they get a yes for pulmonary embolism on their electronic case report form. So that's how it works. So the other side of this is looking at the ECRF, and this is just a quick screenshot. And then for our sickle cell program, each of our – well, and multiple myeloma, which is that program we're enrolling our sites now for sickle cell. It's a little bit more advanced. But this is a picture of the portals that we offer each of our sites. So the dashboard, they can filter across the top to change their patient population. And there are a whole series of metrics across about seven different categories. The categories come across about right here. They can look at the numerator, denominator, and the phenotype if they want to know, like, hold on a second, how did you calculate this? They can click here and get that information. Click here, and they can see a list of patients. Click here, they can get a spreadsheet download, or they can get a picture of this to throw into a PowerPoint. And these little features were important as we were building our program, because we want people to, like, make it easy as much as possible to incorporate into their programs. The other area is data quality, like, how do you deal with data quality, EHR data, like, it's collected at the point of care. Well, the issue is that are the phenotypes correct? Are we pulling all those right little codes to be pulmonary embolism? Did the site actually submit the data structured in the way it was supposed to? Did they include the right patients? So there are still things that you have to address with data quality. We recently adopted, there's guidance coming out from FDA right now, looking at how do you use electronic health record data to support regulated research. They've doing a great job at putting some specifications together. And these domains here is what we've adopted for our data quality framework. So for each domain, we've identified what our goal is, what we want to achieve, and then all of the activities that we're doing to help accomplish those. This is a lot of work. It has to be one of the highest priorities for your programs, at least that's our belief. The other thing is with data quality reports, because it's EHR data, for our sickle cell program, it's all sample data, but you can see here, it'll show, this is what your value was this time, and here's how much it changed since the last time you submitted. So this can be an alert for the sites to be like, whoa, our admissions didn't go up by 300%. Let me dig into that, what's going on? And we're looking at this as well. So how are the data used? Many of us have data that's being used for nested studies, so you're collecting your data, but then you can do some deeper analysis with existing data, or even prospective studies, where you build on top of the programs. Decentralized, we're very soon going to be working on a project with a fairly large vaccine vendor who we're going to be, it's a nested study within our myeloma program, but decentralized, where the sites will tell us who they want enrolled in the program, and then we will be working directly using an app to get the patients consented and also capture PRO data. That will then come into our data hub and match up with their EHR data. Observational research is kind of, you know, it's the data that you collect and using it. Pandemic preparedness. With the decentralized studies and this whole infrastructure, if you're not already, probably many of you did use your programs to help address issues with COVID, and I know many of you actually built out programs very quickly, like we did, to try to get as much information as possible. And then, of course, for quality improvement locally, but then also nationally, working across the country. We are working very closely with many federal agencies in understanding how important they are. With HHS, they gave us a grant, a million dollars a year for two years to advance the growth of our data hub program for sickle cell disease and also build out the learning community. NHLBI, they put a lot of funds into research, so we're working closely with them to be able to, at some point, use our data for research that helps to address the issues that they are trying to accomplish, or at least address. FDA, we've been working with them extremely close because building out our system, one of the biggest priorities for us is to create a mechanism that's capturing real-world data that can be used for real-world evidence generation. We don't work with CMS right now, but many of you may be aware that there are new therapies coming down the pike, which are related to gene editing and genetic therapies, and that's probably an area that CMS, I'm guessing, we don't know for sure, but coverage with evidence development requirement, FDA may want to track these patients out to 15 years. So, that ability to capture longitudinal data will be important. There's a lot of recommended guidance coming out from FDA to industry. If you're not already, I strongly encourage you to track along with those because if you are working with industry for them to access your data, this is what they're going to be focusing on. So, the more you can be aware of this and strategically position your program in ways that works for you, but also might give you an opportunity to work and support industry, it will pay off. Our learning community, this is our national learning community. We piloted it. We haven't released it to all of our sickle cell sites just yet, but this probably looks familiar. You know, it's bringing together folks, focusing on what you want to improve, putting together resources, and then have everybody work together and to use your data hub data, your registry data to measure change over time. So, this is, we're actually working on this part right now. We've built the infrastructure over the last couple of years and having just the mechanism to be able to bring people together and build those change packages. And our sickle cell program is 39 sites. Some of you may know that I used to work at ACC and we have like over 4,400 contracts of sites around the world. I am so proud of this 39 sites. Let me tell you, sickle cell disease has been absent from attention from the medical community and so many, right? And there's a lot of resources going into it. There's about 100,000 patients in the U.S. living with sickle cell disease. It's a horrible disease. You hurt from the inside out. It's horrible. 11,000 patients is unheard of in having that available to use for research. And then just our lessons learned. So, informatics, I talked about that combination of the IT and the medical. That is like finding those resources is very, very challenging. Data quality, and for us, it's EHR data and teasing out all those issues. How are you going to pay for all of this, the revenue models? You got to have really good data quality. Mission versus money. Some of you guys, we started out like mission, but then how long is the mission going to be able to work? So, how do you accommodate being able to have more money, more mission? And then also, with the new HIT, I talked about USCDI, we talked about OMOP, we talked about keeping up on all of that and what does it mean, and having expertise involved. And then a few other things here, too, as you can see, you know, consent portals and contracts and federal agencies, so there's a lot of things happening. But I will mention that it's interesting to be part of a program starting today with all of the structures and standards and resources available today versus 20-some years ago, which I had the opportunity to do something similar before all of this was here. So my message is stay relevant, stay in tune, and stay connected to a lot of the folks who are making decisions that impact our programs. So with that, I'm going to wrap up and have our next speaker, which is, shoot, we did it in order. I think it's Bill. No, Bill's last. Oh. Yeah. Come on over. All right. We'll do it. So Nate Glues and Clint Campi, we actually used to work together some time ago as well. Oh, and I'm here. Yeah, we did. But now he oversees the Academy of Orthopedic Surgeons, and I appreciate you being here. Thank you. Thank you, Kathleen. Can I have the clicker, or I'm going to be stuck on my intro slide? Thank you. And thanks to Kathleen for putting this together. Anyone who's worked with Kathleen knows that she is a force of nature, and this would not have happened without her. I've learned a ton from you. So I'm very honored to be a part of this. Bill and Jim, you guys are true leaders. So it's an honor to be on this panel, and thanks to CMSS for putting this on. This is the AOS family of registries. Five years ago, we had one registry, our hip and knee registry, the AJRR. So we've been on quite a growth spurt across the last five years. We've added a shoulder and elbow registry, a musculoskeletal tumor registry, a fracture trauma registry, and in full collaboration with AANS, the neurosurgeons, the American Spine Registry. So I'll talk some about the challenges, but growth and sustainable growth has been a big issue for us, going from one to five across the last half decade. It's certainly added some gray hairs to my head and my tenure there. Here you're looking at a visualization of the participation in our registries. On the top left here, you can see the number of sites by state, and here you see the number of procedures. The AJRR is really the grandparent registry. It is by far the largest in terms of participation, as well as volume, but the American Spine Registry is really nipping at the heels. That's been our most successful launch. We've had a lot of uptake. I think much of that is due to the collaboration with AANS, as well as the need in the market for a registry that focuses on cervical and lumbar procedures. So pretty good national spread. We've done some studies now that the AJRR, our Hip and Knee Replacement Registry, appears to be pretty broadly representative. Not quite at the scale of what we had at ACC, but certainly at a level where we can begin to draw reasonable conclusions about what is going on in the country as a whole. Another look at the size and scale of our registries, let me see if I can, oh yes, I get the laser pointer. I mentioned launching new registries. The FTR, pardon me, the Fracture Trauma Registry is our newest. We only have 29 sites contracted, eight of them are close to submitting data. Data submission, getting up and running is a pain point, remains a pain point. Kathleen talked a lot about that, how do you enable that. Certainly something we focus on. Our Musculoskeletal Tumor Registry, we knew that would be a niche product. These are very dramatic interventions. They mostly happen in the big, big, big academic medical centers. Certainly a mission-driven registry for us, but it's one that's near and dear to my heart because one, we collaborated directly with the Musculoskeletal Tumor Society from the outset on this, very important in the orthopedic space, and two, this is an example of how something that would have been non-viable as a standalone, if you tried to spin it up and do the contracting and the technology platform and the feedback reporting, just wouldn't work if you tried to do that as a standalone. But when you nest it in a family of registries where we have those resources, have staff, have the technology, have the instruments, you can do something in a smaller space and have that be part of your portfolio. But again, you can see the AJRR, by far our largest product. The shoulder and elbow has been a little slow to launch, and I'll talk about some of the reasons why we think that is a little bit later. Going back to the AJRR, we release every year an annual report. We actually just released this last week in conjunction with the Hip and Knee Society meeting. I encourage you to go to our website and download it. There's a lot of information in here. We're very, very proud of this. You can see there's a few less procedures included in this. We freeze the analytic file quite early for analyses for that year. But again, a lot of good, dense information in here. This is really our kind of jewel in the crown product in terms of reporting out the trends and what else we are seeing in the hip and knee replacement space. Here's a couple of charts from that report. I'll ask an audience participation question here. Does anyone know what this is? You're looking here at the top on hospital volume for hip and knee procedures. Here's ambulatory surgery centers, hip and knee procedures. What do we think happened here? COVID. COVID, right. This is the largest group I've spoken to in about two years. So this has been, you can see volumes rebound here both in hospitals and even more dramatically in ambulatory surgery centers. But now we're kind of seeing a leveling off here and we don't quite know what's driving this. If there's case migration happening to ASCs that aren't part of our participating network, if volumes are just down across the board, it's sort of unclear. This is, oh, pardon me. This is our cumulative growth over time in our procedures, hip and knee replacements. You can see on the top left there, but we're not sure. We're not seeing attrition with participation. Folks are still renewing. They're submitting cases. They're just not submitting the same volume of cases as they were pre-pandemic. So that's going to be a challenge for us, I think, moving forward, coming back from that dip and getting back to where we were. Here are five lessons learned. In the interest of time, I won't go through all five. This is from a presentation we put together recently for a group that is thinking about a biologics registry, which makes my head explode to think about and probably will become our responsibility at some point in time, at least in the orthopedic space. But five key lessons. One is establishing the registry's purpose. That seems like it would be obvious, but how often we forget that. It is a lot easier up front to decide what are you trying to measure, what are you trying to capture, what questions are you trying to answer than it is to bolt things on after the fact once you're already into the market. So begin with the purpose in mind. Data capture. Kathleen talked a lot about this. This is key. Streamlining it where possible, aligning with clinical workflows. We have a standing strategic goal to reduce the burden of participation in the registries and minimize manual data entry where possible, while retaining data quality, which Kathleen talked a lot about. Data linkage. For us in orthopedics, this is huge because many of our outcomes occur over time if you're looking at implant failure, reoperations, revisions, things of that nature, or even patient-reported outcomes, which I'll talk about in a moment. Data linkages are big for us. We're a qualified clinical data registry. We do connect with CMS claims data. We've navigated that arduous process and taken on not just the administrative burden but also the financial burden of acquiring those data on a regular basis, but that allows us to do a lot of the long-term follow-up and look at device failure and reoperation. Strong partnerships. It's true in orthopedics. I'm sure it's true in many of your specialties as well. All of those registries, we have partnerships with the relevant society for that anatomical or procedural space. It's key for us for cross-promotion, getting buy-in, getting participation. And planning for data reuse opportunities, the fifth thing. Every single year, me and my team have goals to have new ways to use the data. That's really our elevator pitch. Submit your data to us once and let us help you do multiple different interesting and cool things with it, meet some of your needs, whether those are payer programs, benchmarking, whatever the case may be, incentives, alternatives to prior authorization, data reuse opportunities. Give people a reason to do it because none of our stuff is mandated. It's all discretionary. Double-clicking on a couple of those. This is data collection. We've really worked a lot to expand beyond the procedure, looking at devices. As I say, the shorthand is who put what into who, when, and why. Getting beyond that and actually looking at how is the patient faring over time brings in a lot of challenges with collecting data in the office setting, linking it back to an index procedure, particularly when the procedure happened a year ago in a hospital somewhere. I mentioned the integration of Medicare claims data. Very important for us. We're continuing to try to explore this with private payers, and it's not going great. So this is an opportunity for us to try to continue to look at longitudinal follow-up, how the patients are faring over time, when reoperations and things like that are occurring. The CMS data has been a boon, but we have a lot of the population out there, the folks that are not Medicare-eligible, that are left out of these long-term analyses that we publish. And again, giving folks a reason to participate. We have actually a very productive partnership with the Joint Commission. They require our hip and knee registry and our spine registry for their advanced certifications in those spaces reflectively. That has been a big participation incentive. We have a couple of private payer programs, including one with Aetna, that we're very proud of around their institutes of quality, and we're looking to build out more of those opportunities. So again, the data reuse is absolutely a key participation driver. Research. We've done a lot to ramp up our research output across the last couple of years in particular. Had a slew of posters and presentations at our AAOS annual meeting this past March, including one that won Best Poster, of which we're very proud. This is also now a major pain point for us, because success begets more inbounds, and we have not done a good job of managing expectations for, how do you get access to the data? How long will it take us to turn it around? Who can get access to the data? And how many of these analyses can we do in any given submission cycle? If you talk to our members, the answer is, if you've got 1,000 good procedures, you should probably do 1,000 analyses. In a world of finite resources, that's simply not possible. So this is an area of work for us to manage expectations. Good problem to have, or, you know, it's a good problem, but as the saying goes, a good problem is still a problem. So something we're working to solve. Ongoing challenges. I've talked about a few of these. Data collection in ambulatory settings. We have a lot of ASCs out there that don't have any HIT technology and don't have a lot of staff. They run a lot leaner than some of the hospitals. So how do you get data out when there's not existing HIT to leverage some of what Kathleen talked about? If that's not there, what do you do? Uptake and participation. I mentioned shoulder and elbow has been kind of slow to launch. We've struggled to put together a compelling value proposition there for folks. Why should they do this? In an era of constrained resources and when an activity is viewed as discretionary, maybe I don't need to do it if it's not that compelling to me, or I do a lower volume of shoulders than I do of hips and knees. PROMs and other non-episodic measures, tough to collect things over time, and again, tie that back to the index procedure. Discussing and managing inbound research requests. I talked about that before. It's great that we've ramped up the research. Now we have to deal with an influx of inbound requests. And finally, Kathleen talked about this too, we have a goal to break even or better financially. We can't just run in at operating loss or continue to take money out of reserves in perpetuity. So these are our challenges and look forward to discussing any and all of these in the future. Thanks for the opportunity to present. Thank you, Nate. Well, we're going to do question and answer after the presentations. So next up, Jim, if you want to join me. Jim Wasinski, he is with the Society for Vascular Surgeons. He oversees the vascular quality initiative, which has many, many registries itself. One of the last things I did when I left ACC was a collaboration with you and putting some programs together, which happened after I left, but it was a pleasure getting to meet you and I'm glad you're here today. Thank you. Thanks so much, Kathleen. All right, first lesson learned, never follow Nathan. You have a very similar presentation, he knocked it out of the park, so be kind. So I've got a similar presentation, but a little different take on it. I'm going to provide very high level contextual information about the registry, but go more into the successes that we're trying to build on and the challenges that I think we're all facing and hopefully get some good input from the crowd. So the Society for Vascular Surgery, our vascular quality initiative, really started out in around 2003, more of a regional effort in the New England, but as it was growing, they then engaged the Society for Vascular Surgery to take it national. We took over in 2011. So similar mission to the others. But three major components to our registry is the society runs it, again, as a separate legal entity. We are a AHRQ patient safety organization. So we manage the IP, the data, work with the volunteers, really run the registry. Then we have 18 regional groups that get together on a semi-annual basis to discuss data and to put in place quality initiatives. Got a lot of great success around that and that's a large reason for the growth of the registry. And then partnerships. I'll be talking a lot about partnerships and one of the main ones is with our data vendor, but they're more than just a data vendor. So we currently use Fivos. They were also known as MedStreaming and M2S. But in addition to using them as the sort of cloud-based receptacle, we also rely on them for contracting, marketing, and probably most important to this is a tight collaboration on management of a lot of our industry programs. So here a quick look at our growth. So I did mention we started out in 2011 with the society taking over the registry. At that point, we had about 100 hospitals in the registry. And you can see now here in 2022, we're really nearing 1,000 centers. And then our milestone. So again, this is where Nathan, you know, I feel September 14th, we hit a million procedures in our registry. We put out a big press release and then I see numbers like, oh, we've got 18 million. But I'm still proud of our million registries. And as Kathleen mentioned, we take a modular approach and that's not the same as all other registries. We've got 14 different registries within the VQI, but the whole notion there is based on the type of procedures that are done within each practice setting and where folks want to really focus their quality initiatives. We let the hospitals and clinicians decide where they want to focus. So these can all be bought separate, but it does bring up problems with pricing and potential bundling strategies that folks ask you to look at. Diversity and participation. We do believe that this is a very strong part of our real world evidence. So we do have a nice mix of the types of centers that are in the VQI from academic to teaching affiliated community practice. And then that 12% other really represents either vein clinics, OBLs, or surgery centers. One of the interesting things you think about in terms of, okay, the VQI, it's the vascular quality initiative. Less than 50% of physicians putting data into the vascular quality initiatives are actually vascular surgeons. And this was one of the reasons that we did partner with the NCDR in three of our registries because we have a nice percentage of cardiologists, radiologists, general surgeons, neurosurgeons putting data into the registry. And because of this, strong partnerships. And I think this is really key to our success because it starts out at design. So knowing that we've got all these different practitioners putting data in, we invite them to the table to serve on our committees to design the registries. And then with our relationship with ACC, they now have a formal board positions from the executive committee all the way through all of our registry committees. So it's that constant input from our different communities to help design and maintain the registries. But then you're also sure you're hearing from the different voices that are doing your procedure. So I think this is a big part of our success. The study for vascular surgery is still in its vascular ultrasound, still in its infancy. But Kathleen, you mentioned the notion of machine learning behind that data. That's what we want to get to with our relationship with the ultrasound is bringing those images and then applying machine learning to that. So that's something we're also looking at. We work with American Heart and Vascular Medicine on our medicine registry. So Nathan mentioned launching new registries. Well, I'll speak to that in a moment. COVID's a wonderful thing. Don't ever watch registries right before COVID. It's not going to work. So, successes. So, this is where I want to spend some time in successes and challenges. So, I think we heard earlier at the breakfast session you were talking about, you heard about guidelines and making sure that we've got data in the registries to support clinical guidelines. And this is something new for us. And I'll be interested to hear what other registries are doing. But about two years ago, we started making sure that the registries are designed to collect data elements for high-level evidence measures in the society's guidelines. And then what we do is we're able to measure that in the registry, provide feedback reports to the centers and at the physician level. So, they're actually tracking compliance with the guidelines. And then we can also do analyses to see are those, is compliance with those guidelines actually resulting in better quality? And then there's a continuous feedback loop to our writing committees that do the guidelines to show, okay, here's what we're seeing in terms of the outcomes based on compliance with these guidelines. So, this has been very powerful for us and we're continuing to do this. So, this is one of the areas where there's a strong integration between the registry and the society's writing committees. Increasing the number of QI and research projects. Again, as Nathan mentioned, so this is a great success, but then becomes a burden. And again, given that we're a PSO, we can't report outcomes publicly. There's ways around it in terms of getting consent and doing it, but typically you don't. So, a lot of registries have various star awards. We have participation awards. Again, because we can't provide publicly reported outcomes. But what that's allowed us to do is really drive the number of quality improvement projects that are happening across all of our centers because we make the actual attendance at the regional meetings and evidence of actually doing quality improvement projects based on the registry data part of their star awards. So, we have this year over 100 centers who have submitted quality improvement charters to the PSO, and we track that. And then that becomes a lot of the basis for our education at our annual meeting. It's been really wildly successful, but then becomes a burden because we host monthly conference calls, quarterly webinars. We produce quality improvement toolkits. It all works really well. It really gets people engaged with the data, and it takes time and resources to do that. And then on the research side, I think there's great variety in how we allow our members to access blinded data sets for research. I think the VQI sits somewhere in the middle. So, we do require that sites submit abstracts that are reviewed by a research advisory committee. We do this every two months on both the arterial and the venous side. But this has resulted in us getting about 200 proposals every year for research. So, you've got a committee who meets, you know, every other month, who has to review these, put the data out, have to generate data use agreements, track if the publications are being done. So, it's really great. We've got well over 500 now papers that have been published based on the data. But, again, it puts a strain on the staffing to do that. Probably one of the most successful things that's led to a lot of our growth, especially relative to industry, again, is the linkage to CMS data. So, again, partnerships, we work closely with Dartmouth and Cornell to be able to do a lot of our claims data linkage. We're a very device heavy specialty, and that's led us to be able to work with industry, the FDA, and CMS. But you have to have clean data. And so, we were one of the first registries to be able to integrate with GoodID, but that data is not clean. And so, a big part of what we did was partner with Symmetrix, and they are helping us clean that data. So, if you're looking to integrate with GoodID, you can't rely on industry to have clean data in there. So, challenges. Automated data extraction, we know that. We're working on standardizing op notes, so we've got agreements in place with Epic and Cerner. In terms of the market factors, we know the economy, market consolidation, COVID. I think a unique thing that we've been talking about lately is problems with staffing, particularly in the analytics area. Very hard to get good statisticians on board. So, if anybody's got any ideas, very willing to hear about that. And then, just the registry revisions. It's just how do you balance what's needed for research and your work with the FDA and CMS versus the minimal data set that various practice settings can put into the registry. And so, then, again, it's a balance. How are you designing your registry to get the right data in so you can serve your various constituencies? Thank you, Jim. Thank you. Yeah, you can see the themes and how they're overlapping. Our next speaker and last speaker is Bill Seward. He oversees the STS cardiac surgery registry, which probably is the oldest registry for medical specialties. So, we're looking forward to hear how things are going after, what, 25 years of business, right? Thanks, Kathleen. Good morning, everyone. It's good to be with all of you. It's a great opportunity to share our experience. So, I'm going to go pretty quick here. I'm going to really mirror, in a lot of ways, the approach that Jim took and really focus on providing you just a quick overview of the STS National Database. And then, you know, quick highlights on some of our major successes and, you know, some of the challenges and our strategies going forward. Okay. I have no disclosures. So, the STS Database is made up of four different components, component registries, and they're really subspecialty-centric. So, our specialty is really made up of primarily adult cardiac surgeons, but there are also, you know, congenital surgeons, thoracic surgeons, focused on lung disease, and then mechanical circulatory support. So, our largest registry is the Adult Cardiac Surgery Database, and that was established in 1989. And we have just over 1,000 participants. The congenital database was established in 2002, and there's about 119 participants. Thoracic Database, also established in 2002, focused on esophageal and lung cancer procedures. Then the Intermax and PDMAX registries are focused on mechanical circulatory support devices. And this is a registry that was actually established in 2005 by UAB and NIH, and STS acquired this registry in 2018. And then the last registry I've listed here is not what we consider part of the STS National Database, but nonetheless a very important asset in terms of our suite of registries is the STS and American College of Cardiology TVT Registry, which is focused on transcatheter valve therapy procedures. So, just a quick overview on the STS Database. Setting aside, again, the TVT Registry, we have over 7.5 million thoracic surgery procedural records across all of the databases. And our participants are provided reports in a cloud-based platform that they use for benchmarking and quality improvement. And, you know, one of the things I think that sets us apart is from a data completeness and a data quality standpoint, which you've heard repeatedly from many of my colleagues, is we actually audit 10% of our records on an annual basis. And it's done externally, and it's primarily done to ensure accuracy, consistency, and completeness. Our data collection is rigorous. We rely on data managers at the institutional level to input data. And it's a very granular and standardized process. And the variables are developed by clinician volunteers. And for us, the procedures that are collected, the data that are collected, it's all in. If you are a participant in the registry, you've got to submit all your data. So, major successes for us over the many years in which we've been working is, you know, we've scientifically validated, you know, improvement in outcomes, morbidity and mortality over time. And probably one of our biggest pluses is, you know, our penetration in the marketplace. I mean, 95% of the participating hospital surgeons are involved in the adult cardiac database and the congenital database, truly making us a national benchmark in cardiac surgery. Very powerful. And for us, we see ourselves very much on the leading edge of developing risk models and risk calculators and composite quality measures for major procedures. We also have a lot of experience working with CMS and other payers on value-based programs. And we have a long history of transparently reporting data. We've been recognized by U.S. News and World Report. Many of our surgeons and institutions use the data that are collected from the STS database to submit to U.S. News and World Report. And, you know, those data, of course, institutions compete against each other for their rankings. Other areas of success, like Jim and everyone else, you know, linking to claims data, very, very important. As well as for us, the CDC's National Death Index, all for longitudinal research and quality improvement. Other areas of important success for us have been our ongoing collaboration and growth with our engagement with industry. Major challenges, again, very consistent with what you've heard from my colleagues so far today. Data collection burden is omnipresent. I mean, honestly, this is one of the things that we all are challenged by. Automated data abstraction is not something that we've figured out yet. We're working hard at this to try to figure out how to get the kind of granular, high-quality, complete data without having to do it manually. Other big challenge, just at a high level, running clinical registries is extremely expensive. And probably the biggest challenge that we see on an ongoing basis is managing external vendors. For us, we're hyper-focused on, you know, demonstrating and consistently reinventing the value proposition for our participants and keeping an eye on competitive administrative data sets, competitive competitors that are working to generate similar data sets. And Jim mentioned, you know, the war for talent. We over the last two or three years have worked very hard to try to develop an internal data analytic team. And it's very hard to find good people and retain them. Key future strategies for us, again, leveraging our in-house analytics for research and quality improvement, longitudinal research, seminal research, and quality is an area of very, very high importance to us. We're also working to try to incorporate patient-reported outcomes into the STS database. Very, very challenging. And, you know, it's very important, but it's a difficult, difficult task. Some of it we're throwing significant resources at to try to figure out. I think I could probably learn something from both Jim and Nate and their efforts here. The other thing, I've talked about reducing data collection burden. You know, leveraging artificial intelligence, machine learning, natural language processing. These are all things that I think everyone's trying to figure out how to use and, you know, for automatic data abstraction. We're working on this, but we, again, haven't figured this completely out. Finally, the last few things, you know, we have a number of different engagements with industry and with regulators, CMS, FDA, to really leverage the infrastructure that we've established over decades. And, you know, where we see the future and real opportunity is, frankly, in pragmatic clinical trials. And what I mean by that is, you know, we have a lot of experience working with industry and with FDA in the post-device approval, post-market surveillance, post-approval studies. We have less depth in pragmatic clinical trials, which is really pre-market. Right? Pre-market approval of devices. And the whole ecosystem of registries and regulation is dramatically changed over the last five to ten years. And where there's, I think, tremendous opportunity is leveraging professional societies, you know, registries to do these kinds of studies and work with industry. It's a lot cheaper, and, you know, real-world evidence is unique. It's not how clinical trials have been run historically, but, you know, there's great promise and great potential there. So let me stop there, and Kathleen maybe can help lead us through some Q&A. Yeah, that's perfect. Yeah, come on over if our other speakers can come up and join us. We have about ten minutes for some questions, so if you have a question. I'm not sure if we have another mic, but we'll hear you and we'll repeat it up here. Let me just make an observation. So thank you very much. We all share something in common. How many of you are involved with or are running registries, have a role in leading registries? Okay, so probably about a third or so of you. We all can share the same pain. It's just a complex field. So thank you for your contributions to the world of medicine. A couple just comments, and then if you have questions. The themes that I jotted down, data burden, the value proposition, why is anybody going to bother? Getting expert resources, recognizing that informatics and health IT, but also the clinical expertise and merging it all together and finding and keeping those resources. Access to data and research, making those decisions, what are the process, and being fair but also being realistic. Building strong strategic partnerships with FDA and CMS and other federal agencies. Working with industry to provide a service to regulated research. Using data to support society needs, association needs, such as guideline compliance, and raising the tide for how care is being provided across the country and the globe, too, because several of you are international as well. I appreciate you brought up, Bill, about using your data for post-market research to support it and not so much in the pre-market. But registry data can help to create contemporaneous control groups, which can help decrease some of the time and money that goes into building pre-market clinical trials and having the contemporaneous controls to compare to the study group. And, you know, that was just sort of a list. It's a little bit of everything, right? It's operationalizing, being strategic, and also staying true to your mission. So as we get started here, let me ask, is there anyone that would like to ask our panel a question? Of course, our panelists are welcome to ask questions, too. Yes, Marty Liggett. Marty Liggett, just so you know, full disclosure, is my boss. She's responsible for the ASH Research Collaborative, and also, as many of you know, she's currently our president for CMSS. Marty. That's a great question. As I mentioned, you know, this is something that we really are hyper focused on, you know, in part because we've been around forever, and I think it's easy to get complacent. But I think probably the elevator pitch for us is, you know, the power of the data that we have collected over time, and that we're continuing to collect, you know, allows us to, you know, influence and provide, you know, clear data for quality improvement. But it also allows us to shape the specialty, the future of the specialty. And, you know, as medicine continues to evolve and technology continues to drive much of what happens in the hospitals and in the ORs, data is everything. Data is power. And, you know, we, there's certainly, I think, ongoing challenges, resource constraints. You know, hospitals are struggling with, you know, the sheer volume of registries that they're being asked to participate in. And so I think, you know, you know, this, this, this, the importance of continually making the case for why, you know, why investing in the resources to collect the data and analyze the data and, you know, develop programs to, to publish and produce results and maintain platforms, you know, that, to, to surface data to participants. These are all things that I think you, you know, help us to make that case. Thanks, Paul. I think it's very similar for us. Some different angles we're taking, physician level reporting, getting down to, again, back to, for us, back to compliance with guidelines and benchmarking with your peers, both for the guidelines and appropriate use is becoming a very hot topic. And so you have to think about the registry's involvement with that. We don't, as a society, necessarily want to be the police of that, but we want to be able to provide the clinicians rapid feedback so they can benchmark their practice. So that's kind of a unique value proposition. The ability for centers to participate in clinical trials and then offset their registry fees by participation in those trials is also a big benefit. And then the access to the research data. Biggest challenge, again, is in the smaller settings, whether it's, you know, vein clinic, the outpatient, where, again, you've got different staffing, different economic factors and just making that you've got to get that data automation down. That's what we or or slim the slim the forms down. But then do you really run two different registries at the same time? That's a tough one. Yeah, I think I think at a 50,000 foot level, the answer is the multi decade, multi administration transmission or transition to value based care. And if that if we're not in the business of defining quality, measuring quality and reporting quality, it's going to be defined, measured and reported for us by people that don't know as much, probably out of administrative claims data sets, not clinical data and in ways that are disadvantageous for the profession and for patients. So I think that's probably the long run view is that you need to get on board with this and get on board with it early so that we have a have a chance of controlling our own fates and destinies. I think when you get down to brass tacks and why does somebody decide to purchase today? I think the answer is it depends. And that's why we try to offer a menu set of downstream data use and reuse opportunities. What may motivate one organization to say we're at the Institute of Quality to maintain that designation? We need to be part of the hip and knee registry at AOS. Someone else that may not move them, but they want that joint commission advanced certification or they want to benchmark themselves against national averages. So I don't think on the day to day there's a silver bullet there. But our overarching approach is you've got to be in this quality measurement and reporting business or it's going to happen to you and you're not going to have a seat at the table. Great, thank you. So you brought something you said purchase. So let me just acknowledge a couple of unique things. The three of you run procedure and surgical. So it's a point in time registries and capturing data before, during and after. The one that Ash has is population based. So it's hematologic conditions. I know ACC has an NMI program. AHA has a couple population programs. And there are nuances in how you execute those, whether they're device, procedural or population. And the other thing you said purchase. Bill, you charge. Can you share just the range of like an annual fee for a site? Yeah, it's variable. I mean, it depends on the registry. You know, the Intermax registry. I'm sorry, the Intermax, PDMAX registry are, are, you know, you set aside TVT. That's probably the most expensive registry. But the Intermax registry, I think, is in the twelve to fifteen thousand range. The adult cardiac depends on a lot of different variables. But, you know, five to eight thousand. Cardiac, congenital is in the similar neighborhood. Thoracic is our cheapest. I mean, we, you know, it's hundreds of dollars. And what am I forgetting? TVT. TVT. And that's that's a big one. The initial, the initial fee is around twenty seven thousand or so. And then it drops dramatically down from there. But these are all I mean, I'm not sort of spilling the beans here. Yeah, it's yeah. And it's their annual fees. And I know we do have to wrap up. But if you don't mind, just the range for the modules. We're about twenty six hundred dollars per module on an annual basis. Average subscription rate, you know, per modules, per hospital is probably like four point five. So, you know, when you factor them together, it's probably not very different. But again, sounds trite. I think for Bill and I, the cost of the registry on an annual basis is really nothing. It's the labor to put the data in. And that's the nut for our registries to crack is that data automation. And then we can get a higher level of purchase. Great. Yeah, for us, the base price is four thousand a year per registry per site. We will do deals. Our finance director is here. She doesn't like when we do deals, but we do deals where we bundle registries and we get multi site discounts. But that's I think hurt us in some ways with our shoulder and elbow registry, because folks will look at that and say we do five hundred or a thousand hips and knees a year and a hundred shoulders. Why should we pay the same price for the shoulder? It's the same running cost for us. It doesn't matter on the volume. But yeah, that's the basis. But I would echo what Jim said as well. I mean, it's the upfront startup costs, the staff and IT that are the bigger, I think, cost driver than the annual subscription. Yeah. And as I mentioned, there's no charge to participate in our programs at this point in time. We're just we're just kicking off. So thank you so much to the three of you for preparing for today. I hope that everyone found this information very helpful. Some of us will be around if I'm going to be actually out this afternoon. So introducing Emily Tucker, who will be here a bit. She's our deputy director for the data hub program, and she pretty much knows almost everything I know. So please feel free to stop her. But it's nice seeing everyone and have a great meeting. Thanks.

medical registries

surgical specialties

procedural specialties

data collection

registry participation

quality improvement

value-based care

pricing models

data automation

evidence-based care