The broadcast is now starting. All attendees are in listen-only mode. Okay. Hello, everyone. Welcome to today's webinar. This is Dr. Helen Burstyn. I'm the CEO of the Council of Medical Specialty Societies, and thank you so much for joining us today for our panel presentation on Best Practices, Lessons Learned, and Strategies to Engage Patients in Clinical Registries, some work we were very, very fortunate to have support from PCORI Engagement Award. With this, I'm delighted to be joined today by our partner, the National Health Council, Eleanor Profeto, who will be my co-moderator for today's presentation. Before we get started, I've got a few housekeeping items for you. There will be a Q&A session at the close of the presentations. Questions can be submitted at any time online by using the question box on the right-hand lower side of your screen. If you do not see the question box, please click the white arrow in the orange box located on the right side of your screen. Please note that this webinar is being recorded and we will post it on the CMSS website for future access. You can immediately access a PDF of the presentation slides using the GoToWebinar dialog box to the right of your screen. At the conclusion of this webinar, you can also access the evaluation via the GoToWebinar. We would really love your feedback on the webinar and help us make future virtual discussions like this better, especially as we look at a future where we suspect we'll be doing a lot more virtual than in person. Please contact us at info at CMSS.org if you need assistance. And we are ready to begin. Let's move on to the discussion. All right, trying to advance. Thank you so much. So, here's our overview for today. First, we're going to have our brief introduction from myself and Eleanor Profeto. We're then going to walk through each of the panelists' presentations and we're delighted to have three great examples of specialty society and patient organization and patient engagement examples to walk through. We'll have an opportunity for some Q&A at the end. Both will review your questions in the question box as well as have any dialogue between the panelists and presenters. And then finally, we'll summarize at the end and again, we'll invite you to join us by sharing your evaluation. First, next slide. First, we want to tell you a little bit about the two organizations that came together to work on this important effort funded by PCORI. I'll start first with CMSS. Many of our specialty societies are likely on the call with us today. We have really been focusing on how we could support and strengthen our specialty societies to meet future challenges, how we could catalyze improvement through convening, collaboration, collective voice, and action. We view the work we've done through this PCORI work as a perfect example here of where we can, in fact, work together to improve a particular area like patient-reported outcomes and patient engagement. And then finally, providing a proactive platform to really assess and address the emerging issues across societies that influence both the future of healthcare and very importantly, influence the care and well-being of the patients we serve. Just to share on the next slide, these are our members. Really, the 46 members of CMSS are incredibly diverse across the medical-surgical fields, and we're really very honored to have their engagement. I will say in our efforts over the last year or so, we have heard from so many of them that patient registries remain a very high priority for them, efforts to engage patients in their work is also a high priority, and in particular, how we move to inclusion of patient-reported outcomes and a more patient-centered approach to both the research endeavors, the quality endeavors, as well as the overall work of our clinical registries, and I would argue even broadly, the work of our societies, including clinical guidelines and other efforts. I'm going to flip to the next slide and invite Eleanor to give us a little bit more about the National Health Council. Eleanor. Helen, thank you so much. I really appreciate your introduction, and I appreciate being involved in this project. I think it's probably getting close to two years now that we originally started talking about this, and I just wanted to give a little bit of an overview about the National Health Council and why we thought it would be important to join into a project like this. You see in front of me, in front of you, over 60 patient advocacy organizations that are a big part of the National Health Council. We have five categories of members, and per our bylaws, the largest category must always be the patient advocacy organizations, and you see on this side the wide variety of shapes and sizes of the organizations that we represent, anything from the largest organizations like the American Heart Association and the American Diabetes Association to many of the smaller groups that represent rare disease populations. So we have a wide variety, but our organization is run by and for patient advocacy organizations, and it's part of every single thing we do. It's what we live and breathe every day, and so when we were invited to take part in this project, it was really important to us because we really want to see the patient community be engaged in these kinds of efforts. We really believe that having patients engaged not just as participants but really as partners in this kind of work really helps make the work stronger and it makes it more meaningful to the patients that are involved and the patients that the registries are serving. So it's a great opportunity for us. We were really happy to be part of it and have been involved every step of the way. Next slide, please. Just a little bit more about the National Health Council. The mission is to provide a united voice for the over 160 million people in the United States with chronic diseases and disabilities and their family caregivers, and as you might guess, we do it through all of those organizations that you saw in the previous slide. We bring those organizations together around the cross-cutting issues that are having an impact on patients' lives, and as you can imagine, we've had a very busy couple of weeks bringing our membership together around what's happening with COVID-19 and pulling together the resources that we can for all those organizations to be able to support their constituents that are out there, and our vision, of course, is that we envision a society in which all people have access to quality, affordable health care, and right now, all of this is just so important with everything that's going on in our country and elsewhere, and so it's one of the important roles of the National Health Council is to always be sure that the patient's voice is part of the conversation. We really look forward to continuing this conversation about how medical specialty societies and patient advocacy organizations can be working together, not just on registries, but in many other ways, and I think, in my view, that's a future next step for us because I think that this first step with registries has been so successful for both sides of the coin, and that I can really see this moving forward in the future to help with registries, but also to move on to other topic areas, so again, Helen and the other folks who have been involved with this, we're so thankful to you for thinking of bringing us in from the very beginning. That's always where we like to be brought in, and so I think this is our relationship as two organizations has been probably a good model for where we want to go and what we're going to talk about more on the one-on-one society and patient group relationships. We kicked it off at the organizational level. Thank you so much. Oh, thank you, Eleanor, and thanks to the National Health Council. It has been a great example of what's possible when you partner. We're going to walk you through a little bit of the work we have done over the last year and a half or so, almost two years now, and in particular, talk about the efforts of this PCORI award. Waiting for the slide to advance. Perfect, thank you. So, we are, I'm sorry, it looks like it went too far now. Excellent, thank you. So, when we were initially putting this award together, and again, thanks so much to PCORI for its wisdom of having these patient engagement awards, we thought there was a real opportunity here to have a project where we could really think about how we could incorporate the patient voice and best practices for organizations, and the clinical data registries presented the obvious example of where we wanted to ensure that patients had a real opportunity to both help with thinking through the measures that matter, the questions that matter, and ensuring that their voices were heard. But we really do believe, and we'll talk more about it as Eleanor just suggested, this is, I think, many of the preferred practices we put into the initial primer and the patient primer to follow really are, we believe, transferable to other areas like clinical guidelines, and we'll hear a little bit about that shortly. As part of this effort, we've gone through a process and very ably led by Sarah Samsel, who is the program lead. We initially began with a membership registry survey to find out what was the level of engagement among our societies with patients, with patient groups. We then did an environmental scan of the societies to begin to understand what were best practices around patient engagement. We went way beyond our societies, both internationally and nationally, to do key informant interviews with other patient organizations, especially societies, research organizations, even federal government like AHRQ and FDA, to really, again, pull in what we could learn, where were the best practices. Then have been working on reporting our findings through a series of meetings we've had, including our registry meeting last May, as well as the development and the dissemination of the two primers, one directed at specialty societies, and the second one really directed at patient organizations for how they can engage effectively with specialty societies. Perfect. This slide actually is, I think I keep going to too many apologies. I'm too impatient. I can't handle the delay. For all of those, you know me on the call, that's classic. So, we were really thrilled as part of our summit on the future of clinical registries to have a wonderful presentation by Sally Okun, the Vice President for Policy and Ethics at Patients Like Me, who really talked about this concept of the next generation of clinical registries, and making the argument that societies are, in fact, very well positioned to create a new value for registries by partnering with patients. And on this slide, she specifically lays out the concepts of what our old generation characteristics versus where we could move. So, certainly the current or older generation registries have been primarily designed and developed by clinicians. We'd like to move to doing it with clinicians in partnership with patients. We want to prioritize the culture of performance improvement in practice, but also think about how this is really a team sport in understanding how the patients play a critical role in both the patient's safety as well as quality improvement. Initially, measurement tools really align to meet provider needs and objectives. We want to move to a concept where, of course, they will continue to meet the needs of clinicians and providers. We also want to make sure they're aligned to meet what matters to patients. Major outputs were initially centered on measuring quality of care and generating new clinical knowledge, which remained critical. But again, we want to do that with having patient input about what's important to measure in terms of quality and generating novel patient experience knowledge. So, why patient engagement matters? And for this group, probably those of you on the phone are very well aware of this. We really have begun to see a growing focus on patient-centeredness in all we do, particularly clinical research, medical care, regulatory environment. We want to see how we can increase the availability and dissemination of evidence used to inform healthcare decision-making. We want to see how we can address critical evidence gaps where decisional dilemmas for both patients, families, caregivers, and clinicians remain. We want to enhance timely communication between the healthcare system and patients, families, and caregivers. We really see, especially society registries and many of the other registries that are out there, really have had an evolving value proposition. Certainly, the focus continues to be around improving quality of care, improving quality reporting for some of the federal programs like MACRA, certainly, but also going beyond that, generating new science and evidence, informing health policy decisions, enhancing society value to members by giving them back information on their performance benchmark to others so they can use that to improve their own care. And finally, how do we begin increasingly interfacing the analytics for electronic health records and other data sources to get a much richer picture of what's happening in clinical care? Cost and claims data are becoming more important than ever. The ability to look at imaging and imaging systems, patient-reported outcomes, which we'll talk a lot about today, and even this growing emphasis on genetics and genomics as that information comes online, how does it become part of that broader value proposition for our clinical registries? As part of this effort, we've been also trying to look at what we could learn from our societies. Who's doing some innovative work out there that we can begin to showcase and highlight as examples to build on? So we, as part of our survey with our specialty societies, for example, we know some groups rely heavily on focus groups, like in surgery and psychiatry. Some groups are having patients on advisory committees, and we'll hear some examples of that today. Patients on guideline and measurement panels, meetings with patient groups, and certainly the reproductive medicine group is going to give you a great example of their collaborative efforts with patient groups today. And even having a registry public advisory board, and you'll hear about that today from our friends at orthopedic surgery. And one of the key goals here for all of us is we really do want to move to this model of incorporating what matters most to patients, and I believe also oftentimes what matters most to clinicians, which is integrating the voice of the patient, in particular patient-reported outcomes. And for those who are developing new patient-reported outcomes or selecting among existing patient-reported outcomes for use, we want to be able to describe all those concepts that are reported as important by patients in the target population or in our closely related population. And we also want to have a rationale for which concepts should be included or excluded. And actually, Eleanor at the National Health Council has been very helpful in this concept of thinking about what are patient-centered PRO measures, and this builds on a lot of the work that Sarah and I and others did at the National Quality Forum. We really want these measures to be consistently understandable and meaningful to patients themselves, but that really does require a level of input up front that is really, we believe, transformative. So as we think about these evolving patient partnerships, we recognize there's great promise. There are barriers we need to address, including some of the following listed here. You've got to get patients, families, and caregivers involved much earlier in our research and registry efforts. We need to think about the convenience and the accessibility of participation. Recognizing people have jobs, it's difficult to take a day off, for example, and participate in another person, another group's meeting. We have to make sure we're getting diverse representation. We have to have materials that are understandable, appropriate literacy levels, materials for non-English speaking patients. We want to make sure we address concerns about risks to privacy and impact on benefit coverage of a disclosure of important personal and health care information. And really importantly, we want to promote that bi-directionality of data sharing between patients and the health system, patients and the clinical registry. What are we learning collectively with our registries that are of import to patients? How do we build on that? How do we do this together? And at the end of the day, it's really about getting to what matters to patients, and I would argue to patients and clinicians, which is what's most relevant, what's most important, and how do we ensure that we do this in a way that reflects shared values. So I now have the distinct pleasure of turning the presentation over to our colleagues from the reproductive medicine field who have a very interesting collaborative model that they'll describe to you of how they have worked together in the registry space. Today's presenters will include Ricardo Aziz, the relatively new CEO of the American Society for Reproductive Medicine, Barbara Calora, the President and CEO of Resolve, the National Infertility Association, and Kevin Doody, the advocacy representative for the Center for Assisted Reproduction Society for Assisted Reproductive Technology, or SART. So with that, I'm going to hand over the controls to Ricardo, and let's hear from reproductive medicine. Thank you. Ricardo, are you with us? Sorry, I was still muted. That's okay. People, people mute me all the time and that's part of being a CEO. Anyway, Ellen, thank you very much for this invitation and for allowing us to present what SART, ASRM, and Resolve are doing together. I'm going to go over a little bit of the history of our organization and just to let people know who we are and why we're so vested in this process. Now, the American Society for Reproductive Medicine was founded in 1944, and it continues to be a multidisciplinary organization dedicated to the advancement of science and practice of reproductive medicine. I think the key here is the multidisciplinarity. We have clinicians of all stripes, physicians, of course, but also other health care providers. We have business administrators. And so we have a very diverse organization that allows us to listen and forces us to listen to different points of views. The Society for Assisted Reproductive Technology was a offspring, if you would, of the American Society for Reproductive Medicine. They were formed in 1985 as an affiliate society. And so I should say that the American Society for Reproductive Medicine tends to function more as a federation than it does as a single organization today, because we have a number of different affiliate societies. And so at that time, SART was created as an affiliate of the American Fertility Society, which, of course, is now renamed the American Society for Reproductive Medicine. Now, interestingly, and just to share a little bit of history around the registry for IVF and other assisted reproductive technologies outcome, because the registry that we're going to be talking about today is about the advanced, if you would, technology, assisted reproductive technologies. And the outcomes related to that. Now, part of this was leveraging sort of a political opportunity in March of 1989, a panel of society members testified at a hearing held by the House Subcommittee on Regulation and Small Business Opportunities, which was then chaired by Representative Ron Wyden of Oregon. And this was about IVF, the growing business of IVF and GIFT, and they, at that point, Representative Wyden had a focus on a survey that was already being undertaken by the American Fertility Society, by SART and AFS of 139 clinics. So, again, SART, in collaboration with AFS, has started their survey in 1985. And so this was a way to highlight the value that this was having for government regulation, as well as for patients. And we were given, at that point, very high marks for our work. And that sort of became the nidus for advancing this registry. Now, the ASRM has been focused on patient education since about the mid-1970s. Infertility, reproductive care, of course, like all of our specialties, but even more so in this setting, is really driven by patient and patient demand. I mean, it is something that we have taken patient and patient input very importantly. We established a patient education committee in the mid-1980s and, in fact, have a separate patient-oriented website, reproductivefacts.org, in our main website. ASRM has been working with our partner organization, which you'll hear about in a second, from Barbara Kalora. So we have been working with Resolve in various ways, almost since their inception in 1974. For example, we more recently have been co-sponsoring annual DC advocacy days, and we have really an enhanced programming partnership in a very tangible manner, focused on advocacy, enhanced patient education and coverage at work. In fact, we work almost as an intertwined organization. So for us, it's been a very important partnership that we continue to do so. I should say that we have been working with Resolve, who is an established organization, so we haven't really tried to partner with patients on a random basis. We have been working and trying to strengthen Resolve as an organization, which is different than the strategies that other organizations may have used. At this point, I'd like to introduce Barb Kalora, who is President and CEO of Resolve, as well as Kevin Doody, who is a member of the SART Executive Council and advocacy representative for SART, for them to take the presentation forward. I think, Kevin, I believe you're on the line, but I'll control your slides. Great. Thank you, Ricardo. And it's a pleasure to be here. As Dr. Aziz mentioned previously, we began as kind of an offshoot of the ASRM and have evolved into an affiliated society whose goal has been from the beginning to monitor the safety and efficacy of in vitro fertilization. So just a reminder that this is a relatively new process. When this organization was formed, IVF was first reported to be successfully done in 1978 in the US in 1980. New procedures like frozen embryo, the ability to cryopreserve and transfer thawed embryos was 1984. So this organization started as an effort within our community to make sure that these new processes were safe and also effective. We have evolved since that initial beginning to be really the primary organization of all professionals that are involved in providing assisted reproductive technologies. ART has been defined as those processes that involve the direct handling of eggs or embryos. Membership within the organization is not voluntary, is not mandatory, but the vast majority of programs that perform clinics and programs that perform in vitro fertilization are SART member clinics. It's estimated that about 90% of all the ART cycles yearly are done by members. So the clinics are members and there are individuals that are also members of SART. These individuals can be physician reproductive specialists, reproductive endocrine infertility subspecialists, or they can be PhDs, laboratory director, embryologists. Additionally, we've more recently added some international membership categories, but that's not the focus, obviously, of today's call. So I believe that SART was the first national medical registry in the USA. Now, medical registries were not new at that time, common in fields such as oncology, but I believe we were the first to be organized on a truly national level. Can you give me the next slide? So when this registry began, clinics would collect the cycle information and patient information and tabulate all of that and submit for aggregation centrally. As it evolved into the early 1990s, this changed into what we call treatment cycle specific data collection. And we call our treatment cycles, our courses of treatment cycles, as opposed to what oncologists would call rounds of chemotherapy or other things. So these cycles can be egg retrieval cycles, or they can be embryo transfer cycles. We collect in our registry really comprehensive patient details and characteristics, including patient health identifiers, social security numbers, et cetera. And this is important for us in terms of our goals of our registry. We also collect all the treatment details, so the medications that are used, the amounts of medicines that are used, the number of eggs that are retrieved, how many fertilized, the embryo development details, et cetera. And then finally, we collect the pregnancy outcomes. So miscarriages, term deliveries, multiple pregnancies, birth defects, birth weights, mode of delivery, pregnancy complications, et cetera. We use our registry in many different ways. One way is to help fulfill the federal mandate of annual clinic specific success reports. So we publish a success report for each of our member clinics as well as a comprehensive national report. We also use this information for external QA or peer review. So clinics can internally use their success rates and compare to the national reports for benchmarking purposes. But the SAHRQ organization does identify IVF programs that are outliers with respect to success rates, that is, low performing clinics or clinics that have too high of a rate of complications. And our primary complication is related to multiple gestation. We also use our registry for research. So there have been dozens, now hundreds of peer review publications that have been done using the SAHRQ database, SAHRQ CORS registry. I should mention SAHRQ CORS. CORS stands for Clinical Outcome Reporting System. So we can use the data both by itself or in conjunction with linkage to other registries, including cancer registries, vital records. We have more recently used them with congenital anomaly registries in four different states and some educational attainment for the offspring evaluation. And finally, our registry is used to provide data to federal and state governments for policy purposes, and this is done fairly frequently. And I should mention, for patient purposes, we've used the data to construct patient predictor tools, instruments that allow patients to put in their information, their age, their height, weight. diagnosis, et cetera, so that they can calculate their likelihood of success with ART treatments. Next slide, Ricardo. The SAHRQ registry organization has a lot of moving parts. So we have an executive council that has on the committee representatives from mental health and from nurses, et cetera. It is also composed of our committee chairs, and we have several different committees. I've listed some of them here, including committees that conduct our quality assurance efforts, our efforts to validate the clinical data, our registry committee advertising and research. Our organization has really evolved beyond simply data collection and reporting to be functional as essentially an accreditation mechanism to ensure that clinics that are members are holding to a high standard. And we have had multiple collaborations over time, including one that's important for today's discussion and that is that we've participated and together with Resolve have led the National Coalition for Oversight of Assisted Reproductive Technologies since 1997. Now, this coalition includes not only SART and Resolve, but also representatives from the Center for Disease Control and Prevention, the National Institute for Child Health Development, the FDA, American Bar Association, and others. And it's really through that organization, I think, that the partnership with Resolve, the patient advocacy organization that Barbara Kalura represents, has strengthened. Additionally, we collaborate with third-party payers. This speaks to our functioning as an accreditation mechanism. Most of the third-party payers do require SART membership for payment for reimbursement for services related to ART. Next slide. I think the SART-Resolve collaboration has really strengthened since 2013. That's when we began to restructure the data collection and reporting process to be able to account for new practice patterns. We have gotten a lot of input from Resolve regarding our data validation methods and how we remediate data that was reported inaccurately. They've given us a lot of advice, and I think this really speaks to the intent of the meeting today. They've given us advice regarding our SART mobile patient app. So, this app was designed to allow collection of our registry data, things such as cycle starts and birth outcomes, directly from patients. And our goal with this is to ultimately gain access to long-term follow-up from both the patients that have undergone ART as well as their offspring. And an example most recently is that we have collaborated with Resolve to organize a patient focus group that will be held this coming Monday as we implement some planned changes to our success report tables, including our table titles, etc., to make sure that they're appropriate to be understood by patients. So, thank you. Hi, everyone. Thank you so much, Dr. Aziz and Dr. Doody. As you've heard, Resolve, the National Infertility Association, is a patient advocacy organization. We've been around a long time, and we're really about helping people who are challenged in their family building journey reach their own resolution. So, the way that we have always viewed the registry, the SART course registry, is as an informational tool for anyone who might be thinking about or needing IVF. Oftentimes, patients go to the SART website, and they will look up the practices and clinics in their area. They might look up practices and clinics in other parts of the country. They'll use that predictor tool that Dr. Doody mentioned, and it will be able to tell them, based on what they believe their treatment protocol will be, perhaps what kind of outcomes they could expect for themselves as well. So, it also helps patients frame conversations as they get ready for a consultation or continue treatment with their provider on how they should be approaching their treatment. Should they be considering doing a different protocol, thinking about maybe donor egg or donor sperm? And it really does provide a great deal of information from the patient perspective, keeping in mind that I think SART has done a really good job of taking the data that used to be produced, you know, in paper and sent out to their members to now being very accessible on the SART website and knowing that patients are going there and patients are looking up this information and trying to make it as patient-friendly as possible. And as you heard Dr. Doody just describe, even to the point where SART is considering making some tweaks and changes to table headers and even some narrative content on that registry data, but first sitting down with Resolve and a focus group of patients that we put together to see what those patients think. You know, as we all know, physicians have clinical terms for everything and those are not always easily understood by patients. And so this is a great attempt to ensure that right from the beginning of these changes we're going to have patient input so that it's easily understood. And also it's been kind of interesting, and Dr. Doody can attest to this, you know, oftentimes what physicians may view as being very important outcomes data to report really isn't that important to patients. And so it's important to get that input and ensure that what the patient really needs and wants from an outcome perspective is what they're getting. I'm going to go ahead. I just have one more slide, I believe. I want to just pick up where Dr. Doody was talking about this entity called NCORT, this National Coalition for the Oversight of ART. When I think about why our collaboration has been so successful, it's because we already had this existing mechanism where we literally sat down face-to-face. For many years, it was twice a year, and in the last few years we've moved to an annual meeting. And interestingly enough, SART and Resolve formed this NCORT group and we co-chair it. So right from the get-go, it was patient and provider focused in this particular entity. And so the registry, the SART course registry, is always discussed at this annual NCORT meeting and very spirited conversations at that meeting about the registry and input from Resolve. And I think really that's where it started, where then SART said, hey, outside of these NCORT meetings, let's engage with Resolve on the registry and how we want to proceed. As you've heard from Dr. Aziz, it has continued to evolve and ASRM and Resolve realized we both had keen interest in access to care programs and initiatives, not only from a public policy perspective, but also from employers and payers. And so we've formed a very close collaboration on those specific initiatives over the last few years. Next slide. So some of these lessons learned, and Dr. Aziz, I think you were going to go over this. I'm happy to do it if you want me to. Okay, I'll do it. So some lessons learned, as we talked about on our experience, that relationship that we developed through that NCORT entity, that national coalition meeting that we had, you know, start early. It's never too early. And even if you do not necessarily have a specific project right now or registry, invest in communicating and engaging with patient education organizations and partnerships. Next. I think this is a great one, you know, never let a good crisis go to waste. You know, this is very keen in our field because we do have some interesting adversaries in our space, and it has served us very well to have that partnership and that relationship as we are challenged with a number of forces in our space, but also looking for opportunities to make our medical treatments more accessible and the patient experience even better. Barbara, I'm on the line, so thank you for that. And again, we appreciate that. But I think, as you said, start early investing. You have a history there. But I think that a lot of people will say, well, we've already are starting late. And the answer is and start now, because frankly, the longer you invest in that patient education, patient oriented partnership, the better the partnership will work. And again, as you said, Barbara, never let a good crisis to waste. We're in the middle of a crisis today, and that will give us lots of lessons and will change the way we do business. So I'm one of those who believes that we should. Now, let's continue a little bit about other lessons that this partnership between ASRAM, Start and Resolve, around the clinical registry and patient outcome registry that we have put in place. We need to make sure that the registry is outcomes focused, relevant and important value to both clinicians and patients. And I say consumers because sometimes some organizations call patients consumers, but for us, it's patients. It's very important that it matters. Lots of times we put together these clinical repositories that have maybe a research focus, but really are not very important to patients and certainly not important to them as individuals. And that is going to really affect the success of that. So it's very important that the outcomes be focused, outcomes focused and also relevant. You have to genuinely seek and listen to patient input. This is not a matter of just saying, well, you know, let's just put a patient rep on our committee or let's just, this is just not the way to do things. Lots of times we sort of do this hand waving around patient input because we do think that perhaps if we just have a patient there, that will be helpful. But the truth is you need a large group of patients voices to be heard because again, not all patients are the same and you have to genuinely really set up systems to both seek and listen to patient input. You need to support the development investment in the future of that patient advocacy organization. I mean, we at ASRAM have been investing and working very hard with Resolve to make sure that Resolve is successful, not just ourselves. And for us, that's a very important part. Many organizations really don't do that proactively, don't help develop their patient advocacy organizations or even develop a patient advocate organization. They tend to work with patients on an ad hoc basis. And I think that one of the things that we have learned is that that's a less than successful avenue and it's best to invest in your partners as you would invest in anything else. And then we have chosen to, in our case, we have chosen to engage a formal ongoing patient advocacy organization, Resolve, rather than individual patients or an informal advocacy group in the development and continued improvement of our registry. As I said before, there are different strategies. You'll be hearing different strategies around how to engage patients and clinical registries in our setting because patients tend to be so critical to the priorities of our specialty. We have chosen to engage really with a formal patient advocacy organization and we found it to be a value. So see, these are some of the lessons learned as many others, of course, but I hope that this has been some of some value to you as you understand how we have been able to leverage SARDCORS, the clinical registry, on behalf of patients and clinicians throughout. Hello, this is Nate Glusenkamp. I'm the Chief of Quality and Registries at the American Academy of Orthopedic Surgeons. I'll pick it up from there. Thanks so much to my colleagues in reproductive medicine for that introduction, as well as to Helen and all of the staff and leadership at CMSS for the opportunity to talk a little bit about how we have incorporated the voice of the patient into the AOS family of registries. The AOS is the National Organization for Orthopedic Surgeons. We now have registries covering hip and knee replacement, shoulder and elbow replacement, musculoskeletal tumor, as well as an exciting new collaboration with the AANS with the neurosurgeons around spine. So very glad to talk about how the voice of the patient is incorporated into those registries. Something that you heard Dr. Doody talk about was the evolution. And I think if we think about or consider our registries in the orthopedic space, probably version 1.0 was set up to look at device failures over the long haul, which are those devices that are underperforming over time. And probably registry 2.0 started to incorporate outcomes, but these were really outcomes that could be measured by claims and had some sort of cost of care implication, things like readmission, infection, reoperation, all of which are outcomes that absolutely matter, but those also aren't the reasons why patients first come to Orthopedic Surgeons. It's not why they seek care. They seek orthopedic care because of pain, because of loss of function, because they're struggling to take their evening walk with their partner because they can't work. So as we really continue to evolve our registries in a direction where we are measuring what is meaningful to the patient, what matters, and then bringing that back to the surgeons for their clinical decision-making, having that voice of the patient is absolutely essential. I think Barbara a moment ago pointed out that what the surgeons want or need to see from a registry isn't necessarily what the patients want or need to see. And I think that's something that we've certainly experienced in orthopedics as well. As we have continued this evolution away from simply device survivorship and looking at outcomes on a clinical basis as well as meaningfully clinically relevant changes in a patient's care or for the long haul or their condition over the long haul as assessed by preoperative and postoperative patient-reported outcome measures, we've also tried to incorporate in the research aims, the type of real-world evidence that we're generating and the kind of publications that we're putting out. So I am very, very pleased to have with me on the line today, Diana Stilwell. Diana is the Vice Chair of our Public Advisory Board. The Public Advisory Board is a group that started out with the AJRR, with our hip and knee registry, and now has an expanded mandate over all of the AAOS or involvement in all of the AAOS registries. If you can advance me one more slide, Diana. This is just an overview of some of what we have done at the Academy, much of which I summarized a moment ago. But we do now have Public Advisory Board representation on all of our governance bodies. I think Ricardo's point a moment ago is very well taken. If you can advance me one more, that it's not enough simply to put a patient rep on a various, on a committee and call it a day. It's important that you have multiple different patient voices, that they are contributing in a substantive and meaningful way, and that we are actually changing the work product in a tangible fashion based on the input. And again, for us, we've gone beyond simply having a rep on each steering committee and have put together this Patient Advisory Board, which impacts and also executes a lot of the communication that we do in a patient-facing manner. So with that, if you advance one more slide, I will hand it over to Diana Stilwell as we continue to prepare for this webinar. It was increasingly apparent that the really interesting information and what I think will be relevant and replicable for other medical professional societies is what Diana can share with you around how she came to be involved, the work of the Public Advisory Board, and what advice she would have for other societies looking at potentially exploring an entity or a body like this to be influential in their further development, their further evolution of their clinical data registries as they seek to capture the voice of the patient and track those outcomes that really matter to the individual seeking care of their loved ones. So without further ado, I'll hand over to Diana Stilwell. Thank you, Nate, and thank you to CMSS for the opportunity to talk with you about our experience as patient representatives partnering with the Academy and their registry program. So I'm gonna talk briefly about three aspects of our experience that may be helpful to other organizations looking to engage patients more substantively in your programs. First of all, what motivates us as patient representatives to be part of this type of program, then what roles we can play in support of the work that your registry is doing, and then finally, some strategies that our group has found helpful in terms of how to engage the public and additional participants to be part of the registry's work. So as you think about how you might involve the public and patients, I thought it might be interesting to just reflect on some of the motivations that have fueled my own participation and my colleagues' participation in the public advisory board at the Academy. First of all, many of us have personal experience. I think we all obviously know someone who's had a joint replacement, and many of the folks on our public advisory board have had that experience themselves personally and have a desire to help other patients. We have some members who've had multiple procedures, and I've often heard them say, I wish I'd known this, I wish someone had told me that, and so that personal experience I think fuels a lot of their interest and engagement. And for some, this is an opportunity to give back to a field, to the orthopedic field that has made such a difference to their individual quality of life through the improvements in function and pain management that procedures have provided them. We also have some nurses and allied health professionals who are on the front lines with patients and really understand their questions and their challenges in a way that might be complementary to what surgeons bring to the table. They have really valuable insights to share based on their experience and that of the patients that they care for. And we all share an interest in understanding how the registry data can, over time, be used by providers, by health systems, by payers, by industry, and by individuals to inform decisions. For example, patients increasingly have choices about where to seek care for joint problems, and registry participation as a starting point can be a symbol of an organization's commitment to quality. And ultimately, the availability of facility or surgeon-specific outcomes in the future as that becomes more feasible can help inform some of the choices that patients might be making. And then finally, for some of us, our participation also and interest grows out of a professional interest. Some of us have day jobs, if you will, with organizations either providing direct patient care or producing patient education materials or developing shared decision-making programs. Over time, these organizations may well be able to use registry data to ensure that the information they're communicating out to their customers and their clients is the most accurate and up-to-date and useful. We're also able to carry insights from the data collection challenges that registries face back into our work and to our colleagues, giving everyone a stronger understanding of what data are available, how they can be used, where there might be limitations, and where changes are coming in those data over time. We're all incredibly enthusiastic about the increasing availability of patient-reported outcomes data. As others have said, those outcomes are especially meaningful to patients, and being part of the Public Advisory Board really strengthens our collective understanding of the value of those data, and we can also be alert for opportunities in the work that we do to reinforce their importance. Some of the roles that we have played as a Public Advisory Board, we serve as committee observers and participants. We're really grateful the Academy has supported our active participation and attendance at meetings, including in-person meetings once a year. They provide tremendous staff support to our committee as well. By observing and participating alongside the surgeons and the analysts and the other professionals who run the registry, we as members and representatives of the public are better able to understand the types and the uses of registry data, and this, in turn, informs our thinking about how we can provide data to the public and the value it can have. It's also been helpful for us to understand the limits of how the data can be shared based on data use agreements or just the maturity of the database, and to gently encourage continued improvements in our ability to share as much data with the public as possible within the constraints that exist. And at times, and I know this is very true for my colleagues who have personal experience with joint replacement, at times we provide a patient's perspective in committee discussions. We're also able to support our registry and committee objectives. Our work with the Publications Committee, for example, our group helped to expand the purpose of that committee to include the creation of public reports that highlight registry data and explain the value it can have for patients and the public in language that's accessible to a lay audience, so beyond the reports that are prepared for the professional audience. We also involve patients and the public on research committees. And this creates an opportunity for the public and patients to have input to research questions and projects, for example, offering suggestions for additional lines of inquiry that might be complementary to a proposed study. And the Public Advisory Board can also play a role in the dissemination of some of those results, for example, similar to a public report on the registry data, but with a focus on a public-facing summary of research that comes out of more detailed looks at that registry data. We also support the educational goals of the registry. As patient-reported outcomes become more prominent, we can help play a role in supporting the registry and communicating with the public about why those outcomes and those measures are important and why it's so important to fill out those surveys that we get after our procedures. And then finally, as Nate was mentioning, our registry program is expanding into additional areas, and those of us who've been with the group for a number of years now can really serve as ambassadors to those new registries as they come online, bringing our positive experience with the more established registries to support the new groups in some of the same ways. And so this is the last slide here, just how we've engaged the public and expanded our membership. We thought this information might be helpful to groups that are thinking about where can we find the types of people and members of the public who would be interested in this kind of work. We've really looked for a range of both lived experience and expertise, so people with personal experience with musculoskeletal procedures. We've leveraged our personal and professional networks, whether that's colleagues, whether that's friends, board members, or people that we sit on other types of groups with. We found that personal experience with the musculoskeletal procedures is helpful. The folks that we found have a wonderful ability to recognize that their personal experience has great value, even if it may not generalize and are able to talk about it in a way that's really, really helpful, and they're very aware of how the use of personal stories and anecdotes can serve an important role in illustrating broader concepts and themes. We found that it's not important to be a data expert, but it is helpful to have enough comfort and confidence to understand the issues and the challenges the registry is facing. And again, the purposes and the limitations of the data that are available. Similarly, clinical expertise is not really required, but a comfort level with the clinical discussions that take place sometimes at committee meetings is very helpful. People who are in roles where they're supporting patients, they're providing patient care, or they're consuming the type of data that a registry produces and making it meaningful for patients, those kinds of backgrounds we found really helpful among our group. We've considered seeking representatives from professional organizations, so like nursing and physical therapy. We think representatives from those groups could be very complementary to our group. And some of our members have also had leadership roles in more traditional patient advocacy groups like the Arthritis Foundation, and that experience is also tremendously helpful. One recommendation I would offer is to give, when you bring new patients or members of the public into your organization, to give them a really good onboarding. I remember when I joined the organization a few years back, the chair at the time spent a lot of time with me reviewing the history of the registry, its current state, its future plans. When groups like ours meet just a few times a year, it can take a long time to learn the jargon, learn the acronyms, the history, the objectives, the strategic goals of the organization. And having a really good orientation in all of that really helped me and I think helps our new members to really hit the ground running and be able to add value as members of the public really quickly. So just some examples of some of the public advisory board activities we've been involved in. Some of our members have produced blog posts and interviews, videos, other educational content that can provide a patient perspective on the registry's website. Some of our members have written blog posts about their personal experiences with joint replacement. We were really impressed to see the chair of the registry even step into that patient role and write a great blog post about his very candid experience with joint replacement. And then finally, the public and the patient-facing reports that I mentioned earlier that we partnered with the Academy staff to develop really make that registry data accessible and meaningful for a much broader audience. And with that, I'll turn it over to our colleagues from the AUA. Thank you, Nate and Diana. I appreciate the opportunity to participate on this panel. My name is Mary Beth Farquhar. I am the Executive Vice President for Research, Quality, and Scientific Affairs at the American Neurological Association. And I'm very pleased to have my colleague, Diane Quayle, President and Co-Founder of Bladder Cancer Advocacy Network. She'll be talking to you after I finish my presentation. All right. There we go. I don't have any conflict of interest and nothing to disclose on this panel. What I wanted to talk to you a little bit about, let me give you some background on the AUA. The AUA was founded in 1902 and has more than 20,000 members worldwide. Our mission is to promote the highest standard of urological care through education, research, and the formulation of healthcare policy. Developed by urologists for urologists, the AUA Quality Registry, or what as we named it, ACWA, is a national QCDR designed to measure, report, and improve healthcare quality and patient outcomes. It was launched in 2014 and currently has over 6.4 million patient encounters logged. We're a little bit new to the registry game, but what we have done is basically automated the system and had some engineers look at how to design it intelligently so that the practices can participate in quality improvement activities while minimizing the associated staff administrative burdens. And here you can see that we do collect patient reported outcomes, as well as other pieces of information from the EHR or electric health record. We use this information in a number of ways called quality improvement, professional development, including lifelong learning, evidence-based decision-making, guidelines, important patient practice, clinical outcomes, patient outcomes, and treatment. ACWA currently contains four patient reported outcome measures. This is one of the measures that we do collect. We use the registry data to identify newly diagnosed prostate cancer patients at participating practices. Users log into a secure portal and review and approve patients before the patient reported outcome email is sent out. This helps ensure that the email is sent to the correct patient. Patient reported outcomes data are collected by a web-based survey. The email includes a link to a secure web portal to send to newly diagnosed prostate cancer patients requesting that they complete the survey. In the email, it explains a summary of the responses will be shared with their physician to track their progress and with the AUA to learn more about prostate cancer and its treatments. Patients without email addresses or who have not completed the survey at the time of the next visit will be asked to complete the survey on the tablet in the physician's office. Patients may consent to allow the data to be only used for clinical care decision-making and or agree to allow the data to be also used for research purposes. After completing the consent form, patients are required to create a username and password to access the survey and their responses. During the first years, the patients are asked to complete the survey every three months. The PRO survey asks questions about urinary, sexual, bowel, and hormonal functions as well as satisfactions with care. The survey is comprised of three standardized measures including expanded prostate cancer index composite instrument, international prostate symptom score, and sexual history inventory for men. Once the survey is completed, the patient will be able to access a summary of responses. Okay. Sorry. Okay. There we go. Once the survey is done, the patient receives a summary of the responses that you can see here, information about the scores, and a dashboard that provides graphics to show progress over time. The physician receives the results as well. So either the patient or the physician can discuss the results during the next visit. This discussion could cover the impact of the patient's quality of life, suggested ways to manage and or improve the symptoms and changes in treatment if appropriate. Why add patient-reported outcomes? Basically to gather information from the patient's perspective and to also include them in decision-making as they proceed through their disease process and through the treatment process. Patient-reported outcomes. You can see this slide shows what it involves and some of the decisions that the physician is making as they try to encourage patients to become more involved in decision-making and also patient engagement. For patient-reported outcomes data for providers, we do provide benchmark performance, and you can see the two measures that we use for prostate cancer there. And for both patients and providers, we encourage, again, and try to do shared decision-making to engage the physicians and patients in those discussions, determine comparative effectiveness of treatments and what's right for the patient, and then monitor the health outcomes over time. Well, with the registry, and since we were relatively new to the registry industry, we faced a few challenges. Again, we were looking at trying to feed back the patient-reported outcomes back to the EHR so there's a record of it, and we have come up with some solutions or proposed solutions to do this. We're still working on these aspects of it, but this will give you an idea of what we're thinking. Again, another challenge that we had is increasing patient participation. It is always a problem to get patients to respond. Again, provide alternative means to collect the data, either in tablet form, paper forms, or on a telephone app. We recently also added text to notify patients. And also review surveys with the patient during or before the patient visits so that they understand what the information will be used for, and then have the option to consent or, you know, have control over their data. Another issue that we ran into was increasing practice participation. Increased marketing and practice engagement. We're engaged in that. We have a really good marketing communications group, as well as a customer service group in the ACWA data department. We also customized the portal so that it can fit different practice workflows. Lessons learned from our time in the registry is to involve patients prior to PRO implementation. All of our measures come out of our practice guidelines panels, of which we have a number of patients participating in those panels that can provide information about their disease condition as well as the treatments that they prefer and their experience as they go through the process. We also need to assess the functionality periodically, not just when an issue arises. A lot of times patients will not let you know if there is an issue with regard to the survey. You'll have to find that out on your own, so it pays to go back periodically and make sure that everything's functioning correctly. Again, utilize various data collection methods that's helpful, that gives you more information, gives you more opportunities to get the data, and identify how data will be shared with patients and providers so that people will know in advance what will happen with regard to their data. Our next steps as a registry, we're going to partner with advocacy groups and we're in the process of developing a board for that. We're also going to monitor the results of other registries that are using PRO measures to determine if we can make a kind of a mega data registry and be able to inform not only best practices for patient preferences, but also best practices for care of the patients. And then we're also going to add some more PRO measures for additional urologic conditions. And with that, it's my pleasure to introduce Diane Quayle. Diane? Oops. Hi, here I am. Thank you. Thank you, Mary Beth. And just a small correction, my last name is pronounced Qualy, and my family would say I would be remiss if I didn't make that correction. But thank you very much. I'm the last one speaking, and I think it's appropriate in that I'm going to talk about something a little different from a patient advocate standpoint. I'm the co-founder of Beacon, the Bladder Cancer Advocacy Network. And I'm going to talk a little bit more about what Dr. Burstein said in the beginning about the importance of engaging patients in your work, not just on registries, but in ways that patients and patient advocates can help in both the development of practice guidelines, as Mary Beth just referred to, that the AUA has done, as well as in other important areas of research. I was fortunate enough a couple of years ago to be invited to be the patient voice at the AUA as we modified the guidelines, both for non-muscle-invasive bladder cancer and then a separate group for revising the guidelines for muscle-invasive bladder cancer. And it was a terrific experience for me as a patient advocate to get a great understanding of the care and thoughtfulness of the process, as well as then to be able to bring the voice of the patient into that process, based both on my own experience, as well as the experience of members of our bladder cancer community. So I wanted to speak about the importance of having the patient voice in all of these areas, but then to be a little practical about how do you find those patients and how do you train those patients to be able to participate in the research process, whether it be registry or guidelines or any other kind of research program. And of course, as a representative of a patient advocacy group, I'm going to suggest and recommend that your societies do try to work directly with a patient advocacy group. At Beacon, we were lucky several years ago, working with urologists, who are bladder cancer specialists, to receive funding from PCORI to launch our patient survey network. And this was intended to assemble a group of patients to engage them to help identify what should be the important research priorities. And we thought at the outset that maybe we'd get about 250 or 300 patients to participate in this survey network, and we now have more than 1,300, which demonstrated to us the great desire and interest from the patients in our bladder cancer community to get involved in these important issues. The patient survey network identified many important research topics in the bladder cancer field, and we shared this with other funding agencies, and it continues to be an ongoing effort. Our next step to extend the success of this patient survey network and harness the energy and enthusiasm that we got from our community, we created a specific training program for our patients. And it's called PEER, P-E-E-R, Standing for Patient Empowerment Through Engagement in Research Training. And so we have developed training modules. What's really important for the patient advocate and the patients that are engaging in this, patients have their own lived experience, which is what they all bring to the table and bring to the discussion. And that's very important. But when you're talking about engaging on a broader plane, we want to make sure that the patients understand, especially in bladder cancer, the broader spectrum of the disease and have a better understanding of treatments at all stages of the disease, choices, options, priorities, all of the issues and challenges in bladder cancer. And so we have created, we have online training modules. What we do, let me back up a bit too, what we do is we just ask for volunteers. We do this through our website, through our emails, through our own social media. Are there any patients or family members, people who we consider patient advocates in the bladder cancer community who are interested in engaging in a variety of research projects? And so they volunteer. It's a bit self-selective. And then we have an online training module that include webinars, articles, other websites to help ensure that our research advocates do have a broad understanding of all of the issues that might come up throughout bladder cancer. And we also offer in-person training at our various meetings, one or two times a year. And it's been, our peer program has been a tremendous success. We now have very thoughtful and engaged patient advocates that we've used for other guidelines committees, that we've used for the Department of Defense, CDMRP programs, where our patient advocates are actually providing the patient viewpoint to the DOD grant funding. We have provided patient advocates to participate in NCI cooperative groups where they help design and implement clinical trials. We have patient advocates who've participated on advisory boards for different pharma companies as they're looking to get more engaged or bringing new drugs to the bladder cancer field. And we also use our patient advocates in Beacon's own research grant program. And we have a wonderful pool to pull from. And we have an engaged community that's not afraid to speak out and really make sure that the patient voice is heard in all of these very different and significant areas. And with that, I think it's time for questions and answers. Thank you, Diane. I'm sorry again for murdering your name. I should know better. I'm going to turn it over to Helen and Eleanor now. Okay, thanks. Great. So we do have a couple of questions that are actually quite related to each other. And so I'm actually going to turn this to Helen because a couple of these questions have to do with, you know, we've presented the case today for the importance of patient engagement. We've presented three wonderful case examples. And so we have a community's attention to, yes, I'd really like to do this. How do I do it? And I know that this has been part of this project of tools and resources and education on how to do it. So Helen, I want to turn it over to you so that you maybe you or Sarah could talk a little bit more on the work that CMSS is doing to really make this real for the specialty societies and the patient groups. Helen, are you muted? Sorry, I'm now unmuted. There I go. I was really eloquent. You missed it. Sorry. I just want to thank all of you for these great presentations. It clearly makes the case that there are lots of ways to do this. But at the end of the day, I think you've all demonstrated how important it is and how enriching it has been for each of your registries and your other related efforts like clinical guidelines. So as Eleanor mentioned, we did have one very specific question about what examples are there about education tools that could help societies and support patient engagement. We did put out just in November our primer for specialty societies on how to engage patients in clinical registries. Thanks to Sarah Sampsel for her great efforts on putting that together. That is available free of charge on our website. We encourage you to do that. Sarah and Eleanor are now working on a companion document, which is a primer directed at patient organizations who would like to work with our society. So we are really hoping that these tools begin to help you understand what could be logical next steps. Another question we got, which is a great one that might be work we need to build on going forward, is a specific question about when launching or building PROMs into work streams, are there best practices to help educate providers of how to promote and work closely with patients? So I think there's some work we've already done. But Eleanor, maybe just the piece there specifically about, you know, and or even others, best practice about how to educate folks about how to promote the importance of these to patients. Yeah, so I was going to add to what you were saying. Anyone on the call is welcome to look at the National Health Council website. We also have a number of tools available there that really talk a little bit more in depth about what good patient engagement looks like. We have a rubric about what does good patient engagement look like, what are the right things to do with examples of things that are more limited as opposed to things that would be more ideal. And so that's one way to take a look at some of the tools that we offer. Our tools are generally focused on building capacity for the patient advocacy organizations, but we've had many types of organizations give us feedback to tell us that our tools are actually more universally applicable. So you're also welcome to look there too. In terms of the question about the PROMs or other PROMs, I think part of the difficulty there is that if they were selected or incorporated into the process without patients being engaged, then it is more difficult to get the patients to answer those questions or to jump in and be enthusiastic about it. I think when patients have been part of the research process, as was explained by a number of our speakers, when they've been able to help with the selection of which questionnaires are most applicable to patients, which questionnaires capture questions that are meaningful to patients, and when is the right time to approach them, to ask them to fill out these questionnaires, and how to approach them to ask them, that's when you get more success with those types of questionnaires. It's having the patients be engaged not just in filling them out, but also helping you to determine how and when and what the approach should be. So I think that's probably one way that could help get those emails opened, is have patients who are actually involved help you to construct the note that goes out and make the request, because they know what they would respond to. I would like to ask any of the speakers who have been involved today if they have any tricks of the trade or tools that they would like to suggest. Be sure that you're off mute. In our setting, this is Ricardo Aziz with the ASRM, American Society of Reproductive Medicine. I think the tricks of the trade were to leverage, in our case, a government mandate. There was great concern about the outcome of assisted reproductive technologies, and we, rather than sort of fight that and create that as a battleground, we actually embrace that and engage our patients to do that. I think that we often, as medical societies, sort of push back against any kind of regulatory questioning, whether it's the outcome of hip surgeries, talk to our orthopedic colleagues, or it's the outcome of hernia surgeries for surgeons, or whatever it may be. We often fight those rather than simply trying to embrace those and leverage that as an opportunity to actually create a meaningful patient outcome registry. And so, I think that that's one of the lessons that we tended to learn early in our development. Anyone else want to jump in on that question? Sure. This is Nate Glusenke with AALS. I think where we've seen success from the sites that are collecting the patient-reported outcomes measures is, one, having the physicians, the surgeons, explain to the patients why this is important, that it's part of understanding their care, it's part of the clinical decision-making. We see a lot more success with even a brief explanation like that than simply having an email go out to someone that they may or may not respond to, maybe a phone call for follow-up. But if they never hear it from their doctor, they may not know why they're being asked to do this. And in a similar vein, where we've seen some of the hyper-engaged sites be extremely successful in capturing both pre- and post-surgical patient reported outcomes measures, it's when there is visibility to the patient on how that patient reported outcome is being used in the clinical decision-making. Some of the sites have started to adopt predictive modeling or things of that nature that can help them make care-pathing decisions with the patients. And one of the inputs on that can be either or both of the functional assessment for the specific jointed issue, as well as a more global health assessment like the PROMIS-10. So I think having the patients have an explanation from their care provider for why they're being asked to do this, and then seeing that come full circle into the care-pathing and the clinical decision-making, it changes the way folks engage with patients. But we've seen a lot of sites anecdotally report good success there. Yeah, and this is Mary Beth Farquhar from the AUA. Circling back to what Eleanor said about engaging patients in the development of measures as well as guidelines, we leverage our membership, which does really well on selecting a number of patients that are well-informed and not afraid to speak out during the discussion. So they are very engaged and very adamant about what they have to say, and then also adamant that they are listened to and are fully 110% engaged. So leverage the membership that you have. Also, a lot of our members are very well-versed in talking with patients about the importance of surveying their preferences as well as their outcomes. So they do have ongoing connection with them in terms of the surveys, and a lot of that time is spent discussing what the results are and how they can go ahead and utilize them for decision-making. So those are the two areas that I think would be helpful at this point. We have one additional question that's come through, and since we're running pretty tight on time now, I'm going to take the liberty of answering this question just from the National Health Council perspective. It's, how closely are you working with organizations such as ICER to inform patient-centered value? Is this in the plan? One of the things that the National Health Council has spent a lot of time doing over the last almost five years now is building capacity within the patient advocacy organization community to really be able to have these conversations with ICER. Many of our members have really become very much engaged, have really become activated and turned some of the ICER reviews around in terms of making them more patient-centered. In fact, today we had a webinar similar to this with representatives from ICER explaining what the new things are and the changes that they've made to their processes this year. So patient groups are certainly becoming very, very engaged with ICER to ensure that value assessments are much more patient-centered and are really focusing on the outcomes that are important to patients. And it's really a very vital part of the engagement process. I'm now going to turn this over back to Helen, and she's going to wrap up for us. Wonderful. Thank you so much, Eleanor. Delighted to have had you with us today. We look like we're about 10 or 15 questions in the chat box. We will go ahead and try to get them answered and posted, but would really just love to thank you for your time today. In particular, thank all of our panelists from our three great organizational partners for giving us exciting insights into what could be by bringing patients and especially societies together. And finally, would be remiss if I didn't also specifically thank PCORI, Patient-Centered Outcomes Research Institute, which supported this work with the Eugene Washington PCORI Engagement Award. Without that support, this work would not have been done and has really been invaluable, I think, to the efforts that Eleanor mentioned at the beginning, which is really the ability to bring together especially societies and patients and patient groups. So we hope this information is useful to you and your efforts to engage patients and clinical registries, clinical guidelines, and other related efforts. A recording of this webinar will be featured on the CMSS website in the next few days. We'll also post, as you can see here, the handouts are posted on here. We'll also make sure it includes a link as well to the primer that we've already developed, as well as the primer for patients that we are working on. Thank you so much to everyone, and this concludes our webinar. Have a wonderful day. Bye-bye. Thank you. I thank you. Thanks, everybody. All right. Are we off? Bye-bye.

webinar

Dr. Helen Burstyn

CMSS

patient engagement

clinical registries

PCORI Engagement Award

healthcare outcomes

patient involvement

case studies

patient-centered

patient-reported outcomes

support systems

continuous feedback