Leveraging Real World Evidence Generation: From Regulatory to Market Access

Tonya Dowd  0:00  
Good afternoon. Good evening, everyone. We are standing between you and happy hour. So I apologize for the timing. But there is champagne outside. If you want to go grab a glass, we will not take any offense to that. So um, thank you for staying. This is a really important topic near and dear to our hearts. Thank you Scott for the warm welcome and to the LSI family. He did a great job introducing me, so I don't need to belabor that. But I have an esteemed panel here of experts. And I want each of them to introduce their selves until and before I get into the meat of the presentation today and the panel discussion. So Gretchen do wanna go ahead and first. Sure. Thanks, Tanya.

Gretchen Nelson  0:37  
 My name is Gretchen Nelson and I am the Director of Clinical Affairs for neurology and digital health at MCRA. I've been at MCRA about four and a half months prior to that I was with Abbott for 12 years in three different divisions cardiac rhythm management, neuro modulation and vascular. I ran clinical trials there from boots on the ground in the sites collecting the data all the way up to managing directing writing protocols, and executing on all clinical levels. Prior to that I was also with j&j for seven years in sales and marketing.

Tonya Dowd  1:15  
Thanks, Gretchen. We do have one panel member who is not with us, although she videoed her introduction. So we will run that now.

Devjani Saha  1:23  
Afternoon everyone, thank you for your time and my sincere apologies for not being able to attend in person. But hopefully I'll still be able to share with you some of my experiences and real world evidence. So currently, I am the director of neuro Regulatory Affairs at MCRA. And prior to coming to MCRA, I was at the FDA for a little over seven years, as first as a lead reviewer and then as a clinical trial and real world, real world evidence policy analyst at the Center for Devices and Radiological Health at the FDA. So as a real world evidence analyst, I reviewed past submissions that have used real world evidence to really identify some insights that we could use to train review teams and strengthen the consistency across the different review areas. I also integrated regulations, and I also interpreted them and as well as guidance to determine the applicability of FDA regulations on real world evidence activities for medical devices in particular. So I'm really looking forward to being a part of this panel, and I hope that I will be able to attend in person.

Tonya Dowd  2:42  
Thanks. Thank you, Devjani, and she will be on demand video through the panel presentation. Nicole.

Nicole Fitzpatrick  2:49  
Okay. So hi, I'm Nicole Fitzpatrick. I'm a senior manager of evidence development and market access at MCRA. I have over 15 years of experience that spans a continuum from research to market access. And I started my career actually back as a neuroscience researcher at the University of Pittsburgh, after which I then joined the University innovation community for several years to help drive promising research projects towards commercialization. And so I worked with a lot of fledgling technologies to help them develop some of their initial evidence and also developed some of their business cases, for moving forward into commercialization.And then prior to joining MCRA, I was actually a leader in the development of the vital innovation program at Highmark Health. And this was a program where we work closely with companies with emerging technologies to develop real world evidence in our system. So Highmark was a payer provider systems, we had access to both clinical and claims data. And we were able to run prospective studies to understand the evidence from a clinical cost and experience standpoint, and really develop evidence packages for driving, reimbursement and coverage decisions.

And the team now that I'm on MCRA, is doing very similar work, assisting clients with developing evidence, and also communicating the value of their solutions in order to drive that market access.

Tonya Dowd  4:12  
Thank you, Nicole.

Mina Fahim  4:13  
Mina, everyone, thanks for sticking around. I'm the lone man out. I'm not part of MCRA but super indebted to them. They're extra consulting on a 510 K with us, I have the pleasure of serving as CEO and president of a company called mediView. We are augmented reality surgical Navigation Company. Basically think about X ray vision for clinicians during percutaneous procedures. So background is in class one through class three products, r&d, go to markets, Structural Heart electrophysiology, and I've always kind of been an intrapreneur and eventually just said, Okay, it's I'm gonna be an entrepreneur and with that the appreciation of what real world data means in our cap capacity is a big deal because we have to be able to demonstrate, hey, it's not just a cool shiny toy that has really cool holograms, but actually adds real value. And so that's an area that we have actually built into our solution. We'll talk a little bit about that in just a little bit about how can we integrate real world data collection into product design. There'll be one area and then my other backgrounds really have been in the space of intrapreneurship in the heart valve space, the cardiac ablation space, the orthopedic space. And, really, for us, working with MCRA has been out on the 510 K side. And I'm sure we working on even more when it comes to the real world evidence and actual data generation side. So it's a pleasure to be part of the panel. And thank you all for letting me participate.

Tonya Dowd  5:41  
Thank you, Mina, and thank you, everyone. Before we get into the meat of the presentation, again, just to introduce MCRA. We are a full service, integrated consulting organization and a full service CRO we have about over 150 employees. We are based in DC, Connecticut and New York, and also remotely based, of course post COVID as well. And we have an experienced very experienced leadership team and staff at MCRA. We really have branched out into multiple therapy areas, including electrophysiology imaging, orthopedics, we were born in the orthopedic and spine world, but really went in are going into now digital health diagnostics, cardiovascular neurology, biologics are really taking the whole gamut of the disease areas, and bringing our expertise to the table for innovators in these spaces.

Our model is really an integrated approach, which is so important because the regulatory strategy needs to inform the clinical strategy and the reimbursement strategy, market access strategy, commercial strategy needs to inform the clinical strategy as well. So if we've all heard the quote, In God, we trust, all others must bring data from W E Deming. That is no I think everybody in this room understands the importance of data and evidence not just for regulatory clearance or approval, but to really establish market adoption. So we'll talk about that a bit more today in terms of the utility of real world evidence in the real world with various stakeholders. So today, on the panel, again, we have the innovators perspective, we have the payers perspective, we have the clinical perspective, and also the regulatory perspective. So it's important because all of these stakeholders really have a different play, and agenda in terms of what they're going to need for real world evidence. So again, looking at that broad area of stakeholders, just in terms of real world evidence, you've got facilities, hospitals, integrated delivery networks, you've got payers, the payer entities, we've got patients, of course, employers, we are very employer centric, and focused healthcare structure and ecosystem here in the United States, and also physicians. So really thinking and taking that into account, what is real world data, and real world evidence, I feel like those acronyms are thrown out quite a bit, but really looking at the definition of you know, real world data. And that is relating to patient health status, and or the delivery of health care routinely collected from a variety of sources. So really, what is that? What is happening to that patient, through electronic health records, medical claims data, doing a retrospective analysis, registry studies, patient generated data, including home health care settings, and also really new and emerging to the market at digital health. So really taking that that data that's coming in asynchronously from digital health applications is going to be moving forward is going to be really important as we move forward in generating real world evidence. So real world evidence. So using the data going into real world evidence, is the clinical evidence regarding the usage and potential benefits of the technology or intervention we're talking about or diagnostic. So generating, generated using different study designs, including RCTs, externally controlled trials, observational studies, registry studies, retrospective claims analyses, so really, let's ground ourselves in you know, those definitions. It's important to, you know, really consider kind of three stakeholders here and what they're looking for in terms of evidence in general. So if we think about the FDA, they're looking at evaluating the device, the device itself for efficacy and safety, and thinking you're asking do the benefits outweigh the risks of for the device? On the other side of the coin, if you think about payers, and purchasers and hospital systems providers, they're looking to evaluate the impact the device has on health outcomes for a very specific and defined patient population, and they want the demonstrated effectiveness, not just in a controlled clinical trial, but how is it really being used? And what impact is it having on the population health in real world settings, and that data, of course needs to be published. So having that end to end generate evidence generation strategy that really speaks to all of your stakeholders and considers every stakeholder you're going to need to, you know, to really work with is important for market adoption. So FDAs use of real world evidence, and you know, they have really a defined use of it, the 21 Century Cures Act of 2016, required that the FDA establish a program to use the real world to use real world evidence in, you know, to support new indications for drugs or devices. There is, you know, there was there was guidance, draft guidance that was published in 2021. And it was informed by a lot of demonstration and pilot projects that were happening. If we think about the pay from the payers perspective, and who's really bearing financial risk for managing a patient population, there is no defined real world evidence program. From the payers perspective, the only one that really comes close is the is CMS' NCD or CED coupled with a CED, so NCD is national coverage decision to which requires coverage with evidence development. That's kind of a really a real real world evidence, protocol or requirement, but that that mechanism for coverage is not the norm. So there's not really a defined pathway or set of regulations that the commercial payers use, for example, or the integrated delivery network. So there's a lot of, there's a lot of, you know, room for improvement there and to establish some commonalities and consistent programs. But I think if we think about the shifting model from a fee for service type of payment structure in the US to alternative payment models, or those types of payment models that focus on value versus volume, I think real world evidence will certainly come much more into the foray as important.

I think the it's important to think about the utility of real world evidence for medical device innovators, most of you in the room are, you know, medical device innovators, payers, employers and consumers. So thinking about the purchasers, payers and employers, they can measure the actual costs that are paid in the system for both to from both the payer and the consumer, and accurately, accurately capture the impact on concomitant therapies, comorbidities, adherence challenges, and other factors that really differentiate the real world from clinical trial settings. If you think about consumers the increase in growth of High Deductible Health Plans, the consumers really want to know, is this technology, drug device diagnostic going to work for me? And is it really worth my spend? Maybe so really thinking about that, as the consumers really become more of an educated purchaser of health care. And then also from the med tech innovator standpoint, you know, using acquire data to develop actionable decision making and clinical utility. So, diagnostic, our digital health and diagnostics is really important. Understanding really what the clinical utility is of your diagnostic, for example, to understand what impact is it making, so you can really demonstrate that and use that in your claims as you work with your customers. And then supporting these, you know, these realized outcomes with value analysis committees, and other stakeholders as important. So really considering bringing in real world evidence to your evidence generation plan in total. So we'll jump right into the panel discussion. And I'm going to move over to the chair here, transition. And the first question. Hi, everyone. First question for the group is how can real world data be used in your respective areas when thinking about the entire product lifecycle? For example, product development, regulatory clinical reimbursement, and post marketing surveillance? Anybody want to go first?

Gretchen Nelson  14:32  
Sure. Well, I think because I've been in industry for so long, I've been forced to think about that whole lifecycle from the thought process all the way through to marketing. And what I've seen is a lot of times clinical studies will be set up to show a result that doesn't always is seen in the real world. So it gets to market and when it's there, people are asking, why aren't I seeing the same thing and physicians are wondering, you know, I see in your results here that you show this much pain relief, or this happens, and I'm not seeing this in my practice. So I think when you think about setting up clinical trials, you have to think of taking it outside of the box of sitting in an office writing up a protocol to what happens in reality, what happens in the physician's office, and running a clinical trial in that matter, will get you the results that you will be able to show in the real world once it's approved. And I think a lot of times, they're you don't have the ability to be like boots on the ground and get and get an understanding of what's that's like, a lot of us come out as PhD scientists, and we're very scientific in our thought, and not really what what's going to happen. So I've been forced to kind of think of how real world data will work, and, you know, champion ideas, when writing up protocols to to increase that awareness, and bring it back to what we're seeing in the real world. Great.

Nicole Fitzpatrick  16:13  
I'll give the kind of payer perspective. So I do think that, you know, starting off a lot of technology companies, rightly so are very focused on sort of regulatory, and getting kind of over that hurdle. And I think that once they're over that hurdle, there sort of a, an open ended question, as far as you know, what, what evidence do I need now? You know, how do I get to full market access? And I think the truth is, you do need additional evidence. And, you know, so just taking a payer standpoint, for example, from the clinical perspective, of course, they're interested in sort of your, you know, standard RCT outcomes, but I think there's additional evidence on the clinical side that they're very interested in as well. So things like, you know, what are the what's the clinical outcomes compared to the current standard of care? I think that's very important. So, you know, is it better than the current standard of care? And if so, how much better? They're also very interested in sort of longer term outcomes. So you know, post one year outcomes at least one year, but even even more is better. But, you know, and then also sort of what is the quality or level of level of evidence that you're bringing to the table from the payer perspective, so they do want to see very strong clinical outcome data, you know, in terms of study design and statistics. And then on the economic side, I think there's also evidence that you need to bring to the table for the payer. So looking at cost and utilization again, you know, how does that compare to the current standard of care? Is it more or less cost effective than the current standard of care? And then, you know, probably the third thing I would say, is around clinical utility. So I think we hear this actually from a lot of payers, you know, is their clinical utility. So how does this new product fit into the current workflow? How does it affect treatment decisioning, downstream? And, you know, I think that you're able to leverage real world data to sort of answer all of those different questions, and start to develop sort of that evidence package that you can then bring in front of payers. Great. Thank you, Nicole.

Mina Fahim  18:19  
Yeah, I mean, data is awesome, right. I mean, I think that's one of the things that I think a lot of the time, though, it's not, from the innovators perspective, it's thought about too late. And one of the things that we really tried to put an emphasis on is having the foresight of what type of data you want to collect, from the time, you know, from the entire health, the entire patient care pathway of what was their referral pathway. And in our world, in cancer ablation, for example, you know, there's only 24% of indicated patients even find their way to ablation, relative to the other options of chemotherapy and resection. And you start asking questions around why. And you actually find that, hey, there's therapies that have reimbursement, there's data there, but we use it as a way to uncover Hey, what claims are they trying to make? Where's it falling short? Where are where are their evidence gaps? And then we use those questions to really influence product design. So that's more on the upstream side of using existing data gaps in data influencing product design, but you asked about the entire lifecycle. And we also think downstream to post-market surveillance. And that's an area and you know, we'll talk about a use case a little bit later about, how can you use post-market surveillance, internal right to the internal company's process they have in place to understand, for example, Kappas and understand problems in the field and then tie that back to corrective actions because right, we're judged on that and those things are audited. And the type of justification for root cause analysis and understanding of how am I going to fix it is a big deal and for a company like us that we're trying to push the spectrum of technology being used in the in the OR I would say XR in the OR it's you got to convince people. And that only way that comes is if you can show tangible benefit with a solution to encourage that adoption, because we've got, we think, pretty sweet technology. But at the end of the day, right, it's the voice of the customer, and the clinician voice that's going to drive it. And quite frankly, you don't get that as a freebie. You don't get that as a luxury unless you really have the clinician buy in to frame what type of data is collected when, and then use that, you know, peer reviewed publication around that entire lifecycle is a big deal for us both upstream and downstream.

Tonya Dowd  20:47  
Yeah, you both touched on something really important. And I think it's important to impart on the audience here. And I think Mina, you mentioned, you know, really the importance of early market assessment, planning, right, and understanding and you Nicole said, Where does the technology fit in the current care continuum? Can you speak a little bit about how important it is to really have an appreciation and understanding upstream? So you can really make those claims downstream and develop the right evidence?

Mina Fahim  21:16  
Yeah. So because right now the time to talk about some of the use cases. Okay, we can Yeah, so I mean, for us, right, one of the things that we want to help is increase access to ablation. And we try to understand why isn't ablation being adopted as much as possible, and you started looking at some of the factors around the margins of ablation, but you have to ask, like, the five why's to get deeper and deeper and deeper, because what we end up doing is understanding, okay, well, if you get the right margins, and if you don't cause any ancillary structural damage on the way to your target, and if you manage to ablate, where you want to ablate it's a lot of ifs, then you can be successful. But what ends up happening is, though, that those are barriers, and for us understanding to say, Okay, now I really need to understand what is the type of margin that I need that is going to say, you know, using a medi view product, we can say that there's going to be a safer or more effective outcome. Efficacy is obviously down the road for us. But we're already thinking that and, you know, our regulatory internal regulatory group, Adam Cargill and his team are saying, Listen, if we ever want to do a PMA someday, we got to start thinking about how do we design the product today, so that if it is put into a PMA, those are expensive, and they take a long time. And if you're going to make that time and money investment, you have to have some predictability in, hey, this is where we hope the outcome comes out to be and we do our diligence to say then let's design our device. Let's design in data, autonomous data collection from the device itself, via sensors, imaging, etc. So that when we get to that point of wanting to demonstrate, hey, safer, potentially more efficacious down the road, you're not having to rewind the clock and start from scratch.

Tonya Dowd  23:08  
Yeah, it's excellent point. Do you have anything to add? Nicole,

Nicole Fitzpatrick  23:12  
yeah, no, I mean, I totally agree. I mean, you need to have to begin early to develop that long term evidence strategy. And I think it the pathway to market access isn't straightforward. I think it's very windy and you really need to, to start early to strategize around what are the key questions that you need to answer? And how are you going to use data to to answer those questions? And so I think starting even, you know, during your, you know, regulatory exploration in thinking about that market access downstream is very important. Yeah.

Tonya Dowd  23:45  
Great. All right, we're gonna go to the regulatory perspective.

Devjani Saha  23:50  
Regulatory real world evidence can be used for a number of applications, including supporting new indications. This, you can often see this when you are collecting real world data on a device that's marketed in the US and manufacturers are looking to support a clearance or approval for the US market. You can also use real world data to expand or modify indications, such as when you're going for something like a general tool claim to something more specific. For example, if there is a surgical tool that has very general claims to cut and dissect, but then you're really going for an indication that supports specific surgeries, you might be able to collect real raw data on the market at device in order to support those specific surgeries. You can also think about real world data as really complementing traditional clinical trials. So for example, you can use real world data as as a control arm for your clinical trial, that it can also be used to establish objective performance criterias and performance goals. This is especially useful when the incidences of certain events that define your performance goal and your performance criteria is rare. And you're really looking at that volume of data in order to understand the incidences of these rare events. As we all have seen, real world data is critical for postmarketing surveilance. And as we saw in the case study, real world data can really be used to reduce some of the burden associated with post market studies that often have direct to follow up patients. So finally, FDA has accepted real world data as a supplementary source of information and marketing submissions. So often, what you'll see in these submissions is a regulatory application that includes a mix of both data collecting from traditional clinical trials, as well as supplementary data and supporting data from real world evidence.

Tonya Dowd  26:07  
Great, great. Thank you, Devjani. So the next question, overall, what are the benefits and challenges of using real world evidence and real world data from each of your perspectives?

Gretchen Nelson  26:21  
I think from a clinical standpoint, the benefit and the challenge could also be the same thing in study design. Because in order to set up a study design to gain the benefits of real world data, you can have an easier study design, you can have a registry, you don't have to do an RCT, you can basically set up something where you're just collecting data and the cost is less. You can have it kind of going in the background, so it can be up to five years. But and then the benefit of it is that you're actually obviously gaining what's happening going on in the real world. And for everything that we've discussed today, you know, you can use it for your clinical trials, your claims, when you're out there, and your physicians are telling you, they're seeing the same thing. The challenges then also lie also in study design, because you have to function in where where do you get the real world data? How do you set it up? The HIPAA compliance with it, the signing of an informed consent in order to get the data, sometimes it doesn't come as easy to look at a clinical trial to collect the world real world data, when you're looking at the design of it from a point of of the HIPAA part of it. So it's challenging to collect the data and being able to submit to the FDA, if you're looking at it from different aspects, such that we've talked about as far as digital health or remote monitoring, or even patient health records, you know, all of that has crucial HIPAA information, and would require some form of informed consent in order to get the data. So it really lies in itself in the two. But I think as we move forward, we work through those challenges. It's only been a couple of years now that we've been focused on this. And we're constantly coming up with new ideas and new ways to design clinical trials. You know, we've seen a huge evolution in decentralized trials that are showing benefit, as far as, you know, keeping patients enrolled, follow up, and then being less expensive. But but overall, we are we are evolving into getting to where we need to be with this real world data and evidence.

Tonya Dowd  28:46  
Nicole, what are some of the benefits and challenges?

Nicole Fitzpatrick  28:49  
You know, I mean, I would say the benefits are really that there are possibilities. So I think that there's possibilities to answer a whole host of different questions. So you know, I mean, from the payer perspective, I think some things that come to mind are, you know, you could explore, you could begin to explore, you know, just the market in general. So what is your target population? And what are the, you know, demographics around that population? What's the patient journey, for instance, if you're looking even through claims, you can start to develop a patient journey based on a diagnosis code or procedure code, and begin to understand the downstream patient journey. And you can start to identify gaps right in the current standard of care. So if you're looking at the current standard of care, you can start to identify gaps where your solution may be able to fill that gap. And you can start to sort of get a picture of the whole target population and the gaps and begin to address that. And then alternatively, if your products already, you know, kind of out there and you're able to access data that has your product in it, you can start to do the same kinds of things. So understand the user demographics, and they may be very different than what you initially expected. For instance, and you can also see sort of that patient journey, you can understand, you know, provider or facilities. So there's there's almost like an endless amount of questions I think that you could ask. And I think similar to Gretchen, that's also the the challenge. So it's, it could be very challenging to try to figure out what are the right questions to ask and how do you appropriately leverage the real world data in order to, you know, develop a good strategy for for acquiring the evidence that you need to get that market access?

Tonya Dowd  30:30  
Yeah. So I think you made an important point, again, using real world data early on to inform really what that strategy is doing longitudinal claims analysis, that makes a lot of sense. Absolutely. Mina?

Mina Fahim  30:42  
Yeah, I mean, one of the challenges, right is the structured collection of data. And I mean, by by just saying the words, I am going to collect data, you're inherently introducing bias. And one of the things that we you know, we have to be careful about is when you define the type of data that you are going to collect, taking the bias away, so you don't falsely create an outcome or evidence from the data. And I think that's really important, whether you're talking about analysis by humans, or machines, it doesn't matter. That's very important. The other part of it, that's a challenge is sometimes you know, for us, for example, you we want to show efficacy, but that's a longitudinal process. Whereas some of the benefits, though, I mean, 3d printing, I think is a really interesting one, when you start talking about how we're it was, he really started out at the three level eventually now it's considered standard of care for some of the procedures like instructional heart, you have to will not have to its aggressive word, I'm sorry, but it's encouraged to do some levels of 3d printing to avoid patient prostheses mismatch. And 3d printing is incredible, because they were able to show that if you take a preoperative scan, and you start sizing devices against that 3d print, that the likelihood of embolization, the likelihood of paravalvular, leakage goes down. And that's incredible when you think about where 3d printed, 3d printing started, and where it is today. So I think that's a really exciting opportunity for those type of again, early identification of opportunity. But also thinking about, hey, you know, for us right now, well, I used to having that, like there, you know, the the use of evidence to demonstrate earlier and maybe simpler value propositions. I don't know if anyone knows this. I didn't until I joined many of you, but ergonomic injury due to musculoskeletal disorder in this country is $120 billion problem. And I thought that was a typo when I read the peer reviewed publication. So I called the authors of the paper, they said, Nope, and here's all the information we've gathered to come up with this, and our minimally viable product, right, there is a head to head study being done right now by one of our collaborators. I didn't even know this was a real thing. They're called an ergonomist. And they are showing that on quantitative and qualitative endpoints that the use of a system like ours, that you can look at a screen where you want to look at it actually helps with ergonomics, and down the line, could it have an impact on, you know, putting a small chip into $120 billion problem? Yeah, it can. So I think that's one of the one of the values is think about early wins, that are still strongly supported to get market access, with simple feature sets before really maturing and evolving to some of the other areas where, you know, it's maybe a more longitudinal claim that you're trying to make,

Tonya Dowd  33:52  
right? All fair points, we're going to hear from the regulatory perspective on the pros and cons.

Devjani Saha  33:58  
Now, really does understand the value of real world data. And they have put a lot of resources into it. They understand that, you know, it really helps them get a better idea of device performance in real world environments. And it allows for the collection of outcome data that may not be feasible in a traditional trial, including long term outcomes, and the ability to look at performance evaluated in a diverse patient population. And really the opportunity to partner with patients using wearable devices and mobile apps and the benefits to sponsors for in terms of reduction of time and costs. However, I think FDA has recognized really, that the use of real world data can be particularly challenging, given that there is an increased chance have data bias and residual confound. So one of the main hurdles of using real world data is really to ensure that there's appropriate mechanisms in place that the FDA agrees with to ensure that that data is really reliable and have sufficient quality to support a regulatory decision.

Tonya Dowd  35:22  
Great. Just Just one other question. Given the advent of remote patient monitoring digital health therapies, and the I think a more of an adoption and and need for real world evidence, do you ever? Does anybody ever see that becoming superior to traditional studies? For any technology? Gretchen,

Gretchen Nelson  35:47  
I don't, I think that there is still a need for RCTs. And I think that there's still a need to show that your device or your software or whatever you're, you're trying to market is better than the or just as good as what's already previously out there. So I think there's always going to be a place for RCTs I think the FDA is always going to want RCTs I don't think we're gonna get away from that. And PMA is in pivotal studies. So don't, possibly for 510 K's I mean, some of them don't even require clinical trials. Right. So I think we could get to a place where we're using a lot of that for 510s, especially if we're trying for reimbursement purposes or market claims. I think that's an avenue.

Nicole Fitzpatrick  36:35  
Yeah. Yeah, no, I feel similarly, I think there's always going to be a place for RCTs except except maybe in some of the 510 Ks, but where you need to show clinical efficacy. efficacy, I think it goes back to your point Mina that you have to control for bias. And you you know, that's the whole point of the RCT right is to sort of eliminate bias and isolate the effect of the solution. And so I think from that standpoint, you're always going to need RCTs.

Mina Fahim  37:04  
I think both of them said it perfectly. And they're the really the experts in this space. But I mean, I fundamentally agree, especially on what type of you know, what product class are you talking about? You know, you know, I think the other part that we haven't touched on is the leveraging mechanisms like IRBs, to inform the future evidence generation strategy is, I think, a very powerful thing to think about. So it's on the other side of the spectrum. So I'm going away from your question, right, from the RCT side. But no, I mean, I don't think for class three in efficacy, you got to show things work. And but like, from a PA, you know, from a 510 K perspective, you know, we've had some part of our strategy that will do some aspects of clinical trials and some of it that we won't, because of the combination of IRB data, of course, my red guy will say, make sure you put the claim, none of this is 510 K cleared, so you can't make any real world decisions on it yet. But, you know, you see indications of where, you know, trends are going on the 510 K site, and obviously, the bench. And if you can demonstrate, I think one of the things that has kind of been a theme I'll just say explicitly from our world is use condition analysis is a really important, I love I was set as you can build study design. But then what about the use condition in the real world, and these are non ideal scenarios that we're in from a clinical perspective, people make decisions for the best care of the patient that may be counter right to the study that was built, and that's good for the patient. And I think that's one thing that we have to keep in mind at the end of the day.

Tonya Dowd  38:40  
We've got a few more minutes, and we have some case studies I know you guys want to talk about. So Nicole, you wanna go first?

Nicole Fitzpatrick  38:46  
Sure, yeah. And I'll kind of go through this quickly, because I know we're running out of time. But this is really from the payer perspective. And so I'm going back to my previous life, when I was at, in the payer system at Highmark Health as part of the vital program. So we worked with a company that had a remote cardiac rehab program. And so here, the unmet need is really that cardiac rehab programs do decrease hospital readmission rates by about 25%. But but only about 10 to 20% of patients actually attend these programs in the brick and mortar setting. And so the solution that this company had was an evidence based digital cardiac rehab program that consisted of a care management service, a web based dashboard. They had a mobile app, they had some wearable and some interconnected devices. And really, the patients could do this at home. So they it was all remote. And then their challenge was really how do they obtain data that resonates with payers to drive coverage or reimbursement for their solution? And so, you know, within the payer provider system at Highmark, we were able to basically leverage claims data to to first understand the current state of cardiac rehab so we were able to explore what is really the opportunity by leveraging retrospective retrospective claims analysis to understand first the target population, so what what patients are eligible for cardiac rehab, and then you know what percentage actually attend to cardiac rehab and get some of those, the stats around that. And then look at that patient journey longitudinally for the standard of care, identify some of the gaps, and really use it to drive the hypothesis for the potential impact of this remote cardiac rehab solution. And so, you know, we were able to basically leverage this real world data as a starting point for this solution. And then we went forward and piloted the solution in the health system to actually validate and understand that impact. So we collected, you know, real world data across clinical, economic and experienced dimensions. And then, you know, we were able to validate some of the economic impact assumption. So here, we were looking at, you know, not just cardiac rehab participation rates with the new remote solution, but also downstream readmission rates as a result, and we were able to understand, you know, the impact of the solution, and essentially develop a real world evidence package for for the payer side,

Tonya Dowd  41:13  
Did it end up getting coverage and payment?

Nicole Fitzpatrick  41:15  
It's still in process, but the data looks lovely. Awesome. TBD.

Tonya Dowd  41:22  
All right. Gretchen you want to.

Gretchen Nelson  41:24  
So here's a case story from a regulatory perspective. I don't I'm this is Devjani world. And she'll chime in at the end here. But I wanted to set it up for everybody, because the FDA, as a condition of approval for the PMAs have requested post approval studies. And these are large studies that sometimes can be as much as 1500 patients, and five years of follow up, which in some of your guys's thought should be Wow, really expensive. Because the more sites you have, the more expensive your study is, the more patients you have, the more expensive your study is, and then the longer term follow up five years, this is a very expensive study. And with the requirement from the FDA is very challenging for a corporation not only on expense, but in what happens over that, that five year period, because they asked for so many patients, you're looking at a very long time for enrollment. So you could be looking at upwards of three to five years just for that enrollment phase of it, and then afterward, moving into the follow up. And what also happens with a five year follow up is struggles due to loss to follow up with these patients. Some of these patients are cardiac patients that may not even make it five years, and other of our pain patients who might feel great after they got their neurostimulator and walk away. And then we're never able to collect any of that data. So what ends up happening is that it doesn't represent the real world. And when we when the companies have gone to the FDA and showed their post approval results, it's not always where they expect it to be because of all of the struggles that they've been with. Devjani is going to come on now and talk about some ways to leverage world world real world data as far as this regulatory perspective.

Devjani Saha  43:18  
To address some of the challenges that Gretchen had highlighted with traditional post approval studies, some of these manufacturers actually leveraged a combination of real world data sources, including linking their own registries to a national death index to really assess the five year mortality of these patients. They also use national registries and claims data to determine the rate of device complications and hospitalizations, as well as get information on procedural complications. Finally, remote monitoring features of the device really allowed them to assess the long term integrity of the device. So if you looked at how they traditionally were doing these post approval studies, where they're really looking at direct to patient follow up, real world evidence completely reshaped how these sponsors were able to address their post, post approval requirements, and allow them to address questions related to the long term safety and effectiveness of their devices, in really the least burdensome manner.

Gretchen Nelson  44:25  
So I'll just sum that up with I don't think we're gonna get away from post approval studies because the FDA approves the PMA based usually on efficacy and some safety and then they really want that long term safety. But what they have said and what Tonya noted in 2016, is that we can use now real world data to suffice for that post approval study. So I think we're in a great place with where we're at being able to not have to spend so much time on post approval studies.

Tonya Dowd  44:55  
Great, thank you. Mina. I know you touched on this earlier but I'll make

Mina Fahim  44:59  
it quick. Okay. And what really our use case was in the world of atrial fibrillation, we started to understand, you know, okay, there's recurrence. And this gets back to the whole thing. I mean, the Kabana trial demonstrated, there's not my opinion that cardiac ablation can be as effective as antiarrhythmics. And if you live 3.4 years, it has a greater ROI over the lifetime of the patients. But again, actually delivering the therapy effectively and safely puts a big Asterix next to that. And what we found out is we actually start getting field complaints about catheters breaking in the field. And unfortunately, unlike our discussion today about early strategy around data collection, I had to manually mine 33,219. I remember it because I did it by hand and tallied the number of times the free text, free response text said catheter failed in the right inferior pulmonary valve or vein catheter failed in the left superior pulmonary valve. And what was crazy, is that when we started mining, the post market surveillance data, we found that the greatest rate of failure in the field per reported post market surveillance resulted in the right inferior pulmonary vein, which also lined up with many peer reviewed studies showing that the greatest gaps and ablation showed up in the right inferior pulmonary vein. So what we did is we started asking ourselves, why are these both? Are they Is there is there some sort of relationship between the two, and what we found out is when that catheter broke, you actually couldn't ablate the posterior ridge of the right inferior pulmonary vein. And what we were able to do is actually build and if anyone wants to nerd out with me about Monte Carlo analysis, please let me know. But basically, we were able to build a model, that was part of our corrective action for the Kappa, that demonstrated, okay, where this catheter is built its capability, its strength, when you use it in a certain way, it reaches its yield point and breaks. And what we were able to say is based on statistical analysis of 200, patient data set that we ran descriptive statistics on, we said, the there is a percentage of the patient population that has this really weird weird angle to go from, you know, the the puncture to the right inferior pulmonary vein. And that percentage of patient population actually had representative representative recurrence data, compared to our failure data that we captured and post market surveillance on. I was jumping up like a little kid, because I was like, I can't do this is actually and then when we the model that we trended, we followed it per in post market surveillance for the next two years. And we were actually within a quarter of a percent of prediction of where things failed. And it was a so that combinate really the summary of this is using the real world evidence in peer reviewed literature to understand efficacy, and therapy trends relative to product performance can impact how you design your solutions.

Tonya Dowd  48:13  
All right, thank you. I'm gonna ask the audience, if anyone has questions before we go to kind of final thoughts from the panel. You guys have all hung in there?

Audience Question  48:22  
Just from here? Yeah. What did you do? Another benefit would be understanding misuse of the product. Right in your world. Kind of back into your pretty much businesses out there, right.

Gretchen Nelson  48:42  
Yeah, I mean, I definitely think you have a point there, because you can also figure out, you know, where it's being used off label and where we want to study next, right? Are we seeing any results in an area that we think that we might be able to generate some data to get a PMA approval? In fact, the FDA is, is now even considering that when getting when we're releasing approval for new indications. They are okay, at some point looking at real world data or other ways that you found it to give a new indication. So that's very, very new. But yes, agree.

Tonya Dowd  49:22  
One other question,

Audience Question  49:25  
The catheter design or don't use it interesting to say,

Mina Fahim  49:31  
 No, I mean, so we don't follow that at all. I mean, we believe in the concept of robust design that you should design the product to be robust in a variety of use conditions that are on label and even if it's foreseeable misuse, how can if it's going to be used off label, how do you design for it, we actually ended up changing the aluminum structure the polymer material, the braid the geometry, and actually change the design of the catheter. And in the field, clinicians like that handle it handles a little bit differently. But for once you guys got something right. Where different actually means better this time. So that's hopefully answered your question. All right,

Tonya Dowd  50:09  
So we are getting the happy hour bell right now cut time time. So thank you all for joining. I know it was it was a difficult time. But thank you so much. I hope this was really informative for everyone. And we will be around the conference the next couple of days. So please reach out to us. Thank you. Thank you. Thank you.