Postmarket safety assessments of medical devices are critically important to ensure public safety after adoption of novel device technologies. Several recent high profile examples of unexpected medical device failures highlight the inadequacy of current systems to quickly identify problems that may ultimately lead to adverse events for patients.1 Regulatory agencies are faced with competing pressures to promote the highest safety standards but without stifling innovation or unnecessarily delaying access to new healthcare technologies for patients. It is often impractical to design premarket approval to confirm device safety, since the number of patients treated in device trials is typically quite small, and the patients are often highly selected (and therefore, not representative of real world practice). Given these limitations, regulatory bodies have used postmarket evaluations to assess medical device safety. Voluntary adverse event reporting has been a foundation of postmarket safety assurance, but reporting rates are extremely low (~0.5%) and there are no denominator data to adequately understand frequencies of specific adverse events.2 For these reasons, there has been tremendous interest in leveraging high-quality data sources, such as clinical registries, and engaging surgeons and proceduralists in research to support postmarket approval safety evaluations.3 However, whether it is feasible to use existing registries to support medical device safety surveillance has not been fully explored.
In this issue of BMJ Surgery, Interventions, & Health Technologies, the report of Banerjee and colleagues explores whether existing clinical registries can be used for postmarket safety surveillance and adequately detect differences in adverse event rates among implantable medical devices.4 Specifically, the authors focus on key attributes including whether registries are adequately powered, collect key outcome variables, and have sufficiently reliable capture of long-term follow-up. The analysis was performed focusing on four distinct medical devices. The investigators estimated the sample sizes that would be required, at various thresholds of statistical power, to detect a 50% increase in adverse event rates across a range of baseline rates. For the various devices and outcome pairs explored, the total number of patients required to follow within the registry ranged from 180 to 3000 depending on the prevalence of the outcome, length of follow-up, and expected performance thresholds. The fundamental result was that for the medical device and outcome pairs considered, the number of patients needed to follow was relatively modest and well within the scale of typical clinical registries. Therefore, registry-based postmarket surveillance appears to be feasible for a variety of devices, though the authors caution that the type and quality of data collected are the primary limiting factors for using registries in this manner.
Essential to the report of Banerjee and colleagues, it is important to understand a priori whether the data source can confidently be used to detect outcome differences. This fundamentally depends on both adequate sample size and reliable collection of appropriate variables. The sample size should be large enough for investigators to be confident that differences which are detected truly exist and are not due to chance. Statistical power is defined as the probability of detecting a difference when there is a true difference between event rates. This relationship relies on several variables beyond sample size alone, including the gold standard event rate, estimated minimum detectable difference in event rates, and a prespecified significance level. Determining sample size is a function of power, estimated event rates, and prespecified significance level. Investigators are additionally tasked with accounting for potential losses to follow-up, missing data, the need and implications of multiple testing (such as looking at data at multiple time points) and the need to conduct subgroup analyses when performing sample size and power calculations.5
Having an adequate number of patients is necessary but hardly sufficient to conduct registry-based studies. Registries should collect both appropriate outcome variables of interest, as well as comprehensive covariates that will permit multivariable risk adjustment that is necessary in most observational studies to reduce confounding. As highlighted by Banerjee and colleagues, in addition to traditional clinical outcomes, reliable patient-centered outcomes should be considered when developing medical device registries. Additionally, existing registries are often constrained by short duration of follow-up and may be limited to in-hospital events. The ability to link registries to additional data sources, such as Center for Medical Service data, has been validated and shown to be feasible for long-term follow-up of clinical outcomes. This is essential for considering key outcome measures such as rehospitalization rates and 30-day, or longer, and mortality.
The Society of Thoracic Surgeons (STS)/American College of Cardiology (ACC) TVT Registry provides an excellent example of a large, national registry which was successfully developed to monitor the safety and efficacy of various transcatheter valvular procedures and devices. The registry was formed in 2011 prior to the approval of the first transcatheter aortic valve device in the USA.6 The registry was designed with input from multiple stakeholders, including the medical device industry, regulatory and reimbursement agencies, clinicians, hospitals, patients, and researchers. The registry collects preprocedure, procedure, 30-day, and 1-year postprocedure data. The Transcatheter Valve Therapy (TVT) registry provides hospitals with quarterly feedback on results with national benchmarks which facilitate improving care. Additionally, the FDA receives comprehensive performance data reports which allow monitoring of device safety. Overall, the STS/TVT Registry is an example of the power of a well-planned registry, meeting the requirements for postmarket surveillance, while supporting the primary purposes of quality assessment, clinical improvement, and device innovation.
Such examples support the notion that registry-based surveillance of medical devices can be successfully performed, and the current study suggests that the number of patients required to adequately power such analyses is achievable. We believe that to best leverage medical device registries, surveillance should be performed as close to real time as possible and ongoing within the accruing dataset.7 8 This approach has been shown to be feasible to allow early detection of important safety signals of a variety of contemporary medical devices leveraging selected cardiovascular clinical registries.9 10 If developed properly, using registries will complement existing strategies and will efficiently improve medical device safety evaluations order to best protect consumers.
