Building Blocks for the Long-acting and Permanent Contraceptives Coordinated Registry Network

Introduction

The evaluation of medical devices throughout the total product life cycle (TPLC) is often limited by the lack of interoperable real-world data (RWD) sources. Between 2012 and 2013, the Center for Devices and Radiological Health (CDRH) of the Food and Drug Administration (FDA) released a plan to strengthen the postmarket infrastructure for the monitoring of medical devices.1 2 This plan has evolved into the implementation of the National Evaluation System for health Technologies (NEST), to more efficiently generate better evidence for medical device evaluation and regulatory decision-making throughout the TPLC by strategically and systematically leveraging real-world evidence and applying advanced analytics unique to medical devices.3 In 2016, the Medical Device Innovation Consortium established the NEST coordinating center (NESTcc) to promote collaborations and oversee activities.4

In 2014, the National Registry Task Force was established as part of the implementation of the CDRH initiative to ‘strengthen the medical device postmarket surveillance system using existing resources and under current authorities and to develop an integrated system that efficiently and effectively achieves its basic functions, from timely identification of postmarket signals to facilitating premarket device clearance and approval’.5 The concept of Coordinated Registry Networks (CRNs) was introduced by the National Registry Task Force in 2015, as no single database exists that can answer all the questions related to the performance of a medical device, indicating an opportunity for databases to be augmented by linkages to other data sources.6 The purpose of a CRN is to develop ‘strategically partnered electronic health information systems that support (1) the implementation of structured device identifiers, core minimum data elements and definitions and (2) the ability to share complementary data across information systems’.6 There are several existing CRNs and other data networks (virtual CRNs) rapidly developing into important data sources for NEST. To support their maturation, the Medical Device Epidemiology Network (MDEpiNet), one of the NESTcc data network collaborators, has worked towards the establishment of the Community of Practice (COP).7 Through this COP, robust CRNs are being developed to serve as a productive node within NEST.

To advance the data infrastructure and capacity to conduct patient-centered outcomes research, the FDA partnered with the Office of the National Coordinator (ONC) for Health Information Technology, the National Library of Medicine (NLM) and a multistakeholder community coordinated by MDEpiNet in a large project funded by the Patient-Centered Outcomes Research Trust Fund administered by the Office of the Assistant Secretary for Planning and Evaluation of the US Department of Health and Human Services.8 In doing so, stakeholders identified the need to create a CRN that would be capable of generating RWD to evaluate the safety and effectiveness of long-acting and permanent contraceptives (both drugs and devices).

Stakeholders acknowledged the importance of creating a core minimum data set that would be applicable to both existing long-acting and permanent contraceptives, and new contraceptives that would enter the market in the future. Core data elements, relevant to reproductive health, that are captured across healthcare systems can help address clinical concerns and questions that arise with respect to contraceptive technologies used in the general population. As a first task, we used the Delphi survey method to reach consensus on a core minimum dataset for the study of current and future contraceptive medical products.

Materials and methods

On 15 September 2017, stakeholders including the FDA, industry, non-profit organizations, patient advocacy group members, payers, professional society leaders, academia and clinical experts met to initiate the CRN.9 At this meeting, the current landscape of data sources for evaluating long-acting and permanent contraceptives were discussed along with stakeholder perspectives. Breakout sessions were held for stakeholders to discuss data elements that should be included in the Delphi survey process to build the core minimum data set.

In the fall of 2017, we established the long-acting and permanent contraceptives working group. This working group included two cochairs from the FDA, two external clinical cochairs and 19 representatives from the American College of Obstetricians and Gynecologists, the Society of Family Planning, American Association of Gynecologist Laparoscopists, FDA (CDRH and Center for Drug Evaluation and Research) and the Centers for Disease Control and Prevention. Working group members were selected by asking each of these stakeholder groups to recruit volunteers to participate in the working group. During this process, the FDA cochairs reviewed incoming participant suggestions to ensure adequate diversity and representation of various stakeholder perspectives. The full list of working group members is reported in online supplemental file 1. Initially, this working group created an initial set of about 300 data elements that were drawn from the meeting recommendations, a review of the literature, regulatory requirements and existing research efforts. The working group streamlined that list to 95 data elements relevant to contraceptive outcomes.

The Delphi survey process was then initiated to identify the core minimum data elements for long-acting and permanent contraceptives. The Delphi survey method is a group decision-making technique that was developed by Norman Dalkey and Olaf Helmer in the 1950s as a way of addressing a specific military problem.10 Traditionally, group decision-making relies on a consensus panel approach that brings experts together and let them discuss an issue until a consensus emerges. Challenges with this traditional approach are that one person with a strong personality can have a large effect on the decision and a lack of anonymity may introduce response bias. The Delphi survey method was developed to retain the strength of a joint consensus, while removing the potential bias from group dynamics and face-to-face responses. With the Delphi survey method, group input is received through a series of anonymous surveys, which are sent to a preselected group of experts. The questionnaires are answered anonymously and individually by each member. Each survey also provides an opportunity for group members to introduce new options and suggestions in between rounds. Results of each survey round are collected, collated and analyzed by a design team. This process is repeated until a group consensus is reached.11–14 Most of these methodologies are standardized for CRN purposes and also used in maturity model framework development,15 and core minimum dataset development for pelvic organ prolapse,16 uterine fibroids,17 peripheral artery interventions,18 and prostrate ablation."19

The focus of the Delphi survey process was to identify a minimum set of data elements relevant to safety concerns for contraceptive technologies that we hypothesized would be either easy to obtain from most electronic health record (EHR) systems or to be entered by clinicians. With these in mind, the Delphi survey process focused on the following categories: (1) patient characteristics, (2) medical and surgical history, (3) procedural data, (4) relevant medications and (5) patient outcomes. Two rounds of surveys were designed and administered by the analysis team at Weill Cornell Medicine and sent to the expert panelists through a secure anonymous online questionnaire (https://www.surveymonkey.com). In each round, we retained data elements that had strong consensus (eg,>50%) and dropped responses that had poor consensus (eg, <40%). For results with moderate consensus, between 40% and 50%, the cochairs discussed considerations and either raised concerns to group members for discussion or revised them to achieve clarification based on respondents’ concerns. Group members were then resurveyed. Consensus was achieved after a total of two rounds were completed.

A patient partner with knowledge of and experience with contraceptives was chosen through a formal standardized selection process and invited to join the working group to ensure that the core minimum dataset reflected the needs and interests of the patient community. The patient partner, selected through this process in spring 2018, reviewed and approved the core minimum data set and currently serves in a volunteer capacity by participating in working group meetings to provide input on patients’ perspectives.20

Results

The Delphi survey method was implemented over a 6-month period from February 2018 to July 2018. The participation rates in the first and second round of the Delphi survey were 83% and 100%, respectively. Completion of the Delphi survey process resulted in 121 data elements identified as relevant to long-acting and permanent contraceptives. Online supplemental appendix 1 lists and categorizes the data elements into medical history, procedure, medications and end points during and after treatment. The level of consensus for both the final data elements and the dropped data elements is reported in online supplemental files 2 and 3. We did not include patient demographic variables in the Delphi surveys. Instead, an existing standard, harmonized set of demographic variables were selected for the core minimum data set that are based on work already conducted by a multistakeholder project sponsored by the Pew Charitable Trusts.21 A critical deliberation that occurred in the group when selecting variables involved considering data that can be extracted from current record systems versus data which needed direct input from the physicians. The general consensus was to minimize unique variable input that would not otherwise be included in routine documentation. The intent is that with data extraction from EHR discrete fields, there will be improved data integrity.

Regarding the reproductive/gynecologic history, consensus among the group was to include not only pregnancy history but other gynecologic historical elements. These variables are believed to be important not only for their influence in provider decision-making regarding contraceptive options but also to determine the potential longitudinal impact that the technology may incur. Some of these items were specifically chosen not only to address issues that have been identified with prior technologies, such as infection or menstrual abnormalities, but also included several broader-based items such as breast and gynecological cancer that may be influenced by these products.

As a contraceptive registry, consensus around the primary end point of pregnancy was easily achieved. Much of the discussion in the group was regarding appropriate capture of variables that will be meaningful for current technologies but robust enough to cover future contraceptive product development. Elements such as thermal injury and perforation needed to be carefully considered for the potentially wide range of contraceptive products. Ultimately, the variables were individualized for the different classifications of long-acting reversible contraception/sterilization such as hysteroscopic sterilization versus contraceptive implants. An important discussion included the use of confirmation imaging for procedures that require or would voluntarily be utilized.

Informatics work

The ultimate goal of the overall Women’s Health Technology CRN (WHT-CRN) project was to create a CRN for numerous women’s health conditions. As such, core minimum data sets were concurrently developed for pelvic organ prolapse, stress urinary incontinence, uterine fibroids as well as long-acting and permanent contraceptives. In order to create a CRN capable of evaluating contraceptive medical product devices used for all of these conditions, it was imperative to harmonize common data elements (CDEs) among all of the clinical areas to ensure interoperability of data sets stemming from future registries. After the core minimum data sets were developed for each individual condition, the WHT-CRN informatics team, consisting of representatives from FDA informatics, NLM and ONC, created the core minimum set of harmonized data elements. First, the informatics team compared terms across each clinical registries’ data elements and questions. They identified and grouped common concepts that occurred in at least two of the registries. Together, the informatics team and clinical working groups reviewed and refined the common concepts in an iterative, collaborative process. During this process, they harmonized individual data element definitions, questions and permissible values, until they arrived at an initial core minimum set of harmonized data elements.

The informatics team drew the permissible values from literature, clinical working group feedback and standard clinical vocabularies, including the Value Set Authority Center (VSAC), which is provided by NLM in collaboration with ONC and the Centers for Medicare & Medicare Service (CMS).22 VSAC is a repository of codes and terms from NLM-hosted standard clinical vocabularies (LOINC, SNOMED CT and others) and an authoring tool for public value sets. The WHT-CRN informatics team created VSAC grouping value sets, that is, lists of several value sets that share a common purpose and similar clinical concepts and are identifiable with VSAC object identifiers.

The initial core minimum set of CDEs was delivered to ONC for inclusion in the WHT-CRN Health Level Seven International Fast Healthcare Interoperability Resources (FHIR) Implementation Guide (IG).23 The WHT-CRN FHIR IG focuses on capturing data related to women’s health and making that data available for exchange to both providers and authorized researchers. In the WHT-CRN FHIR IG, the CDEs are mapped to FHIR resources and profiles. The IG also includes the recommended value sets, capability statements and implementation guidance for developers. The IG leverages the US-Core IG, Structured Data Capture FHIR IG, Data Access Framework for Research IG and the Patient-Reported Outcomes FHIR IG.

ONC partnered with two sets of organizations to develop and pilot test the structured framework and technology described in the WHT-CRN FHIR IG.24 The data collection instruments (CRN instruments) were defined by these CDEs to test six CRN capability statements, which outline conformance requirements (specific profiles, operations, security, search parameters, etc) for each real-world system. The six capability statements are (1) ability to publish a CRN instrument; (2) ability to retrieve the instrument, render the instrument and collect the necessary data; (3) ability to retrieve, render and auto-populate the CRN instrument and collect additional data; (4) ability to retrieve and render the CRN instrument and collect data and transform data into FHIR resources; (5) ability to receive CRN instrument and collected data; and (6) ability to receive CRN instrument, collected data and other FHIR resources. Please see figure 1a + b for illustrations of how the WHT-CRN data will be collected and accessed by organizations. Please see online supplemental file 4 for further clarification on these concepts and a full list of acronyms used in this manuscript.

Request permission

Figure 1

The abstract model, actors, and the data flow for WHT-CRN data collection Figure Legends: Figures 1a and 1b detail the capabilities required to implement a WHT-CRN workflow from the point of data collection to access of that data for research. The abstract model for collecting WHT-CRN data focuses on collection from patients undergoing various treatments of interest using a combination of clinical care delivery systems like EHRs and independent apps. The abstract model for accessing collected data from women’s health registries focuses on the ability of researchers to access the data currently collected and persisted in the registries.

Data elements from the clinical working groups that were not included in the core minimum harmonized set in the IG are nonetheless valuable. For example, data collection instruments may contain CDEs from the core minimum set as well as non-core data elements. The non-core data elements should be equally well specified, that is, uniquely identifiable, clearly defined, with clear provenance, and linked to standardized terminology. The WHT-CRN informatics team has identified and begun defining these elements in the data dictionaries, which will complement the core minimum set.

Discussion

Single-purpose registries face many challenges in addressing questions involving multiple therapies and conditions. By leveraging fewer resources to collect predefined data for a greater number of conditions and therapies, this CRN has a potential to improve real-world evidence generation while saving time and reducing cost. We envision this CRN demonstrating that data from different sources (eg, EHRs, registries) may be used to: (1) evaluate the effectiveness, quality of life and safety associated with differing treatment options; (2) provide a framework for clinical studies to be conducted within the infrastructure, including industry-sponsored studies required to fulfill FDA’s request for premarket and/or postmarket regulatory activities and (3) allow healthcare providers to track physician volume, patient outcomes and quality measures for quality improvement activities and fulfill upcoming CMS, Merit-Based Incentive Payment System (MIPS) and maintenance of certification requirements.

By using a Delphi survey method that engaged multiple stakeholders with varying perspectives, we were able to achieve consensus on a minimum data set of variables capable of evaluating the performance of contraceptives. One of the key steps in creating the CRN for contraceptives was identifying a core data set that would require minimal data entry by clinicians but would also be comprehensive enough to conduct safety and effectiveness studies. It was also important that the core data set would be well suited for new contraceptive technologies that enter the market in the future. Our stakeholders foresee many applications for new contraceptive devices and technologies in the coming years, and we believe that the data elements captured in this CRN are generalizable to both existing devices and novel technologies that may enter the market in the future.

We recognize that our core elements are limited to documentation in EHRs and may not fully reflect the experiences of patients. Our measures are more reflective of safety concerns and less reflective of patient experience. For example, our post-procedure data elements largely reflect concerns for adverse outcomes like surgical complications and unintended pregnancies. However, we do not capture the experiences of ongoing users or reasons for discontinuation such as low libido or dyspareunia. Additional work is planned to develop the data elements that reflect the experiences of the patients beyond the documentation in EHRs.

There are limitations to our methodology to identify core data elements for long-acting and permanent contraceptives. While the Delphi survey method overcomes some of the limitations of group consensus, it is still possible that input was not received from all participating stakeholders. Our process relied on voluntary time from working group members, and there were variable rates of participation in each round. Furthermore, even among those participating, if members were not represented in each round (eg, follow-up phone calls for consensus), certain data elements may have been missed although there was 100% participation in the second round to reduce that concern. Additionally, we recognize that concerns regarding long-acting contraceptives differ by method (implant vs intrauterine system) and are often different than concerns for permanent contraceptive methods. Nevertheless, we leaned towards an inclusive approach to acknowledge these varying concerns. Strengths of our methodology include the use of an iterative approach through two survey rounds and the incorporation of perspectives from varying stakeholder groups. Overall, compared with a traditional consensus panel approach, the Delphi survey method strengthens our confidence in the results by eliminating the potential bias from group dynamics and face-to-face responses.

The consensus on the core minimum data set represents a unified effort by multiple stakeholders. The list of identified data elements is relevant not only for the CRN focused on contraceptives but also for other researchers tasked with examining large databases, both prospectively and retrospectively with respect to clinical outcomes. The core minimum data elements will also improve data monitoring by supporting the quicker identification of clinical concerns and potentially avoiding poor patient outcomes. Additionally, widespread use of the core minimum data elements will support MDEpiNet in its endeavor to speed the development and advancement of CRNs as a robust source of real-world evidence for NEST. The next step of this project will be to collect these data elements through a national infrastructure. This will accrue unbiased and high-quality data on devices and technologies used for contraception in the context of a multipurpose CRN. Finally, the data collected by this CRN have the potential to address past and current strategic priorities of FDA, particularly of increasing the use of real-world evidence, engaging patient partners and promoting collaborative communities.25

Appendix