Objectives The accumulation of data through a prospective, multicenter Coordinated Registry Network (CRN) could be a robust and cost-effective way to gather real-world evidence on the performance of pelvic organ prolapse (POP) technologies for device-based and intervention-based studies. To develop the CRN, a group of POP experts consisting of representatives from professional societies, the Food and Drug Administration, academia, industry, and the patient community, was convened to discuss the role and feasibility of the CRN and to identify the core data elements important to assess POP technologies.
Design A Delphi method approach was employed to achieve consensus on a core minimum dataset for the CRN. A series of surveys were sent to the panel and answered by each expert anonymously and individually. Results from the surveys were collected, collated, and analyzed by the study design team from Weill Cornell Medicine. Questions for the next round were based on the analysis process and discussed with group members via conference call. This process was repeated twice over a 6-month time period during which consensus was achieved.
Results Twenty-one experts participated in the effort and proposed 120 data elements. Participation rates in the first and second round of the Delphi survey were 95.2% and 71.4%, respectively. The working group reached final consensus among responders on 90 data elements capturing relevant general medical and surgical history, procedure and discharge, short-term and long-term follow-up, device factors, and surgery and surgeon factors.
Conclusions The CRN successfully developed a set of core data elements to support the study of POP technologies through convening an expert panel on POP technologies and using the Delphi method. These standardized data elements have the potential to influence patient and provider decisions about treatments and include important outcomes related to efficacy and safety.
- device evaluation
- urology devices
- women's health
- real world evidence
Data availability statement
Data are available on reasonable request. The data that support the findings of this study are available from the corresponding author, CEB, on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
WHAT IS ALREADY KNOWN ON THIS TOPIC
There are a variety of data sources used to evaluate the safety and effectiveness of transvaginal mesh for pelvic organ prolapse (POP) repair; however, these data sources are not harmonized and interoperable.
Regulators, clinicians, patients, and manufacturers identified the need to create a Coordinated Registry Network (CRN) that would be capable of generating real-world data to evaluate the safety and effectiveness of technologies, including mesh implants, used for POP.
WHAT THIS STUDY ADDS
Regulators, clinicians, patients, and manufacturers were engaged to conduct a Delphi process to reach consensus on the core data elements needed to create the CRN.
Stakeholders identified 90 core data elements that would be applicable to both existing and new POP technologies that would enter the market in the future.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE AND/OR POLICY
Core minimum data will help harmonisation of data sources and provide interoperability of existing real-world data sources.
This will enable us to collect high-quality data and conduct active surveillance on devices and technologies used for POP.
Since its first Food and Drug Administration (FDA) clearance in 1996, there has yet to be consensus about the safety and efficacy of transvaginal mesh for pelvic organ prolapse (POP) repair. Numerous studies found that transvaginal POP mesh kits had higher short-term rates of success, and had higher rates of surgical complications and postoperative adverse events.1–13 This uncertainty about the safety and effectiveness of transvaginal mesh for POP was accompanied by an increasing number of adverse events reported to the FDA. In fact, the FDA reported that it received almost 4000 reports of complications between the years 2005 and 2010, including reports of mesh erosion, infection, pain, urinary problems, and recurrent prolapse.14 Since 2011, more than 100 000 product liability claims have been filed by patients with complications from the treatment of POP or stress urinary incontinence (SUI).15 After issuing a number of public health notifications and regulatory actions specific to transvaginal POP mesh,14 16 17 in July 2018, the FDA ordered manufacturers of transvaginal mesh intended for POP repair in the posterior compartment to stop selling and distributing their products.18 Less than a year later on April 16, 2019, the FDA also ordered manufacturers of transvaginal mesh intended for POP repair in the anterior compartment to stop selling and distributing their products.19 Based on its review of the available evidence, the FDA has not taken regulatory actions to remove or limit the use of mesh for SUI repair.
Given this longstanding controversy surrounding transvaginal mesh for POP, we identified the need to create a Coordinated Registry Network (CRN)20 that would be capable of generating real-world data to evaluate the safety and effectiveness of technologies, including mesh implants, used for POP. As a first task within this CRN, we used the Delphi method to reach consensus on a core minimum dataset for the study of current devices and surgical interventions for POP, including transabdominal mesh and biological implants, as well as other devices that enter the market in the future.
On September 15, 2017, stakeholders from the FDA, industry, non-profit organizations, patient advocacy groups, payers, professional society leaders, academia, and clinical experts met at the FDA in Silver Spring, Maryland, to kick-start the CRN by discussing the current landscape of registries evaluating pelvic floor disorder technologies and the perspectives of each stakeholder. In the afternoon, a session was held for stakeholders to propose data elements that should be included in the Delphi process to build the core minimum dataset. Over the following months, the POP working group created an initial set of data elements based on those identified at the September meeting, a review of the literature, regulatory requirements, and existing research efforts.
The Delphi method is a group decision-making technique developed by Olaf Helmer and Norman Dalkey of the Rand Corporation as part of an Air Force study in the early 1950s.21 The standard group decision-making technique, the consensus panel approach, brings experts together in a room and lets them discuss an issue until a consensus emerges. Challenges with this approach are that one person with a strong personality can have a large effect on the decision, and a lack of anonymity may introduce response bias. The Delphi method was developed to retain the strength of a joint consensus, while removing potential bias from group dynamics and face-to-face responses. With the Delphi method, group input is received through a series of anonymous surveys, which are sent to a preselected group of experts. The questionnaires are answered anonymously and individually by each member of the group. Each survey also provides an opportunity for group members to introduce suggestions. Results of each survey round are collected, collated, and analyzed by a design team. This process is repeated until a group consensus is reached.21–24 Most of these methodologies are standardized for CRN purposes and also used in maturity model framework development,25 and core minimum dataset development for sterilization,26 uterine fibroids,27 peripheral artery interventions,28 and prostrate ablation.29
The Delphi process for the POP working group was initiated and completed over a 6-month time period from February to August 2018. Two rounds of surveys were designed and administered by the analysis team at Weill Cornell Medicine and sent to the expert panelists through a secure anonymous online questionnaire. The first-round Delphi survey results were analyzed by the analysis team and discussed in a series of conference calls with the working group co-chairs. Any variable receiving <50% consensus was removed from the list of data elements, with all others retained for the second-round survey. These results and open response suggestions were presented and discussed with the full working group until consensus among respondents was achieved on how to proceed. The analysis team incorporated the results of this discussion into the design of the second-round survey and subsequently distributed the survey to the working group. The same process was repeated until the consensus was achieved on the final minimum core dataset in August 2018.
To ensure that the core minimum dataset reflected the needs and interests of the patient community, a patient partner with knowledge of and experience with POP was sought and chosen through a formal standardized selection process to join the working group and provide input on the core data elements. A public call for patient partners was posted online and a selection committee was assembled to evaluate and interview applicants. The POP patient partner, selected through this process in summer 2018, reviewed and approved the core minimum dataset and currently serves in a volunteer capacity by participating in working group calls and meetings and providing input on patients’ perspectives.
The POP working group consisted of 21 expert members, all POP specialists from Female Pelvic Medicine and Reconstructive Surgery, including both gynecology and urology backgrounds and technology backgrounds. The full list of working group members is reported in online supplemental file 1. The working group co-chairs reduced an initial list of >300 potential data elements to 120 data elements that were included in the Delphi process. Participation rates in the first and second round of the Delphi survey were 95.2% and 71.4%, respectively. Completion of the Delphi surveys resulted in a consensus among the responders on the selection of 90 data elements identified as relevant to POP surgical devices (online supplemental file 1). The data elements were grouped in the following categories: (1) medical history; (2) surgical history; (3) examination; (4) procedure; (5) discharge; (6) short-term follow-up (0–30 days); (7) long-term follow-up (>90 days); (8) device factors; (9) surgery factors and (10) surgeon factors. Patient demographic variables (age, race, etc) were not included in the Delphi selection process as a standard, harmonized set of demographic variables were selected based on work already conducted by a multistakeholder project sponsored by the Pew Charitable Trusts.30 Subjective measures were not included as a working group has been tasked with a means to collect patient reported outcomes to be included in the registry. The level of consensus for both the final data elements and the dropped data elements are reported in online supplemental files 2 and 3.
Medical and surgical history
For the patient’s medical history, there was consensus among participants to capture overall parity, vaginal births, cesarean births, smoking status, menopause, sexual activity and pain, chronic constipation, estrogen therapy, and vaginal bulge symptoms. Lower urinary tract symptoms included mixed, stress, and urgency urinary incontinence. These were determined to be particularly important by the group as they represent risks for worsening POP or potential sequalae of POP repair. The comorbidities are best captured using a combination of the American Society of Anesthesiologists (ASA) physical status classification31 and including specific comorbidities such as diabetes mellitus. Prior surgical history included prior hysterectomy, prior mesh use, prior anti-incontinence surgery, prior prolapse surgery, and prior abdominal surgery. For physical examination parameters, body mass index, Pelvic Organ Prolapse Quantification (POP-Q) stage, and compartment with the greatest anatomic prolapse were included. This would allow for comparisons among surgical repairs as higher degrees of prolapse are more likely to result in recurrence. The hymen is an important ‘cut-off point’, as women with prolapse beyond the hymen have more pelvic floor symptoms and are more likely to report a vaginal bulge than women with prolapse above the hymen.32–35 POP stages can also be evaluated using the Baden-Walker, a clinical system that grades the degree of prolapse from 0 to 4 in three different pelvic compartments. However, this was not chosen for inclusion. All members agreed that the POP-Q assessment is the most accurate and reproducible way to measure prolapse and has the highest intersurgeon reliability.36
Procedure and discharge
Surgery date, total operative time, ASA status, concomitant hysterectomy, concomitant anti-incontinence procedure, and mesh use were determined to be core data elements related to procedure. Furthermore, the type of vaginal or abdominal apical vault suspension used, if anterior, enterocele, or posterior repair was performed, and if an obliterative procedure was performed were included. This is of particular importance since both transvaginal and transabdominal routes are commonly used and each has its own risk profiles. The complications include bleeding requiring transfusion, ureteral injury, urethrotomy, vascular injury, visceral injury, mesh kit trocar injury, other operative complication/injury, aborted procedure, conversion to laparotomy, device malfunction, and death. The most severe complication would be classified using the Clavien-Dindo classification system37 to grade the level of severity of the adverse event and to capture the need for subsequent therapy. Given that a number of complications such as suture exposure or erosion may be managed in either the office, operating room, or emergency room, the Clavien-Dindo system allows for discrimination between interventions performed with and without general anesthesia. It is important to also capture re-operation during index hospitalization, discharge date, and discharge disposition.
Short-term and long-term follow-up
For short-term follow-up (<30 days), there was consensus to capture early complications which include cardiovascular complications, pulmonary complications, systemic infections, bleeding, organ injury, suture exposure or erosion in the vagina or viscera, foreign body or death. For short-term follow-up (31–90 days), complications to be collected include vaginal scarring, shortening, suture exposure, erosion, mesh exposure or erosion, pelvic pain, dyspareunia, fistula, bowel injury, thrombolic event, cardiac event, pulmonary event, and neurovascular event. For long-term complications (>90 days), these would include the same elements as 31–90 days complications with the addition of symptomatic or anatomic recurrence. If there is a recurrence, it is appropriate to grade using the POP-Q system as well as reporting the compartment with the greatest anatomic prolapse. It is important to capture readmissions within 30 and 90 days. For all short-term and long-term complications, the most severe complication would be graded using the Clavien-Dindo scale.
To accurately capture the device used for prolapse repair, the device identifier of the unique device identifier (UDI-DI)38 would be collected for any implant or suture used in a procedure. One or more of the production identifiers of the UDI (UDI-PI) would also be included if they appeared on the device label. The parts of the UDI-PI are: lot, serial number, expiration date, manufacturing date, distinct identification code. In addition, the UDI-DI would be used to pull data from AccessGUDID to auto-populate the company name, brand, clinically relevant size, device type (eg, absorbable, permanent), and other relevant device identification characteristics with master data across all registries.39
Surgery and surgeon factors
Surgery level data elements include practice type, center or hospital identifiers, hospital volume, and whether there was trainee involvement. Surgeons would be classified based on the National Provider Identifier, age, level of training, specialty, board certification, subspecialty certification, and surgeon volume. The panel agreed that surgical errors and a learning curve likely contribute to the rate of complications.40 The learning curve in surgery is a well-defined phenomenon in many other clinical areas and most of these studied operations are fairly large and complex.41 There are very few, if any, high-quality studies looking at the role that the learning curve plays in POP surgery. In the realm of stress incontinence, which has not been subject of FDA action, Welk et al,40 measured the incidence of mesh removal or revision after sling mesh procedures in 59 887 women over a 10-year period. The authors found that 1307 women (2.2%) underwent mesh removal or revision a median of 0.94 years after receiving a mesh implant for SUI. Patients of high-volume surgeons (75th percentile of yearly mesh-based procedures) had a significantly lower risk for experiencing reintervention (removal or revision) with no difference found among specialties (urologist vs gynecologists). These findings support the group’s decision to include surgeon and facility level data for POP and that a learning curve exists.
Achieving consensus on the core minimum dataset for POP was an important first step in our goal of creating a CRN for numerous women’s health conditions (WHT-CRN). Core minimum datasets were concurrently being developed for SUI, uterine fibroids, sterilization, and long-acting and reversible contraception. In order to create a CRN capable of evaluating medical devices used for all of these conditions, it was imperative to harmonize common data elements among all of the clinical areas to ensure interoperability of datasets stemming from future registries. As such, the informatics team compared, identified, and aggregated data elements under common concepts that occurred in at least two of the clinical areas for harmonization. The common concepts were intended to uncover potential gaps. Concurrently, a search was done for the common concepts using the National Institutes of Health (NIH) Common Data Elements (CDE) Repository, a platform that enables linkage of data elements to existing standards and terminologies and acted as input for initial modeling of the unique data elements. Codes were drawn from standard clinical vocabularies and other resources, such as the Value Set Authority Center, which is a repository of codes and terms from LOINC, SNOMED CT, International Classification of Diseases (ICD)-9, and ICD-10, among others and an authoring tool for public value sets. We also gathered potentially relevant Current Procedural Terminology (CPT), Healthcare Common Procedure Coding System (HCPCS), ICD-9, and ICD-10 codes for each clinical condition through online searches and referencing the corresponding codebooks. In addition, the potential UDI-DI for each of the devices used in a clinical area were validated against AccessGUDID. The data elements were modeled using forms associated with each corresponding clinical group in the NIH CDE Repository. Each form contained the full set of harmonized set of data elements and associated permissible values linked to standardized codes and the data elements required in appropriate Health Level Seven International (HL7) profiles.
The WHT-CRN Implementation Guide (IG) builds on interoperable data exchange standards such as the HL7 profiles that can be used to define WHT-CRN data infrastructure. The WHT-CRN IG focuses on capturing data related to women’s health devices and making that data available for exchange to both providers and authorized researchers. The informatics team worked with one existing women’s health registry to pilot the WHT-CRN Fast Healthcare Interoperability Resources (FHIR®) IG and its underlying standards and datasets in a test environment. The input from that testing environment will be used to inform work with other WHT-CRN test and production environments (eg, clinical or provider settings) and/or manufacturing setting. Please see figure 1A,B for illustrations of how the WHT-CRN data will be collected and accessed by organizations. An additional outcome of testing will be to evaluate the ability to capture UDI from scanning the device label, extracting electronic health record data or using manual entry and using the UDI-DI to auto-populate structured device data. The usability of the data in AccessGUDID will be evaluated to ensure that this core data is meeting the goal of providing standard device identification data that can be used to inform regulatory decision making. Please see online supplemental file 4 for further clarification on these concepts and a full list of acronyms used in this manuscript.
Single-purpose registries face many challenges in addressing questions involving multiple therapies and conditions. By leveraging fewer resources to collect predefined data for a greater number of therapies, the CRN has the potential to improve real-world evidence generation for POP technologies while saving time and money. We envision that the data in this CRN can potentially be used to do the following: (1) evaluate the effectiveness and safety associated with differing treatment options; (2) provide a framework for clinical studies to be conducted within the CRN, including industry-sponsored studies required to fulfill the FDA’s request for premarket and postmarket regulatory activities; and (3) allow healthcare providers to track surgeon volume, patient outcomes, and measures for quality improvement activities.
One of the key steps in creating the CRN for POP was identifying a core dataset that would result in minimal data entry by clinicians but would also be comprehensive enough to conduct safety and effectiveness studies. It was also important that the core dataset would be well-suited for new POP technologies that enter the market in the future. Especially given the FDA’s recent order to manufacturers of transvaginal mesh for prolapse repair to stop selling and distributing their products, our stakeholders foresee applications for new POP technologies in the coming years. We believe that the data elements captured in this registry may be generalizable to novel materials for both transvaginal or transabdominal application, novel approaches, and novel devices that may enter the market in the future.
Measures to gauge success in POP surgery can be divided into two main categories: objective measures and subjective (patient-reported) outcomes (PROs). Objective measures include the use of the POP-Q stage. Subjective measures include validated instruments that include vaginal bulging symptoms. The combination of objective and subjective measures may be more clinically relevant and meaningful to the patient than those that include only anatomic criteria to define success. Thus, a combination of an anatomic evaluation (objective) data with a validated instrument (subjective) data provides an appropriate measurement of success after surgical management of POP.42
As such, all stakeholders agreed that incorporating validated instruments to collect PROs would be an important component of the registry. Rather than including these questionnaires in the core minimum dataset, the working group is developing a patient-facing mobile application to collect PROs for the registry. The PROs collected may include any of the following instruments: Pelvic Floor Distress Inventory-20,43 Pelvic Floor Impact Questionnaire-7,43 POP/Urinary Incontinence Sexual Questionnaire-12,44 International Consultation on Incontinence Questionnaire,45 Patient Global Impression of Improvement,46 and Patient-Reported Outcomes Measurement Information System (PROMIS®).47
It is important to interpret the final core minimum dataset in the context of our methodology. The consensus that was reached ultimately represents the opinion of the clinicians who participated in the Delphi process and may not reflect the opinion of all practicing clinicians. However, our results are strengthened by the fact that we incorporated clinical experts from multiple specialty societies and academic research centers and allowed the FDA and patient partners to participate in the review and selection of the data elements. Another strength of our methodology is that all participants shared an equal influence on the outcomes and there was sufficient opportunity for feedback through the process. Overall, compared with a traditional consensus panel approach, the Delphi method strengthens our confidence in the results by eliminating the potential bias from group dynamics and face-to-face responses. In conclusion, the consensus on the POP core minimum dataset represents a unified effort by multiple stakeholders. The establishment of a national infrastructure (registry) collecting these data elements will accrue high-quality data on devices and technologies used for POP in the context of a multipurpose CRN.
Data availability statement
Data are available on reasonable request. The data that support the findings of this study are available from the corresponding author, CEB, on reasonable request.
Patient consent for publication
The Women’s Health CRN work was approved under IRB # 1511016772. Prior to initiation of this study, an MDEpiNet oversight committee was established, and a written protocol was developed which prespecified the study methods. The Delphi process involved feedback from Women’s Health CRN partners and as such no other regulatory requirements were considered relevant.
Twitter @courtneybaird22, @Artsytwits
Contributors AS and DM-D initiated the collaborative project. All authors participated in the initial stakeholder meeting to conceptualize the project and design the study protocol. CEB designed the Delphi surveys, collated and analyzed the Delphi survey results, managed the Delphi process, and drafted the first version of the manuscript. CSB and KK served as working group co-chairs, provided clinical/epidemiological insights on the Delphi survey results, and co-lead discussions with working group members. BC also provided clinical insights and leadership during the construction of the Delphi surveys. All authors served as members of the working group, participated in discussions throughout the Delphi process, reviewed manuscript drafts, and participated in editing and revising the manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. CEB is the guarantor.
Funding This work was supported by the Office of the Secretary Patient-Centered Outcomes Research Trust Fund under Interagency Agreement #750119PE060048.
Competing interests None declared.
Provenance and peer review Commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.