Introduction
Evidence is the currency that medical device developers create from the data of clinical and other studies to support adoption of their products into healthcare. The evidence is used by a range of stakeholders (box 1) in determining whether a new device will be marketed, paid for by health insurance and adopted. Regulators need evidence on safety, performance and efficacy. Health technology assessment (HTA) organizations, payers and policy-makers are interested in clinical and cost-effectiveness. Clinicians and patients want to know about comparative effectiveness, ease of use, durability, and risks in the short and long term.
Range of stakeholders, for whom a spectrum of evidence is needed
Regulators (eg, Food and Drug Administration (FDA) in the USA; Medicines and Healthcare products Regulatory Agency (MHRA) in the UK).
Health Technology Assessment organizations.
Policy-makers.
Payers (commissioners, insurance companies, Centers for Medicare & Medicaid Services (CMS)).
Clinicians.
Hospital procurement managers.
Patients and caregivers.
Much has been written about the challenges facing medical device developers in producing all the evidence needed for a new device to become widely used in patient care.1–7 These challenges are especially difficult for small companies and developers. Trials are vexed by learning curves and rapid iterations of device design, adding to the difficulties of choosing appropriate comparators and outcomes.2 8 9 Speed of evidence generation matters because it is a costly process. The life cycle of most medical devices is short, so delays in getting the device to market will reduce the duration of profitable marketing and may make it non-viable. If a new device offers real advantages to patients, there are public health reasons for enabling patient access with minimal delay.
This article explores the evidence needed to satisfy the range of stakeholders. It describes changes being introduced by regulators in the evidence they require for high-risk devices. While challenging, efficient generation of evidence can be achievable through good planning early in the product development cycle.
Evidence required for regulation
Changes are afoot in device regulation around the world, leading to uncertainties about what evidence will be needed for new high-risk devices, with increasing emphasis on long-term, real-world data.10–13 It is now recognized that the findings of randomized controlled trials (RCTs) may not predict how new devices will perform in everyday use.3 14 In addition, the pace of device iteration often means that the findings of long-term RCTs are no longer relevant by the time they become available. RCTs remain necessary for devices which are conceptually novel or being used for radically new indications; however, a variety of statistical and validation methods are now available for enhancing the usefulness of non-randomized studies such as propensity scoring and linked evidence approaches.3 15 The increased recognition of RCTs’ shortcomings and the additional value of real-world data collection has led to the shift of balance in regulatory decisions toward the latter, based on “conditional approval” and “Coverage with Evidence Development (CED)”.
Data obtained throughout the total product life cycle are particularly important for high-risk and implanted devices. Regulators are increasing their focus on collection and review of data after market approval.16 17 This shift is taking place around the world, supported by guidance from the International Medical Device Regulators Forum (IMDRF) aimed at harmonization of approach.18 Japan and Saudi Arabia have both incorporated harmonized material into their medical device regulations.19 20 China’s regulatory framework has evolved dramatically over the last 10 years and India’s new Medical Device Rules came into force in January 2018.21–24
Evidence required for HTA and reimbursement decisions
HTA and reimbursement decisions have traditionally presupposed that regulators are fully satisfied with evidence on safety and efficacy. However, by necessity, regulatory decisions typically are made when evidence is incomplete: much of the evidence on use of devices in different patient populations or by different clinicians, and long-term effectiveness and safety can only be generated by “real-world” practice after market approval by regulators. To capture this evidence, systems are needed that enable clinical care information to reach the status of valid scientific evidence fit for regulatory, HTA and health-policy decision-making.25
HTA agencies and payers are interested in value for patients and healthcare systems, based on evidence about health outcomes, rather than efficacy alone. In general, HTA organizations evaluate comparative effectiveness to determine whether the new technology performs better than what is in current use. In addition, many of them also use evidence on health utilities to calculate a cost per QALY (quality-adjusted life year).26
Evidence generation for health economic evaluation is often neglected by product developers, but through discussion with clinicians, health economists and most importantly HTA organizations, they should be able to identify key information, normally health-related quality of life data, required for health economic modelling, around the time of product launch.4 Thereafter, resource utilization data will be needed about replacement of the device or component parts, or other necessary interventions, to inform economic modeling.
Finally, healthcare professionals, hospitals/providers and patients will want to know whether the device is safe, whether it works better than the alternatives, how much it costs and whether it is convenient to use. Clinicians are particularly interested in knowing how a device is performing in the hands of colleagues, and they may contribute their own data to trials and registries.
Long-term data collection as a key challenge
Collection of data in the long term remains a challenge but is becoming more routine, with increasing potential to link and merge electronic data from different sources, such as device and disease registries, routinely collected healthcare data, medical insurance claims and unique device identifiers (UDIs). If the main potential benefits of a new device will be realized only in the long term (or if there are potential long-term risks), then it is important to identify the best short-term or surrogate outcomes to support early post-market adoption of the device. It is also important to consider how long-term outcomes will be gathered: is it necessary to establish a new, specific registry or will early liaison with an established registry be a better option?
Continuous acquisition of long-term data for high-risk devices (such as implants) and high-cost technologies (such as new imaging machines) is best planned from the earliest stage, collecting information about all patients in whom the device has been used.27 28 Creating linkages of device-specific data to other datasets (eg, national healthcare and population statistics, or administrative claims data), which may be used in the long term for investigating possible outcomes, will become increasingly valuable. Identifying a core set of data elements (each with agreed definitions) and plans for linkage should be specified from the outset, when describing strategies for generating long-term evidence. Examples of systems which aim to collect data on devices that can be used by regulators are the VISION initiative (http://mdepinet.org/vision-crn/initiative) in the USA, which is the National Evaluation System for Health Technology vascular device demonstration project, and Beyond Compliance initiative in orthopedics in the UK.25 29 30
UDIs are now being attached to all high-risk devices: these will greatly potentiate the capacity to track individual devices, provide information on different versions of each device and collate information about devices of a similar type. Linkage of UDIs to electronic patient records, including administrative claims, will expand the potential for a range of other links and for subsequent analyses. The extent to which this can be achieved will depend on the organization of health systems and attitudes to data access in different countries. In the UK, data linkage studies have been used on very large datasets such as the Clinical Practice Research Datalink which contains primary care data from several million patients.31 32 Sharing data internationally could provide even larger datasets, which would generate safety signals more rapidly than in the past. Efforts are being made to achieve international collaborations, but they need careful attention to different countries’ data protection laws.33 34
Many useful device, procedure and disease registries exist, most held by professional medical societies and national agencies.35 36 The datasets held by device companies have great potential, provided there are clear arrangements for governance and transparency.37 As part of EUnetHTA Joint Action 3, REQueST tool, a quality standards tool for registries, has been developed and validated by HTA agencies.38 The IMDRF has also published useful material.39 40
Value propositions for different countries
Data from well-planned studies on efficacy, performance, safety and clinical effectiveness are likely to be relevant universally. However, clinical practice differs between countries and overall costs of care often vary considerably.41 Device use may vary by indication, involve different types and numbers of staff, or divergent practices (eg, visits to hospital and follow-up). These differences mean that resource use and cost data from other countries may be inapplicable—for example, in the National Institute for Health and Care Excellence (NICE) evaluation of TURIS (TransUrethral Resection of the prostate In Saline).42
Quality-of-life data, required to calculate incremental benefit to patients, are normally generalisable but where a device’s value proposition (box 2) is a return on investment rather than incremental benefit at higher cost, its impact on pathways, clinical practices, staffing and reimbursement may all affect its value case. HTA agencies therefore often use their own country’s standardized data for estimating the likely cost of introducing a new device, compared with current practice.43 A limited cost study in the relevant country may be a better strategy than using more robust data already available from another country. Plans for creating a good value proposition should be considered early during product development and discussed with a health economist.
Types of value proposition for medical devices
The device gives a return on investment by reducing overall costs of care.
The device offers increased benefits for patients but at increased cost.
Early engagement to plan a strategy for evidence generation
Early planning is key to producing evidence in the most efficient and economical way. The input from regulators, HTA experts and payers is crucial in designing studies for market approval, insurance or health service coverage, and adoption of the device. Box 2 summarizes the main types of value propositions for HTA and reimbursement decisions. Listening to advice from patients and clinicians (a broad range of knowledgeable people rather than enthusiasts) is critical in planning, especially when a new device is addressing an unmet need or providing advantages for patients over devices in current use.
There is an inherent tendency for developers to avoid meeting with regulators early in the device development process, but this is not a winning strategy. In the USA, the FDA/Center for Devices and Radiological Health (CDRH) has developed the Pre-submission Program that supports early meetings to discuss innovative study designs and to guide developers in evidence generation for marketing approval.44 For example, FDA may suggest a limited early feasibility study to accelerate plans for initial clinical evidence—often a funding milestone.45 Early, frequent meetings with FDA can help to design lean, non-clinical testing plans (ie, “just-in-time-testing”), while deferring other testing until device design is finalized. When a pivotal clinical trial is planned, the FDA/CDRH Payer Communication Task Force (PCTF) has developed a process to engage the payer/HTA community for advice on their evidentiary needs or potentially for Parallel Review with CMS.46 47
Early advice from a range of stakeholders will enable a strategy which avoids the pitfall of generating evidence for regulators, then HTA, then other audiences, in sequence, as each appears on the horizon. The strategy should include a clear plan for continuous collection of evidence in the long term. Ideally, for high-risk or implantable devices, comprehensive data collection should be started from the first use in man—either by liaison with an existing registry or by establishing a new one.27 28 This can be a demanding task, but early effort will pay dividends in the long term. Any registry should include a clear record of the times when a device undergoes modifications, including UDI information that can differentiate between versions of the device.
Worldwide collaborative initiatives by regulators and payers to facilitate evidence generation
Recognising the importance of good early planning and the uncertainties that developers and manufacturers of devices can face, regulators and HTA agencies now offer processes for providing advice and assistance. In the USA, FDA/CDRH has developed the PCTF integrated with CMS/Coverage and Analysis Group (CAG) to offer a Parallel Review program, which includes pre-clinical trial design advice from CMS and simultaneous review of the trial data.46 47 The PCTF initiatives include options for pre-trial communication with private payers and HTA agencies in the USA and the UK (NICE). NICE has its own Scientific Advice program, which offers a META tool to guide companies on their plans for clinical trials and data collection.48 The MaRS EXCITE initiative in Canada aims at introducing “innovative health companies” to relevant stakeholders.49
There will be some differences in the evidence required in different countries, arising primarily from statutory requirements, but the similarities in the data needed generally exceed these national variations.
For novel devices that require use of new surgical procedures, and for novel surgical procedures developed to support the use of devices, the IDEAL concept provides a framework for the kind of evidence needed at each stage of development.28 36 50
