IDEAL as a guide to designing clinical device studies consistent with the new European Medical Device Regulation

Introduction

The evidence demanded by European medical device regulators is getting tougher, especially for high-risk devices. The new European (EU) Medical Device Regulation1 (MDR) has changed the evidence requirements for CE certification. The regulation states, in general terms, what kind of evidence is required for market approval and for subsequent surveillance, but it does not specify the types of studies which may be most appropriate in providing the evidence. This can pose a major challenge for innovators and developers of devices, many of whom are relatively inexperienced in planning clinical studies and limited in their capacity to fund studies. A framework specifying methodology for producing evidence throughout the lifecycle of new products would therefore be both useful and timely.

IDEAL provides a sequential framework describing the types of studies which are appropriate at each stage of the life cycle of a new procedure or device—from its first use in a human being through to its widespread use. It was originally developed for evaluating new surgical procedures, and subsequently adapted for therapeutic devices (IDEAL-D).2 3 It provides an integrated evaluation pathway, analogous to the phases 1–4 trials sequence used for new medicines, and it could therefore be a useful adjunct to device regulation.

This article analyses the alignment between the evidence demands of the new EU MDR, and the recommendations of IDEAL-D. The findings highlight how adopting IDEAL-D could help innovators in generating appropriate evidence on new devices for MDR licensing purposes. The similarities in regulatory requirements internationally suggest that our conclusions may be relevant to future developments in the USA, the EU and any new UK system.

The Medical Device Regulation

The MDR was developed in the context of device proliferation, increasing speed of innovation, high-profile cases of devices causing harm, and demands for better clinical evidence around risks—especially for implantable devices and high-risk devices. It sets out the evidence requirements for obtaining a CE certificate, and replaces several previous Medical Device Directives. See box 1 for the main changes it introduces.

The MDR represents a shift from a binary ‘certified/not certified’ model to a ‘life-cycle’ approach, placing increased emphasis on ongoing generation of clinical evidence after certification. Revision 4 of the Medical Devices (MEDDEV) 2.7/1 guidance document4 forms the basis of the MDR’s tougher requirements, including the need for clinical investigations for higher risk devices. Devices are classified from class 1 (low risk) through 2a and 2b to class 3 (high risk)5 : all class 2b implantable and class 3 devices will require clinical investigations. Devices considered compliant under previous directives are not guaranteed automatic certification under the new MDR. All existing devices will therefore have to be reviewed, and their supporting evidence enhanced where necessary. However, like the directives, the MDR does not specify the types of studies which should be performed to achieve this.

Evidence requirements for CE certification under the MDR

Under the MDR, the need to demonstrate compliance with standards in areas such as toxicology, functional safety, biocompatability, usability and sterilisation remains unchanged. Most of this testing is done before any clinical use.

Thereafter, the MDR requires clinical evidence from at least one well designed clinical study, to assess the safety and performance of a device in patients, unless sponsors can demonstrate sufficient clinical data applicable to their device from other sources. The evidence needed varies depending on the risk classification of the device.

The precise design of these clinical studies is not specified in the MDR and neither the national regulatory authorities (termed Competent Authorities in the EU) nor the Notified Bodies (which perform the assessments prior to issuing CE certification) are permitted to advise applicants on study design or data requirements (see box 2). This deliberate policy of avoiding detailed guidance on evaluation methods allows flexibility, and prevents any suggestion that certification is guaranteed if a specified type of study is done. It is up to manufacturers to justify study designs.

Box 2

The European (EU) system for medical devices conformity assessment

Certificate of conformity (CE mark)

Required for devices to enter the EU market

Indicates that the device

• Performs the functions is it marketed for

• Is safe in normal use.

Conformity assessments

Are performed by commercial companies called Notified Bodies (NB).

Competent authorities:

Are national government agencies in each EU country which regulate medical devices, by:

• Reviewing the work of NB.

• Advising whether proposed evaluation plans are acceptable.

The process

• Companies with a new class 2 or 3 device need to generate evidence for a clinical evaluation to submit to an NB. (class 1—low risk—devices can obtain a CE mark by a process of self-registration.)

• Companies may seek advice from the Competent Authority, but the authority cannot suggest evaluation plans.

• The evaluation is carried out by the company and the results are analysed and reviewed by the NB

• The NB decides whether the evidence meets the standards of EU law for performance and safety.

• If the NB is satisfied, a CE certificate is issued.

• This process is protected by commercial confidentiality agreements between the company and the NB. Only the national Competent Authority may see the data and inspect the company’s facilities.

Current UK status

In the UK, the Medicines and Healthcare Regulatory Authority is the Competent Authority. UK policy on device regulation is currently under review following Britain’s departure from the EU.

The Competent Authority of each EU country must independently approve any clinical investigation work in its jurisdiction. It must be informed of any adverse events, and can stop or suspend clinical studies nationally . The authorities are provided with the study results, but do not normally review them unless there are clear reasons for concern.

Request permission

Figure 1

Alignment of the IDEAL stages with progress through the Medical Device Regulation (MDR) system for medical device evaluation and approval. CA, Competent Authority; NB, Notified Bodies; RCT, randomised controlled trial.

The ‘clinical evaluation’, incorporating the results from the clinical investigation and/or other relevant clinical data is reviewed by a Notified Body (anywhere in the EU), together with broader technical documentation. The information supplied to the manufacturers must demonstrate that the device is safe and performs correctly when used as intended. For high-risk devices, a CE certificate is issued if there is sufficient evidence that the benefit of the device outweighs the risk, considering its intended purpose. For lower risk devices technical documentation is reviewed on a defined sample of devices. Evidence of superiority over similar devices is not necessary, although consideration of currently available alternative treatments is required. There is no consideration of cost. In certain circumstances, Notified Bodies may place restrictions on a CE certificate—for example, allowing use only in the context of an approved postmarket surveillance study.

Regulatory requirements after CE certification

Unlike previous EU directives, the MDR requires an explicit plan for post-market clinical studies and surveillance, but it does not specify precisely what form these should take. This applies to all device risk classes, but more detailed surveillance is expected for higher risk devices. Manufacturers will be required to submit regular assessments of device performance through Post-Market Clinical Follow-up reports (PMCF) and Periodic Safety Update Reports (PSUR). The EU device database, Eudamed, will compile key safety information.

The IDEAL framework

The IDEAL framework acronym spells out the evolutionary stages of complex therapies: Idea, Development, Exploration, Assessment and Long-term Study. The IDEAL recommendations describe study design and reporting proposals for each stage.6

Box 3 shows the stages in the IDEAL-D framework and some of the key recommendations for studies at each stage. Each stage of IDEAL-D is focused around a key question: stage 0—what is the new device and what does it do? This stage addresses preclinical testing: it was not included in the original IDEAL framework, but is important for devices. A very wide range of studies may be relevant, depending on the nature of the device.

Stage 1 (Idea)—Does the device work as intended in a patient? This stage describes the initial use in patients, including detailed description of patient selection and informed consent, the device, its mode of action, implantation or activation and the outcomes of use.

Stage 2a (Development)—Has the device been developed to a stable state? Modifications to the device, its use or patient selection criteria, and consequent changes in outcomes must be explained and documented. Reporting should therefore include a case-by-case sequential display of outcomes.

Stage 2b (Exploration)—Has clinical consensus been reached on indications, manner of use and quality standards? With wider use, variation develops in both use and patient selection. Variation in experience and training can affect both effectiveness and risks. IDEAL recommends collaborative multicentre cohort studies at this stage, incorporating prespecified subgroup analysis to resolve known controversies, and statistical evaluation of learning curve effects.

For devices, the IDEAL-D variant3 can be used, where stages 2a and 2b can be combined into a single stage 2 study with the potential to alter its focus as it develops.

Stage 3 (Assessment)—Is the new device better than current standard of care?

This question is best addressed in a randomised controlled trial (RCT). However, RCTs may be infeasible for small iterative developments not claiming superior effectiveness and an IDEAL 2b-like study may then be adequate. Real-world evidence (RWE) approaches to evaluating comparative effectiveness are becoming feasible, but require great care in ensuring adequate data quality and avoiding common sources of bias.7

Stage 4 (Long-term)—Does surveillance reveal any unexpected outcomes or trends?

IDEAL-D proposes continuing surveillance, preferably in a comprehensive prospective registry—which may have data linkages to routine healthcare and population databases containing high-quality RWE.

How IDEAL-D could guide evidence generation for CE marking

The MDR illustrates how regulators have adopted a strategy of giving general advice but avoiding specific methodological requirements for clinical evidence for therapeutic devices. At every stage, there are guidance statements which permit a broad range of interpretations. This model clearly has value for regulators, but it poses significant problems for product developers. The temptation to opt for the cheapest way of complying with imprecise statements of principle is likely to depress evidence quality. Conversely, lack of specific guidance increases the risk of being asked for additional studies, resulting in expense and delay for innovators, who are required to develop their own study designs and reporting models instead of conforming to a standard. It also results in major heterogeneity between studies, which may affect equity of treatment for innovators, and the comparability of data in studies of similar devices. The detailed recommendations for study design and reporting in IDEAL-D are fully in tune with the principles of the MDR, and provide clear guidance on how to develop evidence, where the MDR does not.

By defining the types of studies generally needed at different stages of a device’s life cycle, IDEAL-D could provide an independent standard to inform regulatory study design, while still allowing Competent Authorities to comply with their obligation not to prescribe. Since it is an integrated evaluation pathway, IDEAL-D can also allow forward planning of study methods throughout the life cycle.

Studies complying with the recommendations for IDEAL-D stage 0, 1 and the 2a-like part of stage 2 are likely to supply sufficient evidence for CE certification in most cases (figure 1). Devices in the highest risk categories (eg, Transcatheter Aortic Valve Replacement8) might require IDEAL Stage 2b-like or 3 studies, but for most devices these formats would be more relevant for postmarketing clinical follow-up. The preferred design of the proposed new follow-up studies (PMCF and PSUR) under the MDR is not specified, but they will need to study large numbers of patients prospectively, and allow analysis of patient subgroups and variations in device design and in techniques. These are typical features of the IDEAL stage 2b cohort study, which could therefore be a suitable template.

The MDR does not discuss when an RCT (IDEAL stage 3) is necessary for CE certification. Broadly speaking, innovative and highly novel devices which claim superior effectiveness or are high risk (Class 3) will need RCT evidence, while other devices usually will not—but they will require some evidence from clinical studies. For this situation, the IDEAL stage 2b-like design would again be an appropriate choice.

For long-term postmarket surveillance, likely to be needed for class 2 and 3 devices, IDEAL-D recommends prospective registries in stage 4, and specifies what their characteristics should be (figure 1).

IDEAL-D was developed to guide development of high-quality evidence of safety and effectiveness over the life cycle of a device, by focusing on the key questions at each stage. The MDR was developed to rationalise, clarify and tighten device regulation, by systematising the requirements for clinical evidence provision. Their different objectives explain their differences, but they have a great deal in common.

Both IDEAL-D and the MDR emphasise the need to provide different types of evidence at different stages in the device life cycle. Both recommend preclinical studies providing technical information on device performance and safety, and then data on safety and technical success in early clinical studies. Both recognise the importance of high-quality, high-volume data about outcomes in real-world use during early clinical experience. These points of similarity make it possible to align the clinical evaluation process and the IDEAL Framework (see figure 1). In many respects, conformity with the appropriate IDEAL-D stage recommendations is likely to ensure conformity with the regulation. This concordance could help to resolve the problems posed for innovators by the avoidance of specific advice in the MDR. There is a broad consensus among regulators on the types of evidence needed for different devices types at different stages,5 so the template provided by IDEAL-D might be useful in other jurisdictions, as well as providing a simple way to align with MDR requirements.

Conclusion

A system such as IDEAL-D cannot be formally incorporated into MDRs, but if it assists developers in providing relevant evidence, then regulators could signal this by suggesting it is considered by potential applicants for CE certification (and for regulatory approval in other jurisdictions).

By defining the types of studies needed at different stages, IDEAL-D could guide forward planning of study methods throughout a device’s life cycle, helping to optimise speed and efficiency, and to reduce the cost of evidence generation—not only for regulation but also for health technology assessment, commissioning of services, and clinical research.9 This would reduce the waste, cost and inefficiency associated with developing different evidence for a series of different audiences.