Table 3

Saturation grid of elements identified through published literature and engaged stakeholders (pages 1–2 of 3)

	Previously published literature	Registry representative	Health system representative	Clinician	Patient and Patient caregiver	Device manufacturer	Device application reviewers	Data coordinator	Data analyst	Data informatician	Payer/HTA	Total
Identification of medical devices for the development of OPC	X	X	X	X	X	X	X	X	X	X	X	11
Device selection is a careful and conscientious process			X	X				X	X	X		6
Sufficiently mature device with sufficient collected real-world data	X						X	X	X		X	5
Sufficient level of understanding associated with the technology							X	X	X			3
Natural history of the indication understood						X	X		X			3
Priority given to medical devices with specific characteristics	X							X	X			3
Previously published literature ought to be reviewed for existing OPC	X	X	X			X						4
Engagement of key stakeholders	X	X	X	X	X	X	X	X	X	X	X	11
OPC generation is a collaborative effort	X	X	X	X		X	X	X	X			8
Stakeholders involved												–
Regulatory and notified bodies		X	X	X	X	X	X	X	X	X	X	10
Industry/device manufacturers		X	X	X	X	X	X	X	X	X		9
Clinicians		X	X	X	X	X	X	X		X	X	9
Patients				X	X	X	X			X	X	6
Data owners		X	X									2
Payers and HTA bodies		X		X		X	X	X			X	6
Hospital health systems								X			X	2
Professional organizations					X				X		X	3
Epidemiologists and analysts				X					X	X		3
Data informatician										X		1
Multistakeholder collaborative a priori decision making		X	X	X		X	X	X	X			7
Determination of minimally clinically important differences		X	X					X	X			4
Selection of an appropriate data source	X	X	X	X	X	X	X	X	X	X		10
Differing data sources for OPG (Objective Performance Goals) versus OPC (Objective Performance Crtieria) creation	X						X	X	X			4
Prioritize data sources with standardized data elements and libraries		X	X					X		X	X	5
Prioritize data sources that capture clinically meaningful relevant outcomes to patients	X		X	X	X	X		X	X		X	8
Data quality assessed using the IMDRF’s (International Medical Device Regulators Forum) eight characteristics of a registry	X									X		2
Consider national registries, international registries, claims and linked data sources	X		X	X		X						4
Performance of appropriate statistical analyses	X	X	X	X	X	X	X	X	X	X	X	11
Identification of the study Population												–
Clearly define the study cohort with appropriate inclusion and exclusion criteria	X			X			X			X		4
Consult stakeholders, expert opinion and literature to determine appropriate inclusion/exclusion criteria	X			X						X		3
Required sample size needs to be statistically justified and hypothesis driven	X											1
Endpoints												–
Assess and include effectiveness and safety endpoints	X	X	X	X	X	X	X			X	X	9
Discuss the determination of appropriate endpoints and their definitions	X	X	X	X	X	X		X		X	X	9
Include short-term and long-term outcomes	X	X		X	X	X				X	X	7
Select endpoints relevant to the patient	X	X		X	X	X	X			X	X	8
Engage patients to capture any endpoints due to unintended consequences of the device			X	X	X	X				X		5
When available, include functional outcomes such as patient-reported outcomes	X			X		X	X			X		5
When possible, assess soft endpoints		X		X								2
When possible, assess quality endpoints				X						X		2
Assess endpoints at relevant time points to provide suitable comparisons in single-armed trials	X				X						X	3
Assess long-term endpoints at the most prolonged time possible	X				X	X						3
Identification and selection of covariates												–
Report available patient-level, provider-level, facility-level and device-level characteristics	X			X	X					X		4
Capture common co-occurring illnesses	X			X	X					X		4
Differentiate between covariates and confounders	X						X				X	3
Remove irrelevant independent variables from the model	X											1
Missing data												–
Assess the level of missingness	X								X			2
Attempt to determine the type of missingness	X								X			2
Discuss how missing data were handled	X								X			2
Statistical analyses												–
Report and justify the model identification method	X							X	X			3

HTA, health technology assessment; OPC, objective performance criteria.