Evaluation of intervertebral body implant performance using active surveillance of electronic health records

Study design and oversight

A comprehensive written protocol was developed, and plans for interim data reviews and a study oversight committee were established with representation from NESTcc, PEDSnet, Johnson & Johnson, and Lahey Medical Center, which reviewed and approved the final study protocol prior to any data analysis. The institutional review boards of Lahey Medical Center and Children’s Hospital of Philadelphia reviewed and cleared the study protocol prior to the review of any study data.

Study environment, data source and data element extraction

Epic Systems (Verona, Wisconsin, USA) has been used as the EHR system at all Lahey Health hospitals and clinics to support all clinical activities. Data elements were extracted from Lahey’s clinical data warehouse derived from the Epic EHR, containing demographic, clinical, laboratory, and claims data for all patient encounters in any Lahey facility. The universal device identifier (UDI) system12 was not implemented within the Lahey EHR until after the study period, and therefore device implants were identified through manufacturer model numbers recorded in the EHR surgical log. Of note, all patients undergoing spine surgery at Lahey are routinely followed with in-person or telehealth visits at 30 days and 12 months postoperatively, thereby minimizing the risk of the loss to follow-up in the first year after spine surgery at the medical center.

Patient eligibility, device exposures, and endpoint definitions

Patients 18 years or older, undergoing spinal arthrodesis surgery between April 1, 2015, and December 31, 2018, at a Lahey Health System hospital were included in the study (see online supplemental figure S1). Patients were excluded from the analysis if their index surgical procedure included the cervical spine or if the patient had any spine surgery performed in the 12 months prior to their qualifying surgery.

Among the patients eligible for inclusion, treatment with the device of interest (the CB), as well as prespecified alternative devices (the DePuy Synthes OPAL and the Medtronic CAPSTONE PEEK) was identified by matching surgical log device implant records to device model numbers available from the manufactures.

The primary safety outcome was the proportion of patients undergoing spine reoperation for any cause at 1 year. Secondary outcomes included mortality, hospitalization for surgical site infection, any hospitalization within 1 year, and requirement for blood transfusion during index hospitalization. The safety endpoints and clinical covariate definitions used for this study are provided in online supplemental tables S1 and S2 of the Supplemental content.

Covariate and data validation

Each demographic, clinical, and procedural covariate and outcome was validated through manual chart review of a random 5% sample of patients included in each cohort. Discrepancies found in manual chart review were used to refine covariate filter definitions, and the random 5% manual chart review was repeated until there was 100% agreement between the extraction filters and domain expert chart review for all covariates and outcomes.

Risk adjustment methods

Multivariable adjusted logistic regression models were developed to estimate the probability of being treated with the CB. The model included risk factors for the adverse outcomes of interest, as well as factors considered to influence the selection of IVBI. A total of 12 demographic, clinical and procedural variables were included in the final propensity score model (variable definitions available in online supplemental table S6 of the Supplemental content) including age, gender, preoperative body mass index, history of coronary artery disease, heart failure, diabetes, active smoking and history of any prior spine surgery. Procedural covariates included surgeon specific annual spine surgical volume, emergency surgery status, American Society of Anesthesiology physical status classification, and number of spinal levels stabilized during index surgery.

The propensity-matched comparison group was selected on the basis of a non-parsimonious propensity model and matched in a 1:1 manner with CB cases matched within a caliper width of 0.6 of the SD of the logit of the propensity score.13 Missing data were handled using univariate rules, assuming absence of a condition for dichotomous variables, and using the median value for continuous variables. The relative imbalance between the CB and comparator groups was assessed using absolute standardized mean difference in covariate means and proportions, with values greater than 0.10 considered suboptimally balanced.14

Because the number of patients successfully matched could be low and lead to an underpowered analysis, we also performed inverse probability of treatment weighting (IPTW) analysis. IPTW included all patients in both the treatment and control groups, weighted based on the probability of treatment with the device of interest, and therefore minimizes information loss through case exclusion. Weights were trimmed at 5% and 95% to avoid excessive influence of patients with extremes of propensity scores.15

A ‘Falsification Hypothesis’ analysis was also prespecified in order to assess the possibility of significant residual confounding after propensity matching or weighting. For this analysis, patients were evaluated for the development of late postoperative renal dysfunction after postoperative day 30, defined as an increase in serum creatinine by at least 50%. Late kidney dysfunction was not thought to be plausibly related to the implantation of a particular IVBI, and was therefore considered an appropriate endpoint for use in Falsification Hypothesis testing.

For the propensity match analysis, a significant difference was considered present if the confidence intervals (CI) between two independent proportions, as measured by the Wilson method, did not cross zero16 when using an alpha of 0.05. For IPTW, adjusted odds ratio (OR_adj) were considered significant for increased risk of adverse events if the OR_adj was greater than 1.0, with a 95% CI excluding 1.0. All analyses were performed within the DELTA application, relying on R-based statistical packages.7–9

Unadjusted outcomes

A total of 612 patients underwent thoracic or lumbar spinal fusion between April 1, 2015, and December 31, 2018, at a Lahey facility, of which 6 patients were excluded due to prior surgery on the thoracic or lumbar spine within the prior 12 months. The clinical characteristics of the remaining 606 patients identified for analysis are shown in online supplemental table S1 of the Supplemental content, along with the subgroups receiving CB, those receiving an alternative IVBI, and those receiving both CB and an alternative device. A total of 10 patients (1.65%) underwent reoperation within 1 year of the index surgery, 3 patients (0.5%) developed a surgical site infection, 105 patients (17.5%) were hospitalized within 1 year of their index surgery, and 3 patients (0.5%) died within 12 months of surgery (see online supplemental table S2 of the Supplemental content). In addition, 66 patients (10.9%) received blood transfusions immediately following surgery. The 158 patients who received both a CB as well as one of the alternative IVBI were excluded from subsequent safety analyses as outcomes could not be definitively attributed to a specific device type.

Prespecified propensity analyses

Patients treated with the CB IVBI, alternative IVBI, and those patients treated with neither during qualifying thoracic and lumbar spine fusion surgeries were identified from EHR derived data (see online supplemental figure S1 in the Supplemental content). Among the 136 patients receiving only CB during their qualifying surgery, 69 (51%) were successfully matched with patients in the comparator group (receiving an alternative IVBI) after performing a propensity score match. As shown in table 1, post-matching standardized differences were less than 0.10 for 13 of the 18 covariates, with 5 covariates having post-matching standardized differences ranging from 0.12 to 0.19, indicating suboptimal covariate balance.

There were no significant differences observed between the clinical outcomes in the CB treated patients as compared with the patients treated with alternative intervertebral body device implantation in the propensity matched analysis, as demonstrated in table 2. Among the 69 patients treated with the CB in the matched cohort, only 1 patient required reoperation within 12 months, as compared with no patients among the matched control patients.

Online supplemental figure S2 in the Supplemental content provides the results of the prospective active surveillance analysis, demonstrating stability of the results by repeated analysis of the primary endpoint outcome, by calendar quarter, throughout the study period.

As shown in table 3, the OR for the risk of reoperation at 12 months was 1.01 (95% CI: 0.99 to 1.03) as estimated by IPTW metholody, indicating no significantly increased risk for CB relative to the prespecified alternative IVBI. An extended analysis of the primary endpoint of reoperation out to 2 years maintained these null findings (OR 0.998, 95% CI: 0.962 to 1.035, p value 0.91). In addition, there were no significantly increased risks for any of the secondary outcomes at 12 months.

Additional sensitivity analyses

The eligible patient population was expanded in order to perform several post-hoc sensitivity analyses. Patients without a specific comparator device of interest, including patients with no device implanted at the time of index surgery (identified by procedure codes in online supplemental tables S7 and S8), as well as those patients undergoing complex repair (covering more than two spine levels) at the time of their index surgery were included in the analyses. Propensity score matching and IPTW analyses were performed on these additional groups (see online supplemental tables S3-S5 in Supplemental content) and the results were consistent with the primary analysis, with no increased risk of adverse events associated with the use of the CB.

Finally, the results of the Falsification Hypothesis comparing CB and alternative IVBI patients demonstrate a relative risk of 0.67 (95% CI: 0.11 to 3.87, p value 0.017) of developing renal dysfunction in the year following index surgery, indicating no evidence of significant residual confounding in the cohorts analyzed. Residual confounding was also absent in the post-hoc sensitivity analysis given the observed relative risk of developing renal dysfunction (0.25 with 95% CI: 0.15 to 2.75, p value 0.472) for the expanded pool of patients in the CB group compared with patients not exposed to CB.

Limitations

There are several important limitations of this study. The patients studied were treated within a single healthcare system, by a small group of experienced spine surgeons, limiting the generalizability to different environments. Also, the sample size of patients treated with the CB and alternative spine implants were limited, thereby reducing the power of the analysis to identify small differences in adverse outcome rates. However, the application of IPTW methods to reduce the loss of information by including all treated patients, suggests that the sample size did not materially impact the findings. In addition, additional patient groups were explored as comparators, with similar findings, further supporting the primary results. In addition, the residual imbalance in clinical risk factor distribution noted between the treated and comparator groups limit the accuracy of the estimated differences in adverse event rates.

Additional limitations include the risk of residual confounding, which we sought to minimize through robust risk adjustment using propensity matching and propensity weighting. In support of the effectiveness of the methods, the prespecified Falsification Hypothesis analyses found that late renal dysfunction, thought unrelated to choice of surgery, occurred with similar frequency between the matched populations, thereby indicating that there was no evidence for significant residual confounding in the cohorts analyzed.

This study was supported through a competitive research grant from NESTcc in accordance with Federal research regulations, and the study steering committee included regulatory and quality representatives from Johnson & Johnson, the manufacturer of the device studied.