The two Achilles heels of surgical randomized controlled trials: differences in surgical skills and reporting of average performance.
Academic Article
Overview
abstract
Randomized controlled trials of surgery are fundamentally different from randomized controlled trials of medications because it is difficult to blind or mask a surgical procedure or perform "sham" operations. An additional challenge is the variation in skills and surgical proficiency of participating centers and surgeons. Addressing heterogeneity in surgical proficiency remains of paramount importance, especially when randomized controlled trials involve a new or complex procedure such as minimally invasive radical surgery. In the presence of such heterogeneity, it is very cumbersome to evaluate objectively and monitor surgical skills so that most trials simply report associations that are averaged across surgeons and hospitals/centers. Such reporting is not transparent because the rates of complications and adverse outcomes are reported only as averages, and these averages may not apply to the individual participating surgeons or centers. These factors, coupled with the inherent nongeneralizability of findings from such randomized controlled trials, because of the strict inclusion and exclusion criteria for enrollment, may lead to conclusions that no longer apply to real life for individual surgeons or centers. Case in point is a recently published noninferiority randomized controlled trial that reported that minimally invasive radical hysterectomy was associated with lower rates of disease-free survival (86% vs 96.5% at 4.5 years) and overall survival (93.8% vs 99% at 3 years) than open abdominal radical hysterectomy in patients with cervical cancer. However, randomized controlled trials that involve 2 competing complex or new procedures may be affected by tremendous confounding because of variations in surgical proficiency and also nonstandardization for other confounding factors such as patient selection categories (ie, stage of cancer) and adjuvant postoperative therapies that may affect long-term survival. The purpose of this Viewpoint is not to provide an exhaustive review of the trial's shortcomings but to use it as an illustration to focus on 2 challenging areas that most randomized controlled trials of a new complex surgical procedure suffer from: (1) unadjusting or not correcting for surgical skill variability and (2) nontransparent reporting of averaged results. We provide suggestions to overcome these deficiencies through robust methods and statistical approaches.
