Hung AJ, et al. – A pilot study was conducted to explore and validate objective surgeon performance metrics applying a novel recorder to directly capture surgeon manipulations on the da Vinci robot. The authors describe the initial construct and concurrent validation study of objective metrics during pre–selected steps of robot–assisted radical prostatectomy (RARP). Objective metrics revealed experts as more efficient and directed during pre–selected steps of RARP. Moreover, objective metrics had limited correlations to GEARS. These outcomes lay the foundation for establishing standardized metrics for surgeon training and assessment.

Methods

• Researchers recorded kinematic and events data for expert (≥100 cases) and novice (<100 cases) surgeons performing bladder mobilization (BM), seminal–vesicle dissection (SVD), anterior vesicourethral anastomosis (AA) and right pelvic lymphadenectomy (RLD).
• They compared expert/novice metrics applying mixed–effect statistical modeling (construct validation).
• Expert reviewers blindly rated SVD and AA using Global Evaluative Assessment for Robotic Surgery (GEARS).
• Thereafter, intra–class association (ICC) measured inter–rater variability.
• Applying Spearman's test (concurrent validation), objective metrics were correlated to corresponding GEARS metrics.

Results

• They assessed performances of 10 experts (mean 810 cases; 100–2000) and 10 novices (35 cases; 5–80) in 100 RARP cases.
• Construct validation: Experts completed operative steps faster (p<0.001) with lesser instrument–travel distance (p<0.01), lesser aggregate instrument idle time (p<0.001), shorter camera–path length (p<0.001) and more frequent camera movements (p<0.03).
• They exhibited evidence that experts had greater ratio of dominant:non–dominant instrument–path distance for all steps (p<0.04), except AA.
• Concurrent validation: Median experience of 3 expert reviewers was 300 (200–500) cases. ICC between reviewers was 0.6–0.7.
• Kinematic metrics had low correlations with GEARS metrics for AA and SV.