Objective Surgical Skill Metrics

My research focuses on methods for automated, quantitative analysis of surgical skill. Given the wealth of tool-path data coming from simulation platforms and surgical robots, my particular interest is in deriving dynamic metrics that could make full use of this data. Cumulative metrics like time, path length, economy of motion, etc., only give general (though important) feedback about a procedure as a whole: a summary approach. However, dynamic metrics should reveal key information about skill or ineptitude for times within a procedure, and so be able to index when during a procedure such phenomena occurred. This could facilitate proximate feedback that promises to accelerate learning curves for surgical skill.

Literature reporting and exploring dynamic metrics:

• None :) I have yet to find a publication that really pursued this. Some did employ techniques that could be used in a dynamic analysis, but only cumulative information was analyzed. These appear below

Surgical/Clinical Literature relevant for deriving dynamic metrics

• Gallagher AG, Ritter EM, Champion H, et al. Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241:364-72, 2005. [Link] Citation Count: 18o. Introduces the notion of attentional resources: attentional demand is significantly strained by lack of visuo-spatial and psychomotor skills. Also suggests proficiency-based training as an important direction for the field.

• Watterson JD, Beiko DT, Kuan JK, et al. A randomized prospective blinded study validating acquisition of ureteroscopy skills using a computer based virtual reality endourological simulator. J Urol 168:1928-32, 2002. [Link] Citations: 710. Introduced a 1-minute window for error evaluation. Though it's purpose was simply to make error analysis easy and less artifact-prone, the 1-minute window idea could have been easily extended into a video indexing and proximate feedback scheme. However, performance was scored via expert video evaluation, which does not lend itself to objectivity, automation, or scale well.

• Satava, RM, Cuschieri, A, Hamdorf, J. Metrics for objective assessment. Surg Endo 17:220-226, 2003. [Link] Citations: 67. Attempted a comprehensive survey of metrics, metrics categories, skills, etc. as well as the technologies that do or don't provide them. The document was lacking in detail or development of the notions, but I think the idea of comprehensive analysis was important. Besides, how could any lit review about surgical metrics not have a Satava ref?

• The FLS Program (Fundamentals of Laparoscopic Skills) is also important to mention in the skill evaluation context. It incorporates proficiency-based training and a cost effective training approach. Most importantly it's been validated in multiple ways. However, no mention or effort is made for dynamic metrics or proximate feedback. However, FLS tasks present a nice, validated platform for studying basic skill acquisition and developing dynamic surgical metrics. The Link given above has a nice list of references following the inception (MISTEL's) of the tasks, their adoption, and validation. Citation counts average at about 70, with key publications being at 255 (the original, Derossis, American Journal of Surgery 1998), 181 (Fried, Annals of Surgery,2004), 102 (Coll of Surg, 2005).

• Judkins TN, Oleynikov D, Stergiou N. Objective evaluation of expert and novice performance during robotic surgical training tasks. Surg Endosc (2008). [Link]. A similar publication appears in the Journal of Robotic Surgery with the same authors and a similar title. This work focuses on robotic surgery and incorporates some more sophisticated metrics like curvature and relative phase analysis (a technique from the dynamical systems paradigm of motor learning). But, results were only presented as cumulative info.

Engineering/Computer Science literature

• J. Rosen, B. Hannaford, C. Richards, and M. Sinanan, “Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills,” IEEE Trans. Biomed. Eng., vol. 48, no. 5, pp. 579–591, May 2001. [Link] Citations: 95. This is the BioRbotics Lab classic which used tool-path data from lapr. procedures done on live pigs and treated it as a signal processing/machine learning problem. Markov models were used and Hidden Markov Models were later evaluated as well. There are several incremental improvements using this approach (or variations of it) that can easily be google'd.

My suspicion is that someone, somewhere has to have done some work in this area, I just haven't found the publications yet. Any suggestions would be most welcome.