Objectives The Versius surgical system has been developed for use in robot-assisted minimal access surgery (MAS). This study aimed to evaluate the effectiveness of the Versius training program.
Design A 3.5-day program following 10 hours of online didactic training. Participants were assessed during the technical training using the Global Evaluative Assessment of Robotic Skills (GEARS).
Setting Dry box exercises were conducted in classrooms, and wet lab sessions simulated an operating room environment using cadaveric specimens.
Participants Seventeen surgical teams participated; surgeons represented general, colorectal, obstetrics/gynecology, and urology specialties. All surgeons had previous laparoscopic MAS experience, while experience with robotics varied.
Main outcomes measures Participants were scored on a five-point Likert Scale for each of six validated GEARS domains (depth perception, bimanual dexterity, efficiency, force sensitivity, autonomy, and robotic control). Additional metrics used to chart surgeon performance included: combined instrument path length; combined instrument angular path; and time taken to complete each task.
Results Participants demonstrated an overall improvement in performance during the study, with a mean GEARS Score of 21.0 (SD: 1.9) in Assessment 1 increasing to 23.4 (SD: 2.9) in Validation. Greatest improvements were observed in the depth perception and robotic control domains. Greatest differences were observed when stratifying by robotic experience; those with extensive experience consistently scored higher than those with some or no experience.
Conclusions The Versius training program is effective; participants were able to successfully operate the system by program completion, and more surgeons achieved intermediate-level and expert-level GEARS scores in Validation compared with Assessment 1.
Key messages
What is already known about this subject?
Robotic surgical systems, such as Versius, provide an innovative approach to circumvent the inherent challenges associated with traditional minimal access surgery (MAS).
However, robotic systems and MAS have a prolonged learning curve. A training program has been developed to provide surgeons with the core robotic skills needed to perform robot-assisted MAS.
What are the new findings?
This 3.5-day training program is effective; all participants achieved appropriate Global Evaluative Assessment of Robotic Skills scores and were able to successfully operate the Versius surgical system in a non-clinical setting by program completion.
How might these results affect future research or surgical practice?
It is hoped that this training program will help shorten the learning curve for operating using the new system, particularly for surgeons with little or no prior robotic experience.
As such, the program may play a part in reducing the barriers to uptake robot-assisted MAS and subsequently may increase the accessibility of MAS for patients.
Introduction
Laparoscopic minimal access surgery (MAS) is associated with several advantages over open surgery and is increasingly used to perform a number of procedures, including prostatectomies, hysterectomies, and partial nephrectomies.1 2 Advantages include reduced blood loss and pain, fewer infections, and shorter surgical procedure and post operation recovery times; together these benefits ultimately improve clinical outcomes and reduce length of hospital stay.3–5 Despite the associated benefits to patients and hospital workflows, laparoscopic MAS procedures are cognitively demanding and impose significant physical strain on surgeons that can lead to muscle fatigue injuries of the upper limbs, head, and neck.6–8
Robotic systems are increasingly employed in MAS to help alleviate the burden on surgeons and address some of the limitations associated with standard laparoscopic surgery. Robotic systems can provide several assistive features which allow surgeons to operate with improved comfort and visual feedback, as well as greater instrument stability and control. Research suggests that the benefits of robot-assisted MAS may also extend to patients, with fewer intraoperative and postoperative complications compared with standard laparoscopy, yielding lower conversion rates to open surgery.9 10 Despite the multiple benefits associated with robotic systems, barriers to uptake remain, some of which include limitations surrounding surgical access, communication within surgical teams, robot size, mobility, and ergonomics.
Versius, a new teleoperated robotic surgical system, has been designed to address some of the limitations associated with robotic MAS and the unmet needs of surgeons and their teams. First, the system hand controllers were designed ergonomically with surgeon input to optimize comfort while interacting with the system. The surgeon console has an open (ie, non-immersive) design that allows the surgeon to maintain communication with their team during surgery and is height adjustable, providing the option to sit or stand while operating. Visual feedback is provided by the surgeon head-up display of the surgeon console which displays a 3D video feed from the endoscopic camera with a display overlay. The instrument arms have eight articulating joints and the instruments are wristed, providing seven df for precise and stable movements while operating. Additionally, the instrument and visualization bedside units are mobile and practically sized for ease of transport, thus removing the need for a specialized operating room (online supplemental figure S1).
New users of any surgical robotic system undergo a learning curve during the training period in which they develop the skills required to safely and effectively operate the device.11 12 The learning curve describes the period of time and number of robotic-assisted procedures over which surgical skills improve through learning, competency, proficiency, and mastery phases along the curve. Effective training has the potential to reduce the learning curve, such that the target skill level is attained in a shorter time period and in fewer procedures post training.13 Given the number of novel features compared with existing devices, effective training for surgeons, including those with previous robotic experience, will be especially important for use of the Versius surgical system. To aid effective training, the Versius training program was developed to include both didactic and practical, hands-on training, and incorporate tasks designed to develop the motor and cognitive skills of using the surgeon console in a non-clinical setting. Through its use, it is anticipated that surgeons will become familiar with the console and its features.
The aim of this study was to evaluate the effectiveness of the surgeon training program for the Versius surgical system.
Methods
Participants and setting
Surgical teams included a lead surgeon, an assistant surgeon, a scrub nurse, and a circulating nurse. Participants were recruited by an independent recruitment specialist (Schlesinger Associates, New Jersey, USA) and gave informed consent to participate. Surgeons were selected from a range of surgical specialties that would reflect all groups likely to use Versius. These were: colorectal, general (including upper gastrointestinal (GI)), obstetrics and gynecology (OB/GYN), and urology. All participants had previous experience in laparoscopic MAS, while experience with robot-assisted MAS varied. Ethics approval was not required, as this study investigated the performance of a training program for a surgical robotic system in a preclinical setting. All cadavers were donated with consent.
Study design
Prior to attending the on-site training, participating surgeons were required to complete approximately 10 hours of online training and achieve a minimum of 80% in accompanying assessments. Following this, training was conducted in simulated operating rooms at the AdventHealth Nicholson Center, Florida, USA. The on-site training program comprised platform overview and basic training (day 1), training implementation (day 2), training consolidation (day 3), and training reinforcement (day 4 [half-day]). The 3.5-day time period for the on-site training was determined based on a series of formative studies. The program featured the following training components (figure 1): an introduction to the Versius surgical system setup; Versius trainer simulator training; dry box training, in which surgeons used Versius to complete several physical puzzles (ring stack, bead stack, maze base, and suturing); and finally wet lab training, during which Versius was used together with cadaveric specimens to practice core skills and alarm protocols and to simulate surgery. Core skills in wet lab sessions included retraction and manipulation, dividing tissue dissection and blunt dissection, cutting, and knot tying and suturing, all of which were tailored to procedure steps relevant to the surgical specialty of the surgeon.
Versius on-site training program. Assessments 1–3 and Validation are Global Evaluative Assessment of Robotic Skills.
Study procedures and evaluations
Global Evaluative Assessment of Robotic Skills (GEARS)
During selected wet lab training sessions on days 2–4 and during Validation, the surgeons’ robotic surgical skills were assessed using the validated GEARS tool.14 15 Investigator teams were trained to use GEARS by a specialist in robotic surgical training, so that teams were able to advise and support surgeons during the training sessions. For assessments, a minimum of one independent specialist observer scored each surgeon on a five-point Likert Scale for each of six validated GEARS domains (depth perception, bimanual dexterity, efficiency, force sensitivity, autonomy, and robotic control; online supplemental table S1). For most assessments, two assessors simultaneously scored each surgeon independently of each other; scores were then compared following each assessment and the mean was taken as their final score.
Descriptive analyses (mean and SD) were generated for all GEARS analyses, including overall GEARS Score and individual GEARS domain scores. Each of these scores were stratified by (1) surgeon specialty and (2) surgeon level of robotic surgery experience at baseline. Expected overall scores for novice (ie, no robotic experience; <20.9), intermediate (trained to use a robotic device; 20.9–24.0), and expert (routinely use robotic devices; >24.0) surgeons were estimated by combining the results reported by Aghazadeh et al,14 Sanchez et al,16 and Goh et al15 (online supplemental table S2). Individual GEARS domain scores for novice, intermediate, and expert surgeons were only reported by Aghazadeh et al14; these reported median scores were therefore used to define individual domain thresholds for novice, intermediate, and expert surgeons in the current study.
Assessment of surgical skills using the Versius trainer
The Versius trainer simulator provides a digital environment in which the surgeon can control virtual instruments when the simulator is connected to the surgeon console, providing virtual exercises designed to train the basic robotic skills needed to successfully operate Versius. Tasks are designed to provide basic level practice of robotic skills in a non-clinical setting.
Following setup and practice of cursor navigation, clutch control, wrist movement, camera control, and ambidextrous movement, surgeons complete a total of 15 tasks (online supplemental table S3). In order to chart the surgeon’s performance, relevant metrics for each task were recorded. These metrics included: combined instrument path length (the total length of the path taken by the instrumentation in the completion of the task); combined instrument angular path (the total angular path taken by the instrumentation during the completion of the task); and total time taken to complete the task. Descriptive analyses (mean and SD) were generated for all Versius trainer metrics.
Results
Surgeon baseline demographics
In total, 17 surgeons completed the Versius on-site training program and were evaluated through four assessments during the study. Surgeon baseline demographics are summarized in table 1. Surgical specialties were similarly represented with four colorectal, five general (including upper GI), four OB/GYN, and four urology lead surgeons recruited. Robotic experience level was classified as either no previous robotic experience (<5 procedures, 4/17; 23.5%), some robotic experience (5–<30 procedures, 8/17; 47.1%) or extensive robotic experience (≥30 procedures, 5/17; 29.4%). The mean numbers of robotic procedures conducted as primary surgeon and laparoscopic procedures conducted per month were 7.6 (SD: 8.4) and 14.9 (SD: 11.5), respectively.
Table 1
|
Surgeon demographics
GEARS scores over time
The surgeons demonstrated improvements in mean overall GEARS scores over time, increasing from 21.0 (SD: 1.9) in Assessment 1 to 23.4 (SD: 2.9) in Validation (within the intermediate score range (20.9–24.0)) by the end of the training program (figure 2). Overall, from Assessment 1 to Validation, seven surgeons moved from novice to intermediate GEARS skill levels and three moved from intermediate to expert; five remained in the same skill level category, all of whom were at least intermediate, and two surgeons moved from intermediate to novice, both of which were small differences in scores (0.5 and 1.0) from Assessment 1 on the intermediate boundary.
Mean GEARS scores over time. N=17 for each assessment. Error bars represent SE of the mean, data labels show mean (SD). Skill levels were derived from published literature providing GEARS Score ranges as follows: novice (<20.9), intermediate (20.9–24.0), and expert (>24.0).14 GEARS, Global Evaluative Assessment of Robotic Skills.
Improvements in GEARS scores were observed for all surgical specialties and previous robotic experience levels; all stratified groups improved and achieved an intermediate skill level using the Versius surgical system (figure 3). Greatest differences were observed when stratifying by experience (figure 3a). In all assessments, surgeons with extensive experience of using robotics had consistently higher mean GEARS scores than those with some or no robotic experience. Additionally, the margin of improvement in their mean GEARS Score from Assessment 1 to Validation (4.3) was greater than those with some (1.5) or no (2.7) previous robotic experience.
Mean GEARS scores over time by previous robotic experience and surgical specialty. (A) Mean GEARS scores stratified by previous robotic experience. (B) Mean GEARS scores stratified by surgical specialty. Shading represents skill levels derived from published literature providing GEARS Score ranges as follows: novice (<20.9; grey), intermediate (20.9–24; blue), and expert (>24.0; green). GEARS, Global Evaluative Assessment of Robotic Skills.
Changes in mean GEARS scores stratified by surgical specialty are presented in figure 3B. While mean GEARS scores at Assessment 1 were similar across specialties, improvements in mean GEARS scores from Assessment 1 to Validation were smallest in the general and colorectal surgeons’ scores (increase of 1.5 for each group), and greatest in the gynecology and urology surgeons’ scores (increases of 3.0 and 3.3, respectively). However, given that the proportion of surgeons with extensive robotic experience was joint lowest in the colorectal group (0/4; 0.0%) and highest in the urology group (3/4; 75.0%) (online supplemental table S4), these results support the notion that the extent of previous robotic experience is the main driver of robot-assisted surgical performance in training.
Individual GEARS domain scores
Six individual GEARS domains were assessed in the study: depth perception, bimanual dexterity, efficiency, force sensitivity, autonomy and robotic control. GEARS scores for all individual domains improved from Assessment 1 to Validation for all groups stratified by robotic experience (figure 4). All overall mean domain scores for surgeons’ performance were above the respective intermediate skill level thresholds in Validation. When stratifying by robotic experience, the only instance where the intermediate threshold was not achieved in Validation was for the autonomy domain; surgeons with no prior robotic experience recorded a mean GEARS Score of 3.8 (figure 4E). However, this score was within the intermediate score range (3.0–5.0),14 from which the domain threshold (4.0) was derived. Greatest improvements were observed in the depth perception (figure 4A) and robotic control (figure 4F) domains. Greatest improvements were observed in surgeons who had either extensive or no robotic experience; those with some experience generally improved by the smallest margins.
Individual GEARS domain scores. An intermediate skill level (blue shading) determined for each domain using GEARS scores derived from published literature as follows: (A) 3.5; (B) 3.25; (C) 3.0; (D) 3.25; (E) 4.0; and (F) 3.5.14 GEARS, Global Evaluative Assessment of Robotic Skills.
Versius trainer outcomes
Previous studies have demonstrated a correlation between performance on virtual tasks completed using simulation software and real-life surgical performance,17 18 and surgeons have reported a high degree of similarity between surgical tasks carried out virtually and in a real-life setting.19 Although currently unvalidated as a measure of robotic skill, the Versius trainer tasks offer repetitive training for the basic technical and surgical skills needed to use the system, with results conveying detailed insights into the skills of different user groups. Across most tasks assessed, surgeon performance was broadly aligned with the results of the GEARS assessment (online supplemental figure S2a–l). However, there were three outcomes where greater robotic experience at baseline did not automatically result in improved performance (figure 5). Notably, path lengths recorded when performing the endoscope control (figure 5A) and four robotic arms (figure 5B) tasks decreased for all users across all sessions, however, surgeons with no robotics experience recorded lower values compared with surgeons who had some experience, indicating better performance. Furthermore, path lengths recorded for the multitarget task were smaller for surgeons with no experience than for those with some or extensive robotics experience at Assessment 1 (figure 5C).
Versius trainer outcomes of interest. (A–C) Combined path lengths for Versius trainer outcomes. Decrease in path length indicates an improvement in skill for Versius trainer outcomes.
Discussion
The results presented here provide evidence that the Versius training program is effective; all surgeons demonstrated overall improvement over time, and more surgeons achieved GEARS scores in the intermediate or expert categories at Validation compared with Assessment 1. Previous studies have shown a correlation between surgeons’ prior experience in a similar surgical environment, such as microsurgery, and their performance in training programs for robot-assisted surgical systems.17 20 It is perhaps therefore unsurprising that those with extensive prior robotic surgical experience performed best in this study, recording highest scores across all GEARS domains and largest margins of improvement. Our finding that the overall mean GEARS Score at Assessment 1 was within the intermediate threshold is likely reflective of the fact that those with extensive robotic experience composed a substantial proportion of the study participants. Additionally, all surgeons had already completed some training in simulator and classroom sessions prior to this assessment. Nonetheless, surgeons with no previous robotic experience were adequately capable by the end of the program and their scores in Validation were similar to those of the surgeons who had some prior experience.
In this study, GEARS values representing an appropriate level of ability were derived from previously reported values. Although this provides a clear indication that surgeons were becoming capable of successfully using the Versius surgical system, these values have not been validated. Aghazadeh et al defined intermediate surgeons as having performed ≥5 but ≤30 procedures as the primary surgeon.14 The scores obtained by the surgeons following completion of the training program generally exceeded the intermediate scores reported by Aghazadeh et al, highlighting the program’s effectiveness.
The results of this training program confirm that the validated GEARS outcome scores are adequate in differentiating surgeons of differing skill levels. However, scores achieved by each stratified group were broadly similar, with overall scores differing by less than 10% on average. Although this variation in score is sufficient for discerning between surgeons with extensive or no previous robotic experience, the GEARS scores were less effective at mapping small changes within stratified groups. To ensure any changes made to the Versius training program during its future refinement are successful (ie, they improve surgical performance and/or shorten the learning curve), outcome measures are required that can discern small improvements in surgical skill. Consequently, GEARS may not always be the most appropriate outcome measure.
The outcome metrics provided by the Versius trainer simulator provide additional insights into surgical performance. Versius trainer outcomes demonstrated that surgeons with some prior robotic experience may take longer to adapt to certain techniques specific to the Versius surgical system than those with no prior robotic experience. Therefore, surgeons with prior robotic experience may perform well on certain tasks that align with other familiar devices but take longer to adapt to less-familiar techniques, such as the requirement for ‘clutching’ instead of using foot controls. Accounting for differences in baseline abilities in training has been used successfully elsewhere, with several instances of proficiency-based programs within the field of robot-assisted surgery.21 22 It is possible that tailoring coaching to previous experience may provide a more efficient training approach and therefore reduce the learning curve for groups stratified by prior robotic experience. For example, focusing on Versius-specific controls may be particularly important for surgeons with some robotic experience, as they may have adopted certain habits from previous experience using other robotic systems. Such habits may have contributed to the poorer Versius trainer outcomes (path lengths) among those with some experience compared with those with no prior robotics experience. Additionally, surgeons may have been focusing on time taken to complete the Versius trainer tasks, rather than angular length and path length. In the future, it could be helpful to coach the surgeons to focus on efficiency and minimize instrument movements.
Study limitations
Simulation training is an engaging experience capable of replicating a number of clinical scenarios and is recognized as an appropriate training method.23 While the training was conducted in a realistic simulation environment including the use of cadavers, ultimately it will not provide the full experience of a live procedure. Similarly, although cadavers have the advantages of human anatomy, cadaveric tissue has poor handling fidelity compared with live tissue and lacks the live operative experience of dealing with bleeding. This study included a relatively small number of participants; as such, the results may not be generalizable to larger surgeon groups. Additionally, the assessments included in the training program largely focused on the various technical skills needed for the lead surgeon to use the system. However, evidence shows that non-technical skills (NTS), such as team co-ordination, communication, and leadership, are also key to enabling effective delivery of surgical care, particularly given the lead surgeon’s displacement from the patient and their surgical team in robot-assisted surgery.24 Therefore, additional NTS training and assessment is planned to be incorporated in the future, to further evaluate the training program’s effectiveness in improving NTS and overall team performance.
Conclusions
The Versius training program provides surgeons with valuable training experience in simulated robot-assisted operating; performance in tasks steadily improved throughout this study and all participants were able to develop the core robotic skills needed to successfully use the Versius surgical system by program completion. Future use of the Versius training program will provide further insight as to its effectiveness and ways in which it could be improved to further reduce the learning curve.
