World Journal of Urology (2021) 39:2231–2237 https://doi.org/10.1007/s00345-020-03406-6 ORIGINAL ARTICLE Non‑technical skills for urological surgeons (NoTSUS): development and evaluation of curriculum and assessment scale Abdullatif Aydın1 · Cora M. Griffin1 · Oliver Brunckhorst1 · Ahmed Al‑Jabir1 · Nicholas Raison1 · Haleema Aya1 · Craig McIlhenny2 · James Brewin3 · Majid Shabbir1,4 · Joan Palou Redorta5,6 · Muhammad Shamim Khan1,4 · Prokar Dasgupta1,4 · Kamran Ahmed1,7 Received: 24 April 2020 / Accepted: 7 August 2020 / Published online: 18 August 2020 © The Author(s) 2020 Abstract Objective In the last decade non-technical skills (NTS) have emerged as a vital area for improvement within surgery. This study aims to develop and evaluate a Non-technical Skills for Urological Surgeons (NoTSUS) training curriculum and assessment scale. Methods This international, longitudinal and observational study began with a 3-round Delphi methodology to refine cur- riculum contents and rating scale. Sessions with up to four participants were delivered where each candidate undertook an independent scenario within the validated full immersion simulation environment. Candidates were assessed using both the NoTSS (Non-technical Skills for Surgeons) and NoTSUS rating scales by NTS-trained and non-trained experts. A post- training evaluation survey was distributed. Results 62 participants comprising trainees (n = 43) and specialists (n = 19) undertook the NoTSUS course. The NoTSS and NoTSUS scales correlated well, with a mean difference of 3.3 in the overall total (p = 0.10, r = 0.53). However, there was significant differences in scores between the NoTSS-trained and non-trained raters (n = 28, p = 0.03). A one-way ANOVA test revealed significant improvement throughout the four simulation scenarios in each session (p = 0.02). The NoTSUS cur- riculum received positive feedback from participants and demonstrated educational value and acceptability. Conclusions The NoTSUS curriculum has demonstrated high educational value for NTS training aimed at urologists, with marked improvement throughout sessions. Correlation of NoTSUS and NoTSS scales proves its suitability for evaluating NTS in future training. Demonstration of inter-rater reliability indicates that the scale is reliable for use in assessment by expert faculty members. Furthermore, qualitative feedback from participants suggests gain of transferrable skills over the course. Keywords Non-technical skills · Simulation · Urology · NOTSS · NoTSUS Introduction Non-technical skills (NTS) are the cognitive and social abili- Abdullatif Aydın and Cora M. Griffin are joint first authors. ties that complement a clinician’s technical ability, compris- Electronic supplementary material The online version of this ing decision-making, leadership, team work, and situational article (https​://doi.org/10.1007/s0034​5-020-03406​-6) contains awareness [1, 2]. They are often grouped into social skills, supplementary material, which is available to authorized users. 4 * Abdullatif Aydın Department of Urology, Guy’s and St, Thomas’ NHS abdullatif.aydin@kcl.ac.uk Foundation Trust, London, UK 5 1 European School of Urology, European Association MRC Centre for Transplantation, Guy’s Hospital, of Urology, Amsterdam, The Netherlands King’s College London, 5th Floor Southwark Wing, 6 London SE1 9RT, UK Dept. of Urology, Fundació Puigvert, Universitat Autònoma 2 de Barcelona, Barcelona, Spain Department of Urology, NHS Forth Valley, Glasgow, UK 7 3 Department of Urology, King’s College Hospital NHS Department of Urology, Salisbury NHS Foundation Trust, Foundation Trust, London, UK Salisbury, UK 13 Vol.:(0123456789) 2232 World Journal of Urology (2021) 39:2231–2237 cognitive skills, and personal resource factors. Social skills involved 8 specialists. The Non-technical Skills for Urologi- refer to leadership, communication and teamwork, with cal Surgeons (NoTSUS) assessment scale, a modified ver- communication being a particularly important factor as it sion of the Non-technical Skills for Surgeons (NoTSS) scale, is one of the biggest causes for surgical errors. Leadership was also developed using data from the first two rounds, and itself can be subdivided into task and resource management, then involved a final, third round with two NoTSS-trained decision-making, and maintaining standards. Within cog- urology experts (A and B) to finalise the content (Supple- nitive skills lie decision-making, planning and situational mentary Appendix). awareness. Decision-making is often thought of as a more complex skill, which is accumulated throughout training as Study process and simulation a surgeon gains experience and knowledge on which to base their decisions. Personal resource factors include the capac- The developed curriculum was delivered as hands-on train- ity of an individual to cope with stressors and fatigue, which ing courses in the UK and to international attendees at the have been shown to negatively impact technical skills in the European Association of Urology (EAU) Annual Congress. operating room (OR). A total of 17 2-h training sessions with 3–4 participants There are many stressors in surgery, such as distractions were delivered on four independent occasions in Manches- and complications, so a surgeon must be trained to mini- ter (n = 14, 4 sessions), London (n = 4, 1 session), during mise their effect on performance [2]. Factors such as lack London EAU 2017 (n = 24, 6 sessions) and Copenhagen of sleep, causing fatigue, can result in a higher number of EAU 2018 (n = 20, 6 sessions). Each candidate took turns clinical errors and weaken leadership skills [3]. In fact, all to undertake an independent scenario within the previ- NTS components interlink and affect each other, and they ously validated full immersion simulation (FIS) ‘Igloo’ may be trained either individually or concurrently to result environment (Fig. 1; Imperial College, London, UK) [8]. in improved performance in the OR [2]. Participants were required to fully gown and glove during A standardised and validated NTS training program is simulation sessions. Other team members including anaes- currently lacking in the literature for urologists to achieve thetist, assistant nurse and floating nurse were role-played, better training [1, 4, 5]. With the hypothesis that a urology- to improve fidelity. Cameras were integrated into the setup focused NTS training course would be well received and in order to record candidates for later video assessment. The provide transferrable skills, the aims of this study are: (1) to previously validated Uro-Scopic Trainer model (Limbs and develop a simulation-based NTS curriculum for the training Things, UK) [9] was utilised for URS technical performance and assessment of urological surgeons and (2) assess the during scenarios. Two faculty members, with expertise in validity and reliability evidence, as well as the educational urolithiasis and NTS training, supervised participants. Each impact, of the developed curriculum. Furthermore, we aim participant undertook a 15–20 min scenario whilst faculty to evaluate the validity evidence of the developed assess- and the remaining participants in each group observed ses- ment scale. sions through a video-link. Debriefing followed scenarios in a structured manner, utilising the NoTSUS parameters and providing focused feedback. Methods Ethical approval was obtained, as part of the Simulation in Performance evaluation Urological Training and Education (SIMULATE) project (BDM/14/15-68) [6]. Each candidate was assessed using both the NOTSS and NoTSUS rating scales. Seven sessions were led by the NTS- Curriculum development trained experts A and B who performed assessments in real- time and the remaining ten sessions were led by other senior This prospective, international, longitudinal and observa- urologists (n = 4), utilising the same feedback format. Where tional study selected ureteroscopy (URS) as an index pro- NTS-trained experts were absent for supervision, detailed cedure for the technical component of the curriculum given videos of these sessions (n = 10) were provided to them for the availability of training models [7]. Initially, to refine the assessment (as seen in Fig. 1). Expert A assessed partici- curriculum content and scenarios, a two-round Delphi was pants from all sessions (n = 57) and Expert B assessed par- conducted whereby questionnaires were sent round to train- ticipants from 11 sessions (n = 40). The non-trained experts ees (i.e. urology residents-in-training) and specialists (i.e. provided assessment scores for 29 participants. An evalua- board-qualified urologists), including experts in NTS and tion questionnaire was distributed to participants after the urolithiasis procedures. The first round involved a total of 47 simulation to evaluate the NoTSUS course and self-per- respondents, 23 of whom were specialists. The second round ceived improvement, employing a 5-point Likert scale for 13 World Journal of Urology (2021) 39:2231–2237 2233 Fig. 1  A view from training sessions within the Full Immersion Simulation ‘Igloo’ environment quantitative feedback and comment boxes for qualitative populations. A one-way ANOVA test was used to measure feedback. improvement in NoTSUS scores over the four simulation scenarios in each session. Tukey’s multiple comparisons test Outcome measures was then used to further investigate the improvement of the cohort comparing sessions 1 and 3, and sessions 1 and 4. A Primary outcome measures were validity, reliability and P value < 0.05 was considered to be statistically significant educational value. Validity was measured by improvement for all tests. in candidate performance over the course judged by NoT- SUS score. Reliability parameters included the measurement of inter-rater reliability and the comparison of the NoTSUS Results scale with the gold-standard NoTSS scale. The perceived educational value was judged using candidate responses The Delphi study resulted in the development of four sce- to the post-simulation questionnaire. Secondary measures narios: (1) briefing the emergency OR team and performing included the fidelity of the scenarios, again judged by par- an adequate WHO checklist; (2) management of an intra- ticipants’ responses on the questionnaire. operative septic shock emergency; (3) interacting with an inexperienced scrub nurse and (4) troubleshooting with Statistical analysis faulty instrumentation. The NoTSUS assessment scale was adapted into a 5-point Likert scale (as opposed to the 4-point Microsoft Excel (Redmond, WA, USA) was used to col- NoTSS scale) by experts for ease of marking and further late all quantitative and qualitative data. Descriptive statis- expanded into five sections (Supplementary Appendix). tics were used for questionnaire data. Pearson’s correlation coefficient and paired t-tests were utilised to investigate Demographics correlation and agreement, respectively, in scores for inter- rater reliability, using ­SPSS® Statistics version 26 ­(IBM®, Overall, 62 participants received NoTSUS training con- Armonk, NY, USA). The former was also used for correla- sisting of trainees (n = 43) and specialist (n = 19) surgeons tion between NoTSUS vs NoTSS scales. GraphPad Prism with a wide range of experience (1–7 years of training version 8 (San Diego, CA, USA) was utilised to demon- and 1–20 years of specialist practice) from all over the strate all graphs and perform other basic statistical analyses. globe. The mean age of participants was 33.7 years (range: Unpaired t-tests were performed for the differences between 24–57 years) with 27% female participation. Neither level 13 2234 World Journal of Urology (2021) 39:2231–2237 Fig. 2  Correlation between 50 A the Non-technical Skills for 40 50 MEAN Surgeons (NoTSS) and Non- 30 NoTSS technical Skills for Urological Surgeons (NoTSUS) scales 20 40 amongst both raters A and B 10 Mean NoTSS 0 30 0 10 20 30 40 50 60 70 NoTSUS 60 20 B 50 40 10 NoTSS 30 20 0 10 0 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 Mean NoTSUS NoTSUS of ureteroscopy experience (p = 0.28) nor level of overall mean scores (n = 28, p = 0.03) between the NoTSS-trained clinical experience (p value = 0.78), determined by train- (n = 2) and non-trained raters (n = 4). ing years, was found to significantly impact mean NoTSUS scores. There were no statistically significant differences Improvement between sessions found in the scores between participants who had previously undertaken a form of NTS training (n = 15) and those who A one-way ANOVA test revealed significant improve- had not (n = 47; p = 0.57). ment throughout the four simulation scenarios in each ses- sion (p = 0.04; Fig. 4). Participants demonstrated marked Correlation of scales and assessors improvements between sessions 1 and 3, with a mean improvement of 10.9 points (p = 0.03), but there was no Correlation of the NoTSS and NoTSUS scores for each statistically significant improvement noted between other participant gave a correlation coefficient of r = 0.93 and sessions. r = 0.88, respectively, for expert raters A (n = 43, p < 0.0001) and B (n = 26, p < 0.0001) (Fig. 2). The two expert raters did Educational value not significantly differ in their scoring (Fig. 3), with a mean difference of 3.3 points in the overall total, as confirmed by The course was well received by participants (Fig. 5). a paired t-test of candidate scores (n = 36, p = 0.10, r = 0.53). The mean score on a 5-point Likert scale for all param- Scores showed moderate correlation throughout all domains eters asked about in the post-simulation survey was 4.5 of NoTSUS. However, there was a significant difference in out of 5. Individual aspects of FIS received an average A 70 15 B 60 50 NoTSUS Score Domain Score 10 40 30 5 20 10 p=0.002; r=0.45 p=0.09; r=0.41 p=0.28; r=0.53 p=0.11; r=0.50 p=0.05; r=0.46 p=0.10; r=0.53 0 0 Communication Management Decision Situational Resource Total & Team skills skills making awareness skills Fig. 3  Inter-rater reliability of all NoTSUS domains and total between the two expert raters 13 World Journal of Urology (2021) 39:2231–2237 2235 60 Discussion 50 NTS training has developed considerably in the past decade, with many more courses and validated training tools becom- ing available. Surgical simulation literature has mainly NoTSUS Score 40 focused on trainees and medical students rather than expe- rienced surgeons. However, experienced surgeons and those 30 post-fellowship may also become exhausted and experience burn-out due to poor NTS training [10, 11]. As NTS do not 20 always correlate with experience, there is evidently room for training to be incorporated at higher stages of medical 10 training [10–12]. This study aimed to add to the current 36.0 41.0 47.0 38.8 evidence-base with the NoTSUS training course. 0 In this study, the level of experience of participants did 1 2 3 4 not significantly affect their NTS performance, whether Scenario it was measured by number of URS procedures or over- all clinical experience (in years). Although this does not Fig. 4  Improvement in mean NoTSUS scores over consecutive sce- demonstrate construct validity, it supports the hypothesis narios (p = 0.02) that despite having superior technical skills as experienced surgeons, there is scope for improvement of NTS at all Participant Evaluation levels of training [10, 11]. However, some studies do show a difference in NTS between junior and senior participants Team interactions were realistic 4.6 [13–15]. This contradiction is likely due to the wide vari- ation between studies, which look at very different sam- I felt like I was in a real OR 4.3 ple sizes and populations, and use different scenarios and I behaved as I do in real OR 4.2 examiners to test candidates. In contrast to previous studies, 36% of participants had 4.4 FIS was good for displaying NTS previously received some form of NTS training and 19% I have gained transferrable OR skills 4.7 had also been assessed, either formally or informally. This shows a positive trend in the availability and popularity of V.productive & enjoyable course 4.8 training [16]. However, no difference was shown between Significantly improved my NTS 4.7 participants who had previously undertaken a NTS training course and those who had not. Although the style of NTS 0 1 2 3 4 5 training undertaken previously varied greatly, one can specu- Mean Likert Score late that courses are either not specialised enough to improve NTS, or that participants do not retain the skills they develop Fig. 5  Post-training evaluation survey of educational value (Likert on courses. Despite some studies claiming that participants scale 1–5) retain skills for up to 6 months, others note no difference if previous NTS training has been undertaken [11, 16, 17]. mean Likert score of 4.4 for realism, including for light- The NoTSUS rating scale is significantly correlated to the ing (4.24/5), sound (4.34/5), posters (4.10/5), anaesthetist extensively validated NoTSS scale, proving its suitability for (4.66/5), scrub nurse (4.56/5) and scenario (4.65/5). Quali- evaluating NTS. This scale is therefore appropriate for con- tative feedback received about FIS included that it ‘was a tinued use in further NTS training curricula. Inter-rater reli- highlight’ of training, had ‘well-led debriefing sessions’, ability is demonstrated by comparison of the NoTSUS scores a ‘good, varied range of scenarios’ and ‘great discussion of the two independent expert examiners. A more detailed points’. There was also enthusiasm with regard to propa- investigation should be attempted between a wider pool of gating FIS and setting it up in further areas. Generally, examiners to confirm inter-rater reliability. However, there feedback reflected that the course had ‘brilliant learning was significant differences between faculty members who points’ and ‘should be mandatory for all’. One participant had not received NoTSS training and the two NTS experts, reported that they ‘will pay far more attention to [NTS] highlighting the value of training the trainers. in practice’ and it helped with ‘awareness’ of NTS. Video The overall improvement in the scores of participants feedback was appreciated and was a ‘very well organised, with consecutive scenarios in each group of four highlights productive day’. 13 2236 World Journal of Urology (2021) 39:2231–2237 the value of effective debriefing between scenarios. Based Conclusion on their NoTSUS scores, participants did not make a sig- nificant improvement between scenarios one and four (the In summary, the developed NoTSUS curriculum is a use- first and last scenarios undertaken), perhaps due to less ful addition to surgical training, and its analysis adds rel- numbers undertaking scenario four (n = 11) as opposed to evant information to the ever-growing literature on NTS the former three scenarios (n = 17). This may also be due training in healthcare. The NoTSUS scale was able to to other factors such as fatigue or loss of interest. Debrief- reliably mark participants for scenarios, correlating to the ing aspect of the course was poorly documented and there validated NoTSS scale and demonstrating inter-rater reli- is no feedback on the process. This could be improved for ability. Improvement of skills over the course was demon- the NoTSUS curriculum by using a validated framework strated through total NoTSUS scores and all participants [18], such as the ‘Diamond’ framework [18–23], which is identified self-perceived improvement and satisfaction in specifically designed to help assessors focus on NTS; this all areas of the course. may be important if less experienced faculty are employed for teaching. Candidates observing one another during Acknowledgements This study was partly funded by The Urology each of their scenarios is clearly very helpful; participants Foundation, and conducted as a European School of Urology (ESU) hands-on training course during EAU 2017 (London) and EAU 2018 in other studies agree that it is a good learning opportunity (Copenhagen). The authors would like to thank Olympus, Boston Sci- [24]. OR-based debriefing would be a useful transferable entific and Limbs & Things for providing instruments, consumables, skill from this training to potentially improve retention of and the Uro-Scopic Trainer, respectively. We are also most grateful to skills [25]. Resources should be created or modified for Ton Brouwers (ESU), Catherine Lovegrove, Francesca Kum, Jemma Moody, Mohammed Husnain Iqbal, David Eldred-Evans, Nawal Khan, this purpose, as it differs from simulation debriefing and Raheej Khan, Faizan Dar, Takashige Abe, Nicholas Faure Walker, must fit other logistical criteria such as being brief to fit and Sharanya Palaneer in their assistance in conducting the training in with the busy OR schedule [26]. sessions. The overwhelmingly positive feedback given about the NoTSUS course and the use of FIS is a reason to continue Authors’ contributions AA: project development, data collection, data analysis, manuscript writing/editing. CMG: data collection, data analy- to improve and expand the curriculum in future, and to sis, manuscript writing. OB: project development, data collection. AAJ: keep improving the number of trainees with exposure to data collection, data analysis, manuscript editing. NR: data collection. NTS training. The self-perceived improvement of candi- HA: data collection. CM: data collection, data analysis. JWB: data dates, together with the positive feedback and overall sat- collection, data analysis. MS: data collection, data analysis. JP: project development. MSK: project development. PD: project development. isfaction with the realism of the scenarios, demonstrates KA: project development. educational value and acceptability of the course. Compliance with ethical standard Limitations Conflicts of interest The authors declare that they have no conflict of interest. This study also has a number of limitations. Firstly, Human participants and/or animal No human participants and/or ani- recruited participants were heterogenous in terms of expe- mal data were used in this educational study. rience and exposure to NTS and in different phases of their careers. In its current form, the NoTSUS course is aimed Informed consent Informed consent was sought from all participants. at all levels of urologists. Although less time consuming, training multiple different levels of expertise at once in Open Access This article is licensed under a Creative Commons Attri- this way may reduce teaching quality for seniors; future bution 4.0 International License, which permits use, sharing, adapta- training could benefit from further specialised scenarios tion, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, for higher and lower levels [27]. Furthermore, there were provide a link to the Creative Commons licence, and indicate if changes six sessions which only had 3 participants, which may have were made. The images or other third party material in this article are played a role in the limited results attained in improvement included in the article’s Creative Commons licence, unless indicated through sessions. Secondly, training sessions were not led otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not by the same faculty members. This may have affected permitted by statutory regulation or exceeds the permitted use, you will delivery of the curriculum. The NTS experts were not able need to obtain permission directly from the copyright holder. To view a to assess all sessions and utilise both assessment scales copy of this licence, visit http://creat​iveco​mmons​.org/licen​ses/by/4.0/. for all. This may have affected the results of this study. Finally, participants should subsequently be assessed in the OR, to evaluate transfer of skills. 13 World Journal of Urology (2021) 39:2231–2237 2237 References approach to enhancing operating room performance. Ann Surg 244:139–147 16. Stewart-Parker E, Galloway R, Vig S (2017) S-TEAMS: a 1. Griffin C, Aydin A, Brunckhorst O et al (2020) Non-technical truly multiprofessional course focusing on nontechnical skills skills: a review of training and evaluation in urology. World J Urol to improve patient safety in the operating theater. J Surg Educ. 38:1653–1661 74:137–144 2. Brunckhorst O, Khan MS, Dasgupta P, Ahmed K (2017) Non- 17. Truta TS, Boeriu CM, Copotoiu SM et al (2018) Improving non- technical skill training and the use of scenarios in modern surgical technical skills of an interprofessional emergency medical team education. Curr Opin Urol 27:330–336 through a one day crisis resource management training. Medicine. 3. Parry DA, Oeppen RS, Amin MSA, Brennan PA (2018) Sleep: its 97:e11828 importance and the effects of deprivation on surgeons and other 18. Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A (2016) healthcare professionals. Br J Oral Maxillofac Surg 56:663–666 More than one way to debrief: a critical review of healthcare simu- 4. Aydin A, Ahmed K, Shafi AM, Khan MS, Dasgupta P (2016) The lation debriefing methods. Simul Healthcare 11:209–217 role of simulation in urological training—a quantitative study of 19. Garden AL, Le Fevre DM, Waddington HL, Weller JM (2015) practice and opinions. Surgeon 14:301–307 Debriefing after simulation-based non-technical skill training 5. Somasundram K, Spence H, Colquhoun AJ, McIlhenny C, Biyani in healthcare: a systematic review of effective practice. Anaesth CS, Jain S (2018) Simulation in urology to train non-technical Intensive Care 43:300–308 skills in ward rounds. BJU Int. 122:705–712 20. Jaye P, Thomas L, Reedy G (2015) ‘The Diamond’: a structure for 6. Aydin A, Ahmed K, Van Hemelrijck M et al (2020) Simulation simulation debrief. Clin Teach 12:171–175 in Urological Training and Education (SIMULATE): protocol 21. Isaranuwatchai W, Alam F, Hoch J, Boet S (2017) A cost-effec- and curriculum development of the first multicentre international tiveness analysis of self-debriefing versus instructor debriefing randomized controlled trial assessing the transferability of simu- for simulated crises in perioperative medicine in Canada. J Educ lation-based surgical training. BJU Int. 126:202–211 Eval Health Prof 13:44 7. Aydin A, Raison N, Khan MS, Dasgupta P, Ahmed K (2016) 22. Aghera A, Emery M, Bounds R et al (2018) A randomized trial Simulation-based training and assessment in urological surgery. of SMART goal enhanced debriefing after simulation to promote Nat Rev Urol. 13:503–519 educational actions. West J Emerg Med 19:112–120 8. Kassab E, Tun JK, Arora S et al (2011) “Blowing up the barri- 23. Levett-Jones T, Lapkin S (2014) A systematic review of the effec- ers” in surgical training: exploring and validating the concept of tiveness of simulation debriefing in health professional education. distributed simulation. Ann Surg 254:1059–1065 Nurse Educ Today 34:e58–e63 9. Aydin A, Shafi AM, Shamim Khan M, Dasgupta P, Ahmed K 24. Martin D, Bekiaris B, Hansen G (2017) Mobile emergency simu- (2016) Current status of simulation and training models in uro- lation training for rural health providers. Rural Remote Health logical surgery: a systematic review. J Urol 196:312–320 17:4057 10. Ounounou E, Aydin A, Brunckhorst O, Khan MS, Dasgupta P, 25. Brunckhorst O, Khan MS, Dasgupta P, Ahmed K (2015) Effective Ahmed K (2019) Nontechnical skills in surgery: a systematic non-technical skills are imperative to robot-assisted surgery. BJU review of current training modalities. J Surg Educ. 76:14–24 Int. 116:842–844 11. Gostlow H, Marlow N, Thomas MJ et al (2017) Non-technical 26. Halamek LP (2016) Simulation and debriefing in neonatology skills of surgical trainees and experienced surgeons. Br J Surg 2016: mission incomplete. Semin Perinatol 40:489–493 104:777–785 27. de Vries AH, Schout BM, van Merrienboer JJ et al (2017) High 12. Preece R (2015) The current role of simulation in urological train- educational impact of a national simulation-based urological cur- ing. Central European journal of urology. 68:207–211 riculum including technical and non-technical skills. Surg Endosc 13. Abdelshehid CS, Quach S, Nelson C et al (2013) High-fidelity 31:928–936 simulation-based team training in urology: evaluation of technical and nontechnical skills of urology residents during laparoscopic Publisher’s Note Springer Nature remains neutral with regard to partial nephrectomy. J Surg Educ. 70:588–595 jurisdictional claims in published maps and institutional affiliations. 14. Brewin J, Tang J, Dasgupta P et al (2015) Full immersion sim- ulation: validation of a distributed simulation environment for technical and non-technical skills training in Urology. BJU Int. 116:156–162 15. Moorthy K, Munz Y, Forrest D et al (2006) Surgical crisis manage- ment skills training and assessment: a simulation[corrected]-based 13