AAPS Main SiteTrainee Candidate Ranking Criteria Correlated with Fellow Performance: a Controlled, Matched Cohort Evaluation
Jesse C. Selber, MD, MPH, Winnie Tong, MD, Courtney El-Zokm, MD, Jun Liu, MSc, Amir Ibrahim, MD, Charles Butler, MD.
M,D, Anderson Cancer Center, Houston, TX, USA.
PURPOSE:
One of the major goals of the trainee interview is to identify candidates who are most likely to perform well in the training program and ultimately as independent practitioners. However, the effectiveness of the various ranking processes has not been fully examined. The purpose of this study was to evaluate whether our current process of rating and ranking microvascular fellow candidates is correlated with performance over the training year. The goal was to determine which candidate factors have the highest positive predictive value for good performance so that we can weight them appropriately in future rankings.
METHODS:
We reviewed the selection data for microsurgery fellows at The University of Texas MD Anderson Cancer Center from 2008 to 2012. We compared the recruitment ranking to candidate performance evaluations at the end of training. Both the ranking and final evaluations are standardized, internet-based scoring systems. Fifteen faculty members evaluated 29 fellows for the study, systematically grading them on seven characteristics: presentation, plastic surgery (PS) training, academic potential (AP), personality, social skills, communication skills, and team player ability. Each interviewer also gave an overall score and a final ranking. After completing one year of training, each fellow was scored from 1 to 5 by the faculty in six categorical areas: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and system-based practice. Spearman correlation coefficients (r) were used to measure the correlations between interview performance and end-of-training performance.
RESULTS:
PS training (an aggregate of training program strength, mentorship, and individual experience) had the strongest correlation to overall fellow performance (r:0.678, p<0.001) (Table 1). The overall candidate ranking was correlated with final performance (r:-0.378, p=0.043). Compared with other characteristics, PS training and AP were≈ more highly correlated with patient care, medical knowledge, practice-based learning, interpersonal and communication skills, professionalism, and system-based practice. The overall impression (r:0.470, p=0.010) (r:-0.378, p=0.043) was better correlated with final performance than was the final ranking. Candidates who scored highly as good presenters or team players were more likely be ranked higher but did not necessarily perform better.
CONCLUSION:
Candidate ranking was less well correlated with fellow performance than were PS training and AP, the two highest predictors of performance in our study. In the subsequent group discussion, we made changes to the aggregate faculty scores to create the rank list; however, this did not improve our ability to identify the best performers. Adjusting the ranking to more heavily weight PS training and AP is likely beneficial, as is determining what specifically contributes to high candidate scores in these areas. Our ranking system will be altered to account for these findings.
| Interview Subjects | r | p value |
| Overall Impression | 0.470 | 0.010 |
| Presentation | 0.326 | 0.085 |
| Plastic Surgery Training | 0.678 | <0.001 |
| Academic Potential | 0.560 | 0.002 |
| Personality | 0.121 | 0.531 |
| Social Skills | 0.167 | 0.386 |
| Communication Skills | 0.343 | 0.068 |
| Team Player | 0.383 | 0.041 |
| Ranking | -0.378 | 0.043 |
Back to Program
