Login now to access Regular content available to all registered users.
Abstract
Discussion Forum (0)
Introduction: In 2022, the United States Medical Licensing Examination (USMLE) Step 1 will adopt pass/fail scoring. In the absence of numerical values, program directors will have to rely on alternate metrics to evaluate applicants for residency. We sought to validate metrics predictive of performance.
Methods: We retrospectively collected electronic residency application service (ERAS) application data for Wayne State University Obstetrics and Gynecology residents between 2013-2018 (n=75). Yearly CREOG in-service examination scores were used as markers of residency performance. Statistical analysis was performed using SPSS with significance set at p < 0.05.
Results: Mean USMLE score correlated with CREOG performance (p < 0.001) and Step 1 had the tightest association (p < 0.001). MSPE (p=0.002) and class percentile (p=0.002) also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with CREOGs in our cohort (p=0.961 and p=0.896). Setting the passing threshold for Step 1 to 209 correlated with average CREOGs of 199 (R2=0.45).
Conclusion/Implications: While no single metric had the predictive performance of Step 1, our regression suggests that the combination of class percentile and MSPE wording with Step 2 scores may allow program directors to regain some ability to predict a candidate's performance in its absence. Additional tests and methods for assessment may be needed to assess residency applicants in the absence of numerical Step 1 scores.
Methods: We retrospectively collected electronic residency application service (ERAS) application data for Wayne State University Obstetrics and Gynecology residents between 2013-2018 (n=75). Yearly CREOG in-service examination scores were used as markers of residency performance. Statistical analysis was performed using SPSS with significance set at p < 0.05.
Results: Mean USMLE score correlated with CREOG performance (p < 0.001) and Step 1 had the tightest association (p < 0.001). MSPE (p=0.002) and class percentile (p=0.002) also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with CREOGs in our cohort (p=0.961 and p=0.896). Setting the passing threshold for Step 1 to 209 correlated with average CREOGs of 199 (R2=0.45).
Conclusion/Implications: While no single metric had the predictive performance of Step 1, our regression suggests that the combination of class percentile and MSPE wording with Step 2 scores may allow program directors to regain some ability to predict a candidate's performance in its absence. Additional tests and methods for assessment may be needed to assess residency applicants in the absence of numerical Step 1 scores.
Introduction: In 2022, the United States Medical Licensing Examination (USMLE) Step 1 will adopt pass/fail scoring. In the absence of numerical values, program directors will have to rely on alternate metrics to evaluate applicants for residency. We sought to validate metrics predictive of performance.
Methods: We retrospectively collected electronic residency application service (ERAS) application data for Wayne State University Obstetrics and Gynecology residents between 2013-2018 (n=75). Yearly CREOG in-service examination scores were used as markers of residency performance. Statistical analysis was performed using SPSS with significance set at p < 0.05.
Results: Mean USMLE score correlated with CREOG performance (p < 0.001) and Step 1 had the tightest association (p < 0.001). MSPE (p=0.002) and class percentile (p=0.002) also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with CREOGs in our cohort (p=0.961 and p=0.896). Setting the passing threshold for Step 1 to 209 correlated with average CREOGs of 199 (R2=0.45).
Conclusion/Implications: While no single metric had the predictive performance of Step 1, our regression suggests that the combination of class percentile and MSPE wording with Step 2 scores may allow program directors to regain some ability to predict a candidate's performance in its absence. Additional tests and methods for assessment may be needed to assess residency applicants in the absence of numerical Step 1 scores.
Methods: We retrospectively collected electronic residency application service (ERAS) application data for Wayne State University Obstetrics and Gynecology residents between 2013-2018 (n=75). Yearly CREOG in-service examination scores were used as markers of residency performance. Statistical analysis was performed using SPSS with significance set at p < 0.05.
Results: Mean USMLE score correlated with CREOG performance (p < 0.001) and Step 1 had the tightest association (p < 0.001). MSPE (p=0.002) and class percentile (p=0.002) also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with CREOGs in our cohort (p=0.961 and p=0.896). Setting the passing threshold for Step 1 to 209 correlated with average CREOGs of 199 (R2=0.45).
Conclusion/Implications: While no single metric had the predictive performance of Step 1, our regression suggests that the combination of class percentile and MSPE wording with Step 2 scores may allow program directors to regain some ability to predict a candidate's performance in its absence. Additional tests and methods for assessment may be needed to assess residency applicants in the absence of numerical Step 1 scores.
Impact of USMLE Step 1 Pass/Fail scoring on OB/GYN Residency Applicant Evaluation
Dr. Joseph Levy
Affiliations:
Wayne State University
Wayne State University
Abstract
Discussion Forum (0)
Introduction: In 2022, the United States Medical Licensing Examination (USMLE) Step 1 will adopt pass/fail scoring. In the absence of numerical values, program directors will have to rely on alternate metrics to evaluate applicants for residency. We sought to validate metrics predictive of performance.
Methods: We retrospectively collected electronic residency application service (ERAS) application data for Wayne State University Obstetrics and Gynecology residents between 2013-2018 (n=75). Yearly CREOG in-service examination scores were used as markers of residency performance. Statistical analysis was performed using SPSS with significance set at p < 0.05.
Results: Mean USMLE score correlated with CREOG performance (p < 0.001) and Step 1 had the tightest association (p < 0.001). MSPE (p=0.002) and class percentile (p=0.002) also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with CREOGs in our cohort (p=0.961 and p=0.896). Setting the passing threshold for Step 1 to 209 correlated with average CREOGs of 199 (R2=0.45).
Conclusion/Implications: While no single metric had the predictive performance of Step 1, our regression suggests that the combination of class percentile and MSPE wording with Step 2 scores may allow program directors to regain some ability to predict a candidate's performance in its absence. Additional tests and methods for assessment may be needed to assess residency applicants in the absence of numerical Step 1 scores.
Methods: We retrospectively collected electronic residency application service (ERAS) application data for Wayne State University Obstetrics and Gynecology residents between 2013-2018 (n=75). Yearly CREOG in-service examination scores were used as markers of residency performance. Statistical analysis was performed using SPSS with significance set at p < 0.05.
Results: Mean USMLE score correlated with CREOG performance (p < 0.001) and Step 1 had the tightest association (p < 0.001). MSPE (p=0.002) and class percentile (p=0.002) also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with CREOGs in our cohort (p=0.961 and p=0.896). Setting the passing threshold for Step 1 to 209 correlated with average CREOGs of 199 (R2=0.45).
Conclusion/Implications: While no single metric had the predictive performance of Step 1, our regression suggests that the combination of class percentile and MSPE wording with Step 2 scores may allow program directors to regain some ability to predict a candidate's performance in its absence. Additional tests and methods for assessment may be needed to assess residency applicants in the absence of numerical Step 1 scores.
Introduction: In 2022, the United States Medical Licensing Examination (USMLE) Step 1 will adopt pass/fail scoring. In the absence of numerical values, program directors will have to rely on alternate metrics to evaluate applicants for residency. We sought to validate metrics predictive of performance.
Methods: We retrospectively collected electronic residency application service (ERAS) application data for Wayne State University Obstetrics and Gynecology residents between 2013-2018 (n=75). Yearly CREOG in-service examination scores were used as markers of residency performance. Statistical analysis was performed using SPSS with significance set at p < 0.05.
Results: Mean USMLE score correlated with CREOG performance (p < 0.001) and Step 1 had the tightest association (p < 0.001). MSPE (p=0.002) and class percentile (p=0.002) also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with CREOGs in our cohort (p=0.961 and p=0.896). Setting the passing threshold for Step 1 to 209 correlated with average CREOGs of 199 (R2=0.45).
Conclusion/Implications: While no single metric had the predictive performance of Step 1, our regression suggests that the combination of class percentile and MSPE wording with Step 2 scores may allow program directors to regain some ability to predict a candidate's performance in its absence. Additional tests and methods for assessment may be needed to assess residency applicants in the absence of numerical Step 1 scores.
Methods: We retrospectively collected electronic residency application service (ERAS) application data for Wayne State University Obstetrics and Gynecology residents between 2013-2018 (n=75). Yearly CREOG in-service examination scores were used as markers of residency performance. Statistical analysis was performed using SPSS with significance set at p < 0.05.
Results: Mean USMLE score correlated with CREOG performance (p < 0.001) and Step 1 had the tightest association (p < 0.001). MSPE (p=0.002) and class percentile (p=0.002) also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with CREOGs in our cohort (p=0.961 and p=0.896). Setting the passing threshold for Step 1 to 209 correlated with average CREOGs of 199 (R2=0.45).
Conclusion/Implications: While no single metric had the predictive performance of Step 1, our regression suggests that the combination of class percentile and MSPE wording with Step 2 scores may allow program directors to regain some ability to predict a candidate's performance in its absence. Additional tests and methods for assessment may be needed to assess residency applicants in the absence of numerical Step 1 scores.
{{ help_message }}
{{filter}}