The Predictive Validity of the ABFM’s In-Training Examination

2015

Author(s)

O’Neill, Thomas R, Li, Z, Peabody, Michael R, Lybarger, M, Royal, Kenneth D, and Puffer, James C

Topic(s)

Education & Training, and Family Medicine Certification

Keyword(s)

Cognitive Expertise, Graduate Medical Education, In-Training Examination, and Psychometrics

Volume

Family Medicine

Source

Family Medicine

BACKGROUND AND OBJECTIVES:

Our objective was to examine the predictive validity of the American Board of Family Medicine’s (ABFM) In-Training Examination (ITE) with regard to predicting outcomes on the ABFM certification examination.

METHODS:

This study used a repeated measures design across three levels of medical training (PGY1–PGY2, PGY2–PGY3, and PGY3–initial certification) with three different cohorts (2010–2011, 2011–2012, and 2012–2013) to examine: (1) how well the residents’ ITE scores correlated with their test scores in the following year, (2) what the typical score increase was across training years, and (3) what was the sensitivity, specificity, positive predictive value, and negative predictive value of the PGY3 scores with regard to predicting future results on the MC-FP Examination. RESULTS: ITE scores generally correlate at about .7 with the following year’s ITE or with the following year’s certification examination. The mean growth from PGY1 to PGY2 was 52 points, from PGY2 to PGY3 was 34 points, and from PGY3 to initial certification was 27 points. The sensitivity, specificity, positive predictive value, and negative predictive value were .91, .47, .96, and .27, respectively. CONCLUSION: The ITE is a useful predictor of future ITE and initial certification examination performance.