Psychometric Evaluation of Advanced Practice Student Competencies Using Standardized Patients

Introduction.  For over ten years, standardized patients (SPs) have been used in medical school curricula to teach and evaluate clinical skills in a safe and controlled environment.  The reliability and validity of SPs have been well established within medical education showing 88-92% agreement on performance checklists between faculty and SPs which has served as the foundation for the U.S. Medical Licensing Examination Step II where all medical students must pass multiple SP encounters for licensure. The more recent introduction of SP performance checklists into nursing school curricula has not been accompanied with an equally rigorous evaluation of their psychometric properties, thus it is unclear whether their utility is equivalent to those found in medical education.

Methods.  This paper reports the findings of a study designed to evaluate a valid and reliable method for assessing the clinical competencies of advanced practice nursing students – nurse practitioners and clinical nurse specialists - in the early stages of their academic program. Students were enrolled in a graduate nursing program at a private university in the southwestern region of the United States.  The purpose of this study was twofold. First, the performance checklists for evaluating student competencies in history-taking, physical examination, communication, and organization were re-designed to improve construct validity; second, the investigators set out to determine the inter-rater reliability of the 50-72 item performance checklist evaluation tool between faculty and SPs.

Results.   After construct validity of the evaluation tool was established, data analysis of faculty and SP scores on the redesigned performance checklists found significant agreement between 87.04% to 92.14% on history and physical examination items.  Differences in faculty-SP scores in the communication and organizational domains were found only in 2 of 20 items.

Conclusion: Study results support the ability of SPs, as demonstrated in the medical literature with medical students, to accurately evaluate advanced practice nursing student performance.  The potential implication of this study include a significant impact and reduction of nursing faculty workload hours in the assessment process of advance practice students by incorporating standardized patients into the evaluation process.