NP Admissions--Revising Graduate Nursing Interview Scoring to Improve Inter-Rater Reliability

The MSN-nurse practitioner program at a university has been in existence since 1976. In 2011, a BSN-DNP and MSN-DNP program was approved and the first cohort of students was admitted in January 2013. Since the admission criteria for the MSN program had resulted in a six-year graduation/completion rate of 90%, these criteria were reviewed and revised to reflect slightly more competitive criteria for the DNP program. Required undergraduate GPA was established at 3.0 for MSN and DNP, GRE writing score was 3.0 for MSN and 3.5 for DNP, and the face to face, scored interview was required for both programs. Prior to interview revision, the first cohort of DNP students’ graduation rate was 100%. Though the all of the first DNP cohort graduated and MSN admission and graduation rates were stable, faculty identified inter-rater reliability of interview scoring as a problem across graduate nursing admissions. Faculty also noted that though three reference letters were required for each applicant, these were not being utilized in admissions decisions. 

A committee comprised of three faculty members reviewed the current interview questions and scoring mechanism by reviewing ten interview score sheets from each of the last three years as scored by different pairs of faculty members. Though two faculty normally interviewed each applicant together, wide variations (± 5-15 points) in scoring were verified on 50% of interviews. 

The interview score sheet was revised from a system of scoring each question from a minimum or maximum value to a detailed rubric on most questions (references, faculty evaluation, and bonus point areas excluded). The overall interview score was decreased from 100 possible points to 77 in an effort to tighten scoring and improve overall scoring consistency. 

On final review, faculty agreed to continue with two faculty interviewing each applicant and to implement the revised interview scoring sheet with the current MSN and DNP applicants. A comparison of interview score consistency will be reviewed following early admissions and upon close of all admissions for the next academic year beginning in August 2016. These will continue to be tracked and data trended for admissions, retention, and graduation rates.