This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Gross, L. J. Search for Related Content PubMed Articles by Gross, L. J. Social Bookmarking                         What's this?

Interrater Reliability Reconsidered

Performance Assessment using One Examiner per Candidate

National Board of Examiners in Optometry

The purpose of this study was to determine if adequate levels of interrater reliability could be attained on a national standardizedperformance test, using one examiner per observation. Traditionally, performance assessments have relied on two examiners because of the subjectivity involved in grading. However, for nationally administered health-professions performance tests involving process evaluation, cost and logistical issues render two examiners unrealistic. Datafor this study were obtained during a national clinical skills examination for optometry. Surplus examiners were available to serve as secondary examiners in four evaluation stations, yielding 101 paired candidate observations. The results indicated high levels of interrater reliability. At each station, the agreement between examiners in their dichotomous yes-no observations was 90% or greater The agreement among examiner-determined scores based on the differentially weighted items was greater These results indicate that adequate, psychometrically sound examiner judgments and scores can be attained with one examiner per observation.

Evaluation & the Health Professions, Vol. 17, No. 4, 465-484 (1994)
DOI: 10.1177/016327879401700407


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?