Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Click here to sign up for SAGE Journal Email Alerts today!

Sign In to gain access to subscriptions and/or personal tools.
Evaluation & the Health Professions
This Article
Right arrow Full Text (PDF)
Right arrow References
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Right arrow Citing Articles via Scopus
Google Scholar
Right arrow Articles by Raymond, M. R.
Right arrow Articles by Houston, W. M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Raymond, M. R.
Right arrow Articles by Houston, W. M.
Social Bookmarking
Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?

Correcting Performance-Rating Errors in Oral Examinations

Mark R. Raymond

Lynn C. Webb

Walter M. Houston

American College Testing, Iowa City, LA

Although oral examinations are widely used for making decisions regarding an individual s level of competence, they are frequently of limited reliability. A significant part of the error in oral performance ratings is due to the tendency for some evaluators to be lenient and others to be stringent in their assignment of ratings. This article describes and evaluates a simple method to identify and correct for errors of leniency and stringency. The method, which is based on a regression model recommended by Wilson (1988), extends and simplifies the procedures recommended by Cason and Cason (1984, 1985). The method provides an estimate of each individual's performance that has been corrected for errors of leniency and stringency. In addition, it produces for each rater an index of leniency or stringency and several other statistics useful in evaluating the properties of rating data. The regression method is applied to performance ratings from three separate administrations of an oral examination in a medical specialty. The results indicate modest but significant levels of leniency and stringency error; correcting for such errors would change the pass/fail decisions for about 6% of the examinees. Limitations of the procedure, as well as the need for additional research, ore discussed.

Evaluation & the Health Professions, Vol. 14, No. 1, 100-122 (1991)
DOI: 10.1177/016327879101400107
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter    What's this?


This article has been cited by other articles:


Home page
 Arch. Dis. Child.Home page
N Khera, H Davies, H Davies, T Lissauer, D Skuse, R Wakeford, and J Stroobant
How should paediatric examiners be trained?
Arch. Dis. Child., January 1, 2005; 90(1): 43 - 47.
[Abstract] [Full Text] [PDF]

Home page
 BMJHome page
R. Wakeford, L. Southgate, and V. Wass
Improving oral examinations: selecting, training, and monitoring examiners for the MRCGP
BMJ, October 7, 1995; 311(7010): 931 - 935.
[Full Text]

Home page
 Educational and Psychological MeasurementHome page
M. E. Lunz, J. A. Stahl, and B. D. Wright
Interjudge Reliability and Decision Reproducibility
Educational and Psychological Measurement, December 1, 1994; 54(4): 913 - 925.
[Abstract]

Home page
 Applied Psychological MeasurementHome page
W. M. Houston, M. R. Raymond, and J. C. Svec
Adjustments for Rater Effects in Performance Assessment
Applied Psychological Measurement, December 1, 1991; 15(4): 409 - 421.
[Abstract]