This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Lunz, M. E. Articles by Stahl, J. A. Search for Related Content PubMed Articles by Lunz, M. E. Articles by Stahl, J. A. Social Bookmarking                         What's this?

Judge Consistency and Severity Across Grading Periods

John A. Stahl

American Society of Clinical Pathologists

The purpose of this research project was to confirm that differences in the severity of judges and the stringency of grading periods occur, regardless of the nature of the assessment or the examination materials used. Three rather different examinations that require judges were analyzed, using an extended Rasch model to determine whether differences in judge severity and grading-period stringency were observable for all three examinations. Significant variation in judge severity and some variation across grading periods were found on all three examinations. This implies that regardless of the nature of the examination, items, or judges, examinee/measures cannot be considered independent of the particular judges involved unless correction for severity is made systematically. Accounting for judge severity and gradinig-period stringency is extremely important when pass/fail decisions that are meant to generalize to competence are made, as in certification examinations.

Evaluation & the Health Professions, Vol. 13, No. 4, 425-444 (1990)
DOI: 10.1177/016327879001300405


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				Y.-H. Kim An investigation into native and non-native teachers' judgments of oral English performance: A mixed methods approach Language Testing, April 1, 2009; 26(2): 187 - 217. [Abstract] [PDF]

				C. Elder, G. Barkhuizen, U. Knoch, and J. von Randow Evaluating rater responses to an online training program for L2 writing assessment Language Testing, January 1, 2007; 24(1): 37 - 64. [Abstract] [PDF]

				B. Garrett, E. Towles, H. Kleinert, and J. Kearns Portfolios in Large-Scale Alternate Assessment Systems: Frameworks for Reliability Assessment for Effective Intervention, January 1, 2003; 28(2): 17 - 27. [Abstract] [PDF]

				K. Kondo-Brown A FACETS analysis of rater bias in measuring Japanese second language writing performance Language Testing, January 1, 2002; 19(1): 3 - 31. [Abstract] [PDF]

				J. A. Upshur and C. E. Turner Systematic effects in the rating of second-language speaking ability: test method and learner discourse Language Testing, January 1, 1999; 16(1): 82 - 111. [Abstract] [PDF]

				S. C. Weigle Using FACETS to model rater training effects Language Testing, April 1, 1998; 15(2): 263 - 287. [Abstract] [PDF]

				L. H. Ludlow and S. M. Haley Effect of Context in Rating of Mobility Activities in Children with Disabilities: An Assessment Using the Pediatric Evaluation of Disability Inventory Educational and Psychological Measurement, February 1, 1996; 56(1): 122 - 129. [Abstract]

				T. Lumley and T.F. McNamara Rater characteristics and rater bias: implications for training Language Testing, March 1, 1995; 12(1): 54 - 71. [Abstract] [PDF]