|
Sign In to gain access to subscriptions and/or personal tools.
|
Examination of a Clinical Teaching Effectiveness Instrument Used for Summative Faculty Assessment
S. Beth Bierer
Cleveland Clinic Lerner College of Medicine of Case Western Reserve University
Alan L. Hull
Cleveland Clinic Lerner College of Medicine of Case Western Reserve University
This study explores whether a clinical teaching effectiveness (CTE) instrument provides valid scores for summative faculty assessment. The sample included all CTE instruments (n = 10,087) that learners (N = 1,194) completed to assess clinical teachers (N = 872) during 1 academic year. The authors investigated response processes (e.g., missing data, straight-line responses, level of learner), internal structure (e.g., confirmatory and exploratory factor analysis), teaching ratings by learner group (medical student or resident), and relation to other variables (e.g., correlation with global rating). Response processes identified a high prevalence of straight-line responses (same rating across all items) and differential patterns of missing data by learner group. Medical students rated their teachers higher than residents, and CTE scores had different factor structures depending on learner group. High correlation coefficients of CTE items with a single rating of overall teaching performance suggest that learners consider global performance when assessing clinical teaching performance.
Key Words: clinical teaching effectiveness • validity
References
- Abrami, P.C., & d'Apollonia, S. (1990). The dimensionality of ratings and their use in personnel decisions. In M. Theall & J. Franklin (Eds.), Student ratings of instruction: Issues for improving practice; New directions for teaching and learning (No. 43, pp. 97-111). San Francisco: Jossey-Bass.
- Albanese, M.A. (1999). Challenges in using rater judgments in medical education. Journal of Evaluation in Clinical Practice, 6(3), 305-319.[Web of Science]
- Albanese, M.A., Prucha, C., Barnet, J., & Gjerde, C. (1997). The effect of right or left placement of the positive response on Likert-type scales used by medical students for rating instruction. Academic Medicine, 72(7), 627-630.[Web of Science][Medline]
[Order article via Infotrieve]
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
- Assessing change in medical education—The road to implementation [Special issue]. (1993). Academic Medicine, 68(6).
- Cashin, W.E. (1990). Student ratings of teaching: Recommendations for use (Tech. Rep. No. 22). Manhattan, KS: Kansas State University, Center for Faculty Evaluation and Development.
- Cashin, W.E., & Downey, R.G. (1992). Using global student rating items for summative evaluation. Journal of Educational Psychology, 84(4), 563-572.[CrossRef][Web of Science]
- Copeland, H.L., & Hewson, M.G. (2000). Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Academic Medicine, 75(2), 161-166.[Web of Science][Medline]
[Order article via Infotrieve]
- Dillman, D. (2000). Mail and internet surveys: The tailored design method. New York: Wiley.
- Dillman, D., Tortora, R., Conradt, J., & Bowker, D. (1998, August). Influence of plain versus fancy design on response rates for Web surveys. Paper presented at the annual meeting of the American Statistical Association, Dallas, TX.
- Hayward, R., Williams, B., Gruppen, L., & Rosenbaum, D. (1995). Measuring attending physician performance in a general medicine outpatient clinic. Journal of General Internal Medicine, 10, 504-509.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Howell, L., Hogarth, M., & Anders, T. (2002). Creating a mission-based reporting system at an academic health center. Academic Medicine, 77(2), 130-138.[Web of Science][Medline]
[Order article via Infotrieve]
- Hoyle, R., & Panter, A. (1995). Writing about structural equation models. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 158-176). Thousand Oaks, CA: Sage.
- Irby, D.M. (1978). Clinical teacher effectiveness in medicine. Journal of Medical Education, 53, 808-815.[Web of Science][Medline]
[Order article via Infotrieve]
- Irby, D.M., Gillmore, G., & Ramsey, P. (1987). Factors affecting ratings of clinical teachers by medical students and residents. Journal of Medical Education, 67, 1-7.
- Irby, D.M., Ramsey, P., Gillmore, G., & Schaad, D. (1991). Characteristics of effective clinical teachers of ambulatory care medicine. Academic Medicine, 66(1), 54-55.[Web of Science][Medline]
[Order article via Infotrieve]
- Kline, R. (1998). Principles and practice of structural equation modeling. New York: Guilford.
- Jarrell, B., Mallot, D., Peartree, L., & Calia, F. (2002). Looking at the forest instead of counting the trees: An alternative method for measuring faculty's clinical education efforts. Academic Medicine, 77(12), 1255-1261.[Web of Science][Medline]
[Order article via Infotrieve]
- Johnston, M., & Gifford, R. (1996). A model for distributing teaching funds to faculty. Academic Medicine, 71(2), 138-140.[Web of Science][Medline]
[Order article via Infotrieve]
- Litzelman, D., Stratos, G., Marriott, D., & Skeff, K. (1998). Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Academic Medicine, 73(6), 688-695.[Web of Science][Medline]
[Order article via Infotrieve]
- Mallon, W., & Jones, R. (2002). How do medical schools use measurement systems to track faculty activity and productivity in teaching? Academic Medicine, 77(2), 115-123.[Web of Science][Medline]
[Order article via Infotrieve]
- Marsh, H.W., & Roche, L.A. (1997). Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. American Psychologist, 52(11), 1187-1197.[CrossRef]
- McKeachie, W. (1997). Student ratings: The validity of use. American Psychologist, 52, 1218-1225.[CrossRef]
- Nutter, D., Bond, J., Coller, B., D'Alessandri, R., Gewertz, B., Nora, L., et al. (2000). Measuring faculty effort and contributions in medical education. Academic Medicine, 75(2), 200-207.[Web of Science]
- Parsell, G., & Bligh, J. (2001). Recent perspectives on clinical teaching. Medical Education, 35, 409-414.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Paukert, J., & Richards, B. (2000). How medical students and residents describe the roles and characteristics of their influential clinical teachers. Academic Medicine, 75(8), 843-845.[Web of Science][Medline]
[Order article via Infotrieve]
- Roop, S., & Pangaro, L. (2001). Effect of clinical teaching on student performance during a medicine clerkship. American Journal of Medicine, 110, 205-209.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Schuwirth, L.W., & van der Vleuten, C.P. (2004). Changing education, changing assessment, changing research? Medical Education, 38, 805-812.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Scriven, M. (1989). The design and use of forms for the student evaluation of teaching. Instructional Evaluation, 10, 1-13.
- Shea, J., & Bellini, L. (2002). Evaluations of clinical faculty: The impact of level of learner and time of year. Teaching and Learning in Medicine, 14(2), 87-91.
- Stevens, J. (1996). Confirmatory and exploratory factor analysis. In Applied multivariate statistics for the social sciences (3rd ed., pp. 362-428). Mahwah, NJ: Erlbaum.
- Stritter, F., Hain, J., & Grimes, D. (1975). Clinical teaching reexamined. Journal of Medical Education, 50, 876-882.[Web of Science][Medline]
[Order article via Infotrieve]
- Ullian, J., Bland, C., & Simpson, D. (1994). An alternative approach to defining the role of the clinical teacher. Academic Medicine, 69(10), 832-838.[Web of Science][Medline]
[Order article via Infotrieve]
- van der Hem-Stokroos, H., van der Vleuten, C., Daelmans, H., Haarman, H., & Scherpbier, A. (2005). Reliability of the clinical teaching effectiveness instrument. Medical Education, 39, 904-910.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- West, S.G., Finch, J.F., & Curran, P.J. (1995). Structural equation models with nonnormal variables: problems and remedies. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 56-75). Thousand Oaks, CA: Sage.
- Williams, B., Litzelman, D., Babbott, S., Lubitz, R., & Hofer, T. (2002). Validation of a global measure of faculty's clinical teaching performance. Academic Medicine, 77(2), 177-180.[Web of Science][Medline]
[Order article via Infotrieve]
- Williams, B., Pillsbury, M., Stern, D., & Grum, C. (2001). Comparison of resident and medical student evaluation of faculty teaching. Evaluation & the Health Professions, 24(1), 53-60.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
Evaluation & the Health Professions, Vol. 30, No. 4,
339-361 (2007)
DOI: 10.1177/0163278707307906
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
A. Schubert
Faculty Teaching Scores: Validating Evaluations, Evaluating Validation
Anesth. Analg.,
October 1, 2008;
107(4):
1098 - 1099.
[Full Text]
[PDF]
|
|
|
|
|
