|
Sign In to gain access to subscriptions and/or personal tools.
|
Assessment of Family Physicians' Performance Using Patient ChartsInterrater Reliability and Concordance With Chart-Stimulated Recall Interview
François Goulet
Collège des médecins du Québec, Montreal, goulet.cmq{at}sympatico.ca.
André Jacques
Collège des médecins du Québec, Montreal
Robert Gagnon
Collège des médecins du Québec, Montreal
Pierre Racette
Collège des médecins du Québec, Montreal
William Sieber
University of California, San Diego
Peer-assessment processes with chart review have been used for many years to assess the clinical performance of physicians. The Quebec medical licensing authority has been required by provincial law to assess the practicing Quebec physicians on a nonvoluntary basis. During the period from January 2001 to November 2004, 25 family physicians in active practice were randomly selected from a pool of about 300. For each physician, 25 to 40 patients' medical charts were randomly selected to evaluate the interrater reliability of peer-review assessment of medical charts and to compare ratings based on chart review with a chart-stimulated recall interview to those based on chart review alone. The concordance between chart review alone and that of chart review with chart-stimulated recall interview was 75% for chart keeping, 69% for clinical investigation, 81% for diagnostic accuracy, and 74% for treatment plan. Ratings based on chart review alone achieve moderate levels of reliability (Kappa = 0.44 to 0.56). It appears that some important information about quality of care is missed when only chart review is used.
Key Words: family physicians' clinical performance assessments • peer-assessment review • interrater reliability • chart-stimulated recall interview
References
- Brennan, T.A., Leape, L.L., Laird, N.M., Hebert, L., Localio, A.R., Lawthers, A.G., et al. (1991). Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. New England Journal of Medicine, 324(6), 370-376.[Abstract]
- Brook, R.H., & Lohr, K.N. (1987). Monitoring quality of care in the Medicare program. Two proposed systems. Journal of the American Medical Association, 258(21), 3138-3141.[CrossRef]
- Cunnington, J.P., Hanna, E., Turnbull, J., Kaigas, T.B., & Norman, G.R. (1997). Defensible assessment of the competency of the practicing physician. Academic Medicine, 72(1), 9-12.[Web of Science][Medline]
[Order article via Infotrieve]
- Dubois, R.W., Rogers, W.H., Moxley, III, J.H., Draper, D., & Brook, R.H. (1987). Hospital inpatient mortality. Is it a predictor of quality? New England Journal of Medicine, 317(26), 1674-1680.[Abstract]
- Eliasson, G., Berg, L., Carlsson, P., Lindstrom, K., & Bengtsson, C. (1998). Facilitating quality improvement in primary health care by practice visiting. Quality in Health Care, 7(1), 48-54.[Web of Science][Medline]
[Order article via Infotrieve]
- Faulkner, D. (1998). Dear doctor: You have been selected for assessment by the Peer Assessment Program. Members; Dialogue, 6(4), 8-13.
- Fleiss, J.L. (1981). Statistical methods for rates and proportions (2nd ed.). New York: Wiley.
- Goldman, R.L. (1992). The reliability of peer assessments of quality of care. Journal of the American Medical Association, 267(7), 958-960.[CrossRef]
- Goldman, R.L. (1994). The reliability of peer assessments. A meta-analysis. Evaluation & the Health Professions, 17(1), 3-21.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Goulet, F., Jacques, A., Gagnon, R., Bourbeau, D., Laberge, D., Melanson, J., et al. (2002). Performance assessment. Family physicians in Montreal meet the mark! Canadian Family Physician, 48, 1337-1344.
- Hofer, T.P., Asch, S.M., Hayward, R.A., Rubenstein, L.V., Hogan, M.M., Adams, J., & Kerr, E.A. (2004). Profiling quality of care: Is there a role for peer review? BMC Health Services Research, 4(1), 9.[CrossRef]
- Landis, J.R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
- Lockyer, J., & Harrison, V. (1994). Performance assessment: The role of chart review analysis and CME. In D. A. Davis & R. D. Fox (Eds.), The physician as learner: Linking research to practice (pp. 169-186). Chicago: American Medical Association.
- Mann, K.V. (2002). Continuing medical education: Methods for both needs assessment and performance assessment. In G. R. Norman, C. P. M. van der Vleuten, & D. I. Newble (Eds.), International handbook of research in medical education: Part one (pp. 428-429). Dordrecht, The Netherlands: Kluwer Academic Publishers.
- McAuley, R.G., & Henderson, H.W. (1984). Results of the peer assessment program of the College of Physicians and Surgeons of Ontario. Canadian Medical Association Journal, 131(6), 557-561.[Abstract]
- McAuley, R.G., Paul, W.M., Morrison, G.H., Beckett, R.F., & Goldsmith, C.H. (1990). Five-year results of the peer assessment program of the College of Physicians and Surgeons of Ontario. Canadian Medical Association Journal, 143(11), 1193-1199.[Abstract]
- Norman, G.R., Davis, D.A., Lamb, S., Hanna, E., Caulford, P., & Kaigas, T. (1993). Competency assessment of primary care physicians as part of a peer review program. Journal of the American Medical Association, 270(9), 1046-1051.[CrossRef]
- Norton, P.G., Dunn, E.V., & Soberman, L. (1994). Family practice in Ontario. How physician demographics affect practice patterns. Canadian Family Physician, 40, 249-256.
- Norton, P.G., Dunn, E.V., & Soberman, L. (1997). What factors affect quality of care? Using the Peer Assessment Program in Ontario family practices. Canadian Family Physician, 43, 1739-1744.
- Page, G.G., Bates, J., Dyer, S.M., Vincent, D.R., Bordage, G., Jacques, A., et al. (1995). Physician-assessment and physician-enhancement programs in Canada. Canadian Medical Association Journal, 153(12), 1723-1728.[Abstract]
- Ram, P., van der Vleuten, C., Rethans, J.J., Grol, R., & Aretz, K. (1999). Assessment of practicing family physicians: Comparison of observation in a multiple-station examination using standardized patients with observation of consultations in daily practice. Academic Medicine, 74(1), 62-69.[Web of Science][Medline]
[Order article via Infotrieve]
- Rethans, J.J., Martin, E., & Metsemakers, J. (1994). To what extent do clinical notes by general practitioners reflect actual medical performance? A study using simulated patients. British Journal of General Practice, 44(381), 153-156.[Web of Science][Medline]
[Order article via Infotrieve]
- Royal College of General Practitioners. (1985 ). What sort of doctor: Assessing quality of care in general practice. London: Royal College of General Practitioners.
- Salvatori, P., Baptiste, S., & Ward, M. (2000). Development of a tool to measure clinical competence in occupational therapy: a pilot study? Canadian Journal of Occupational Therapy, 67(1), 51-60.
- Smith, M.A., Atherly, A.J., Kane, R.L., & Pacala, J.T. (1997). Peer review of the quality of care. Reliability and sources of variability for outcome and process assessments. Journal of the American Medical Association, 278(19), 1573-1578.[CrossRef]
- Thomas, E.J., Lipsitz, S.R., Studdert, D.M., & Brennan, T.A. (2002). The reliability of medical record review for estimating adverse event rates. Annals of Internal Medicine, 136(11), 812-816.[Abstract/Free Full Text]
- Tugwell, P., & Dok, C. (1985). Medical record review. In V. R. Neufeld & G. R. Norman, (Eds.), Assessing clinical competence (pp.142-182). New York: Springer-Verlag New York, Inc.
- Wakefield, D.S., Helms, C.M., & Helms, L. (1995). The peer review process: The art of judgment. Journal of Healthcare Quality, 17(3), 11-16, 51.
- Wilson, A.J., & McDonald, P. (1994). Comparison of patient questionnaire, medical record, and audio tape in assessment of health promotion in general practice consultations. BMJ (Clinical Research Ed.), 309(6967), 1483-1485.[Medline]
[Order article via Infotrieve]
- Wu, L., & Ashton, C.M. (1997). Chart review. A need for reappraisal. Evaluation & the Health Professions, 20(2), 146-163.[CrossRef][Web of Science][Medline]
[Order article via Infotrieve]
Evaluation & the Health Professions, Vol. 30, No. 4,
376-392 (2007)
DOI: 10.1177/0163278707307924
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
D. Davis, G. Bordage, C. L. K. Moores, N. Bennett, S. S. Marinopoulos, P. E. Mazmanian, T. Dorman, and D. McCrory
The Science of Continuing Medical Education: Terms, Tools, and Gaps: Effectiveness of Continuing Medical Education: American College of Chest Physicians Evidence-Based Educational Guidelines
Chest,
March 1, 2009;
135(3_suppl):
8S - 16S.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
