Is Peer Assessment Reliable in Objectively Structured Clinical Examination?

Objective: The study was designed to evaluate the reliability of the peer assessment in the objectively structured clinical examination (OSCE) for the summative assessment of 4th grade students at the end of general surgery clerkship. Method: The study was planned prospectively with the permission of the Dean of Medicine Faculty and approval of the ethics committee. The 6th grade students who were in the surgery rotation participated in the study as peer assessors (PA). Both peers and department of general surgery assessed the students. Pass/fail point was accepted as 60. The scores of OSCE and performance evaluation given by peers and faculty were compared statistically.Results: Twenty-three students completed general surgery clerkship. Ten students (43.5%) were female. According to performance scores given by the faculty, 15 (65.2%) of the 23 students were successful, while all students were considered successful (having a grade of 60 or more) based on the scores of peer evaluation. There was a significant difference between the faculty members and PA with regard to the performance evaluation (p=0.008). The faculty members found five students (27.8%) successful in the OSCE (having a grade of 60 or more). However, ten students (43.5%) received a score of at least six from peer evaluation. Although there was a difference, it was not significant (p=0.063). Gender did not affect scoring in performance evaluation and OSCE application.Conclusion: Although there was a difference between faculty members and peer evaluators in the performance evaluation, no difference was observed in OSCE. In conclusion, OSCE assessment by peer evaluators is reliable.

Anahtar Kelime:

Objektif Yapılandırılmış Klinik Sınavda Akran Değerlendirmesi Güvenilir midir?

Öz:

Amaç:Genelcerrahikliniğinde4.sınıföğrencilerine“objektifyapılandırılmış klinik sınav“ (OYKS) uygulanmasında akran değerlendirmesiningüvenilirolupolmadığınıdeğerlendirmekveöğrencinin servis içindeki performansının akran değerlendirmesi üzerine etkilerini araştırmak amacıyla bu çalışmayı tasarladık. Yöntem: Çalışma, Tıp Fakültesi Dekanlığından ve etik kuruldan izin alınarak prospektif olarak planlandı. OYKS uygulamasına katılan 4. sınıf stajyer öğrenciler çalışma grubu olarak belirlendi. Altıncı sınıf öğrencileri ise akran değerlendirici olarak çalışmada yer aldı. Bu çalışma için akran değerlendiricilerin ve öğretim üyelerinin performans puanı vermeleri ve OYKS uygulamasına katılarak değerlendirmeleri istendi. Sonuçlar istatiksel yöntemlerle değerlendirildi.Bulgular: OYKS uygulamasına katılan 23 öğrenciden 10’u (%43,5) kız öğrenciydi. Genel cerrahi stajından sorumlu öğretim üyeleri tarafından verilen ortalama performans puanlarına göre, 23 öğrenciden 15’i (%65,2) başarılı oldu. Akran değerlendiricilerinin performans puanı ortalamalarına göreisetümöğrencilerinbaşarılıolduğugörüldü.Öğretimüyeleriileakrandeğerlendiricilerinverdikleriperformanspuanlarıarasındaistatistiksel olarak anlamlı fark bulunmuştur (p=0,008). Öğretim üyeleri ve akran değerlendiriciler tarafından stajyerlere verilen OYKS puanları karşılaştırıldı. Öğretim üyelerinin verdikleri puana göre, beş (%27,8) öğrenci altı veya daha yüksek puan alarak başarılı oldu. Akran değerlendiricilerin puanlamasına göre ise on (%43,5) öğrenci altı veya daha yüksek puan alarak başarılı oldu. Buna göre akran değerlendiricilerin OYKS’de daha fazla öğrenciyi başarılı bulduğu görüldü. Ancak iki grubun verdiği puanlarda istatistiksel olarak anlamlı fark yoktu (p=0,063). Cinsiyetin performans değerlendirmesinde ve OYKS uygulamasında puanlamayı etkilemediği tespit edildi. Sonuç: Çalışmamızda performans değerlendirmesinde öğretim üyeleri ile akran değerlendiriciler arasında fark olduğunu, OYKS uygulaması değerlendirmesinde ise farklılık olmadığını bulduk. Sonuç olarak akran değerlendirmesinin güvenilir olduğunu düşünüyoruz

Anahtar Kelime:

Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Erişime Açık

1. Rosof AB, Felch WC. Continuing medical education: a primer. Westport: Greenwood Publishing Group, 1992.
2. Tabish SA. Assessment methods in medical education. Int J Health Sci 2010;2(2):3-7.
3. Shumway JM, Harden RM, Association for Medical Education in Europe. AMEE Guide No. 25: The assessment of learning outcomes for the competent and reflective physician. Med Teach 2003;25(6):569-584.
4. Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. Br Med J 1975;1(5955):447-451.
5. Khan KZ, Ramachandran S, Gaunt K, Pushkar P. The objective structured clinical examination (OSCE): AMEE guide no. 81. Part I: an historical and theoretical perspective. Med Teach 2013;35(9):e1437-e1446.
6. Lindblom-ylänne S, Pihlajamäki H, Kotkas T. Self-peer-and teacher-assessment of student essays. Active Learn High Educ 2006;7(1):51-62.
7. Speyer R, Pilz W, Van Der Kruis J, Brunings JW. Reliability and validity of student peer assessment in medical education: a systematic review. Med Teach 2011;33(11):e572-e585.
8. Somervell H. Issues in assessment, enterprise and higher education: The case for self‐peer and collaborative assessment. Assess Eval High Educ 1993;18(3):221-233.
9. Cushing A, Abbott S, Lothian D, Hall A, Westwood OM. Peer feedback as an aid to learning-What do we want? Feedback. When do we want it? Now! Med Teach 2011;33(2):e105-e112.
10. Arnold L, Shue CK, Kalishman S, Prislin M, Pohl C, Pohl H, et al. Can there be a single system for peer assessment of professionalism among medical students? A multi-institutional study. Acad Med 2007;82(6):578-586.
11. Nofziger AC, Naumburg EH, Davis BJ, Mooney CJ, Epstein RM. Impact of peer assessment on the professional development of medical students: a qualitative study. Acad Med 2010;85(1):140-147.
12. English R, Brookes ST, Avery K, Blazeby JM, Ben-Shlomo Y. The effectiveness and reliability of peer-marking in first-year medical students. Med Educ 2006;40(10):965-972.
13. Norcini JJ. Peer assessment of competence. Med Educ 2003;37(6):539-543.
14. Mavis BE, Ogle KS, Lovell KL, Madden LM. Medical students as standardized patients to assess interviewing skills for pain evaluation. Med Educ 2002;36(2):135-140.
15. Burgess A, Clark T, Chapman R, Mellis C. Senior medical students as peer examiners in an OSCE. Med Teach 2013;35(1):58-62.
16. Chenot JF, Simmenroth-Nayda A, Koch A, Fischer T, Scherer M, Emmert B, et al. Can student tutors act as examiners in an objective structured clinical examination? Med Educ 2007;41(11):1032-1038.
17. Basehore PM, Pomerantz SC, Gentile M. Reliability and benefits of medical student peers in rating complex clinical skills. Med Teach 2014;36(5):409-414.
18. Schleicher I, Leitner K, Juenger J, Moeltner A, Ruesseler M, Bender B, et al. Examiner effect on the objective structured clinical exam-a study at five medical schools. BMC Med Educ 2017;17(1):71.
19. Casey M, Wilkinson D, Fitzgerald J, Eley D, Connor J. Clinical communication skills learning outcomes among first year medical students are consistent irrespective of participation in an interview for admission to medical school. Med Teach 2014;36(7):640-642.
20. Denney ML, Freeman A, Wakeford R. MRCGP CSA: are the examiners biased, favouring their own by sex, ethnicity, and degree source? Br J Gen Pract 2013;63(616):e718-e725.
21. McManus IC, Elder AT, Dacre J. Investigating possible ethnicity and sex bias in clinical examiners: an analysis of data from the MRCP (UK) PACES and nPACES examinations. BMC Med Educ 2013;13(1):103.

APA	DAGMURA H, Daldal E, BAKIR H, dasiran f, ozsoy z, demir o, Okan İ, bulbuloglu e (2021). Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. , 7 - 13. 10.4274/BMB.galenos.2020.09.054
Chicago	DAGMURA HASAN,Daldal Emin,BAKIR Hüseyin,dasiran fatih,ozsoy zeki,demir osman,Okan İsmail,bulbuloglu ertan Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. (2021): 7 - 13. 10.4274/BMB.galenos.2020.09.054
MLA	DAGMURA HASAN,Daldal Emin,BAKIR Hüseyin,dasiran fatih,ozsoy zeki,demir osman,Okan İsmail,bulbuloglu ertan Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. , 2021, ss.7 - 13. 10.4274/BMB.galenos.2020.09.054
AMA	DAGMURA H,Daldal E,BAKIR H,dasiran f,ozsoy z,demir o,Okan İ,bulbuloglu e Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. . 2021; 7 - 13. 10.4274/BMB.galenos.2020.09.054
Vancouver	DAGMURA H,Daldal E,BAKIR H,dasiran f,ozsoy z,demir o,Okan İ,bulbuloglu e Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. . 2021; 7 - 13. 10.4274/BMB.galenos.2020.09.054
IEEE	DAGMURA H,Daldal E,BAKIR H,dasiran f,ozsoy z,demir o,Okan İ,bulbuloglu e "Is Peer Assessment Reliable in Objectively Structured Clinical Examination?." , ss.7 - 13, 2021. 10.4274/BMB.galenos.2020.09.054
ISNAD	DAGMURA, HASAN vd. "Is Peer Assessment Reliable in Objectively Structured Clinical Examination?". (2021), 7-13. https://doi.org/10.4274/BMB.galenos.2020.09.054

APA	DAGMURA H, Daldal E, BAKIR H, dasiran f, ozsoy z, demir o, Okan İ, bulbuloglu e (2021). Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. Bağcılar Tıp Bülteni, 6(1), 7 - 13. 10.4274/BMB.galenos.2020.09.054
Chicago	DAGMURA HASAN,Daldal Emin,BAKIR Hüseyin,dasiran fatih,ozsoy zeki,demir osman,Okan İsmail,bulbuloglu ertan Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. Bağcılar Tıp Bülteni 6, no.1 (2021): 7 - 13. 10.4274/BMB.galenos.2020.09.054
MLA	DAGMURA HASAN,Daldal Emin,BAKIR Hüseyin,dasiran fatih,ozsoy zeki,demir osman,Okan İsmail,bulbuloglu ertan Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. Bağcılar Tıp Bülteni, vol.6, no.1, 2021, ss.7 - 13. 10.4274/BMB.galenos.2020.09.054
AMA	DAGMURA H,Daldal E,BAKIR H,dasiran f,ozsoy z,demir o,Okan İ,bulbuloglu e Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. Bağcılar Tıp Bülteni. 2021; 6(1): 7 - 13. 10.4274/BMB.galenos.2020.09.054
Vancouver	DAGMURA H,Daldal E,BAKIR H,dasiran f,ozsoy z,demir o,Okan İ,bulbuloglu e Is Peer Assessment Reliable in Objectively Structured Clinical Examination?. Bağcılar Tıp Bülteni. 2021; 6(1): 7 - 13. 10.4274/BMB.galenos.2020.09.054
IEEE	DAGMURA H,Daldal E,BAKIR H,dasiran f,ozsoy z,demir o,Okan İ,bulbuloglu e "Is Peer Assessment Reliable in Objectively Structured Clinical Examination?." Bağcılar Tıp Bülteni, 6, ss.7 - 13, 2021. 10.4274/BMB.galenos.2020.09.054
ISNAD	DAGMURA, HASAN vd. "Is Peer Assessment Reliable in Objectively Structured Clinical Examination?". Bağcılar Tıp Bülteni 6/1 (2021), 7-13. https://doi.org/10.4274/BMB.galenos.2020.09.054