Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation

Michael B. Cahapay (College of Education, Mindanao State University, General Santos City, Philippines)
International Journal of Assessment Tools in Education
7 1
Yıl: 2021 Cilt: 8 Sayı: 1 Sayfa Aralığı: 135 - 144 Metin Dili: İngilizce DOI: 10.21449/ijate.856143 İndeks Tarihi: 01-06-2021

Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation

Öz:
One of the widely known evaluation models adapted to education is theKirkpatrick model. However, this model has limitations when used by evaluatorsespecially in the complex environment of higher education. Addressing the scarcityof a collective effort on discussing these limitations, this review paper aims topresent a descriptive analysis of the limitations of the Kirkpatrick evaluation modelin the higher education field. Three themes of limitations were found out:propensity towards the use of the lower levels of the model; rigidity wich leavesout other essential aspects of the evaluand; and paucity of evidence on the causalchains among the levels. It is suggested that, when employing the Kirkpatrickmodel in higher education, evaluators should address these limitations byconsidering more appropriate methods, integrating contextual inputs in theevaluation framework, and establishing causal relationships among the levels.These suggestions to address the limitations of the model are discussed at the endof the study.
Anahtar Kelime:

Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Erişime Açık
  • Abdulghani, H., A Al Drees, A.M., Khamis, N., & Irshad, M. (2014). Research methodology workshops evaluation using the Kirkpatrick’s model: Translating theory into practice. Medical Teacher, 36(1), 24-29. https://doi.org/10.3109/0142159x.2014.886012
  • Abernathy D.J. (1999). Thinking outside the evaluation box. Training Development, 53(2), 18- 23. https://eric.ed.gov/?id=EJ578905
  • Alliger, G., & Janak, E.. Kirkpatrick’s Levels of Training Criteria: Thirty Years Later. Personnel Psychology, 42(2), 331-341. https://doi.org/10.1111/j.1744- 6570.1989.tb00661.x
  • Arthur, W., Jr., Bennet, W., Edens, P.S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88(2), 234-245. https://doi.org/10.1037/0021-9010.88.2.234
  • Arthur, W., Jr., Tubre, T. C., Paul, D. S., & Edens, P. S. (2010). Teaching effectiveness: The relation- ship between reaction and learning criteria. Educational Psychology, 23(3), 275- 285. https://doi.org/10.1080/0144341032000060110
  • Aryadoust, V. (2017). Adapting Levels 1 and 2 of Kirkpatrick’s model of training evaluation to examine the effectiveness of a tertiary-level writing course. Pedagogies: An International Journal, 12(2), 151-179. https://doi.org/10.1080/1554480X.2016.1242426
  • Bates, R. (2004). A critical analysis of evaluation practice: the Kirkpatrick Model and the principle of beneficence. Evaluation and Program Planning, 27, 341-347. https://doi.or g/10.1016/j.evalprogplan.2004.04.011
  • Baskin, C. (2001, December). Using Kirkpatrick’s four-level-evaluation model to explore the effectiveness of collaborative online group work. In Proceedings of the Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (pp. 37-44). Melbourne, Australia: Biomedical Multimedia Unit, The University of Melbourne.
  • Bowen, G. A. (2009) Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27-40. https://doi.org/10.3316/qrj0902027
  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
  • Chang, N. & Chen, L. (2014). Evaluating the learning effectiveness of an online information literacy class based on the Kirkpatrick framework. Libri, 64(3), 211-223. https://doi.org/ 10.1515/libri-2014-0016
  • Covington, J.A. (2012). Efficacy of webinar training for continuing professional education: applications for school personnel in k-12 settings [Doctoral dissertation, University of North Carolina] https://eric.ed.gov/?id=ED550661
  • Dewi. L.R., & Kartowagiran, B. (2018). An evaluation of internship program by using Kirkpatrick evaluation model. Research and Evaluation in Education, 4(2), 155-163. https://doi.org/10.21831/reid.v4i2.22495
  • Embi, Z.C., Neo, T.K., & Neo, M. (2017). Using Kirkpatrick’s evaluation model in a multimedia-based blended learning environment. Journal of Multimedia Information System, 4(3), 115-122, 2383-7632. http://dx.doi.org/10.9717/JMIS.2017.4.3.115
  • Farjad, S. (2012). The Evaluation effectiveness of training courses in university by Kirkpatrick model. Procedia – Social and Behavioral Sciences, 46, 2837-2841. https://doi.org/10.10 16/j.sbspro.2012.05.573
  • Frey, B. (2018). The SAGE encyclopedia of educational research, measurement, and evaluation. SAGE Publications, Inc. https://doi.org/10.4135/9781506326139
  • Guptill, A. (2016). Secondary sources in their natural habitats. Writing in College. https://milnepublishing.geneseo.edu/writing-in-college-from-competence-toexcellence/chapter/secondary-sources-in-their- natural-habitats/
  • Haupt, G., & Blignaut, S. (2007). Uncovering learning outcomes: explicating obscurity in learning of aesthetics in design and technology education. International Journal of Technology and Education, 18(4), 361-374. https://doi.org/10.1007/s10798-007-9029-1
  • Hilbert, J., Preskill, H., & Russ-Eft, D. (1997). Evaluating training’s effectiveness. In L. Bassi & D. Russ- Eft (Eds.), What works: Assessment, development, and measurement (pp. 109–150). American Society for Training and Development.
  • Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training and Development, 13, 3-9.
  • Kirkpatrick, D. L. (1976). Evaluation of training. In R. L. Craig (Ed.), Training and development handbook: A guide to human resource development (2nd ed., pp. 301–319). New York: McGraw-Hill.
  • Kirkpatrick D. L, & Kirkpatrick J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler Publication.
  • Lambert, N. (2011). Ban happy sheets! - Understanding and using evaluation. Nurse Education Today, 32(1), 1-4. https://doi.org/10.1016/j.nedt.2011.05.020
  • Liao, S. C., & Hsu, S. Y. (2019). Evaluating a continuing medical education program: New World Kirkpatrick Model Approach. International Journal of Management, Economics and Social Sciences, 8(4), 266-279. http://dx.doi.org/10.32327/IJMESS/8.4.2019.17
  • Lillo-Crespo, M., Sierras-Davo, M. C., McRae, R., & Rooney. K. (2017). Developing a framework for evaluating the impact of Healthcare Improvement Science Education across Europe: A qualitative study. Journal of Educational Evaluation in Health Profession, 14, 28. https://doi.org/10.3352/jeehp.2017.14.28
  • Masood, R., & Usmani, M. A. W. (2015). A study for program evaluation through Kirkpatrick’s model. Khyber Medical University Journal, 2(7), 76-80. https://www.kmuj.kmu.edu.pk /article/view/15377
  • Miller, B.J. (2018). Utilizing the Kirkpatrick model to evaluate a collegiate high-impact leadership development program [Master’s thesis, Texas A&M University, College Station, Texas]. https://oaktrust.library.tamu.edu
  • Moreira I.C., Ramos, I., Ventura, S.R., Rodrigues, P.P. (2018). Learner’s perception, knowledge and behaviour assessment within a breast imaging E-Learning course for radiographers. European Journal of Radiology, 111, 47-55. https://doi.org/10.1016/j.ejr ad.2018.12.006
  • Morrison J. (2003). ABC of learning and teaching in medicine: Evaluation. British Medical Journal, 326(7385), 385-387. https://doi.org/10.1136/bmj.326.7385.385
  • Nickols, F. W. (2000). Evaluating training: There is no "cookbook" approach. http://home.att.net/ ~nickols/evaluating_ training.htm
  • Paull, M., Whitsed, C., & Girardi, A. (2016). Applying the Kirkpatrick model: Evaluating an Interaction for Learning Framework curriculum intervention. Issues in Educational Research, 26(3), 490-502. https://www.iier.org.au/iier26/paull.pdf
  • Quintas, C., Fernandes Silva, I., & Tiexiera, A. (2017). Assessing an e-Learning and b-Learning Model - A study of perceived satisfaction. International Journal of Information and Education Technology, 7(4), 265-268. https://doi.org/10.18178/ijiet.2017.7.4.878
  • Reio, T. G., Rocco, T. S., Smith, D. H., & Chang, E. (2017). A Critique of Kirkpatrick’s Evaluation Model. New Horizons in Adult Education & Human Resource Development 29(2), 35-53. https://doi.org/10.1002/nha3.20178
  • Rouse D. N. (2011). Employing Kirkpatrick’s evaluation framework to determine the effectiveness of health information management courses and programs. Perspectives of Health Information Management, 8, 1-5. https://www.ncbi.nlm.nih.gov/pubmed/21464 860
  • Sahin, V. (2006). Evaluation of the in-service teacher training program “The Certificate for Teachers of English” at the Middle East Technical University School of Foreign Languages. [Doctoral dissertation, Middle East Technical University] https://etd.lib.metu.edu.tr/upload/12607752/index.pdf
  • Shelton, S., & Alliger, G. M. (1993). Who’s afraid of level 4 evaluation? A practical approach. Training and Development Journal, 47, 43-46. https://eric.ed.gov/?id=EJ463549
  • Sileyew, K. J. (2019). Research design and methodology. In E. Abu- Taieh, A. El Mouatasim, & I.H. Al Hadid (Eds.), Cyberspace. IntechOpen. https://doi.org/10.5772/intechopen.78 887
  • Steele, L. M., Mulhearn, T. J., Medeiros, K. E., Watts, L. L., Connelly, S., & Mumford, M. D. (2016). How do we know what works? A review and critique of current practices in ethics training evaluation. Accountability in Research, 23(6), 319-350. http://dx.doi.org/10.108 0/08989621.2016.1186547
  • Tamkin, P., Yarnall, J., & Kerrin, M. (2002). Kirkpatrick and beyond: A review of models of training evaluation (Report No. 392). Institute for Employment Studies
  • Topno, H. (2012). Evaluation of training and development: An analysis of various models. IOSR Journal of Business and Management, 5(2), 16-22. https://doi.org/10.9790/487x0521622
  • Yardley, S., & Dornan, T. (2012). Kirkpatrick’s levels and education ‘evidence’. Medical Education, 46(1), 97-106. https://doi.org/10.1111/j.1365-2923.2011.04076.x
APA Cahapay M (2021). Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. , 135 - 144. 10.21449/ijate.856143
Chicago Cahapay Michael Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. (2021): 135 - 144. 10.21449/ijate.856143
MLA Cahapay Michael Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. , 2021, ss.135 - 144. 10.21449/ijate.856143
AMA Cahapay M Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. . 2021; 135 - 144. 10.21449/ijate.856143
Vancouver Cahapay M Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. . 2021; 135 - 144. 10.21449/ijate.856143
IEEE Cahapay M "Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation." , ss.135 - 144, 2021. 10.21449/ijate.856143
ISNAD Cahapay, Michael. "Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation". (2021), 135-144. https://doi.org/10.21449/ijate.856143
APA Cahapay M (2021). Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. International Journal of Assessment Tools in Education, 8(1), 135 - 144. 10.21449/ijate.856143
Chicago Cahapay Michael Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. International Journal of Assessment Tools in Education 8, no.1 (2021): 135 - 144. 10.21449/ijate.856143
MLA Cahapay Michael Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. International Journal of Assessment Tools in Education, vol.8, no.1, 2021, ss.135 - 144. 10.21449/ijate.856143
AMA Cahapay M Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. International Journal of Assessment Tools in Education. 2021; 8(1): 135 - 144. 10.21449/ijate.856143
Vancouver Cahapay M Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation. International Journal of Assessment Tools in Education. 2021; 8(1): 135 - 144. 10.21449/ijate.856143
IEEE Cahapay M "Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation." International Journal of Assessment Tools in Education, 8, ss.135 - 144, 2021. 10.21449/ijate.856143
ISNAD Cahapay, Michael. "Kirkpatrick Model: Its Limitations as Used in Higher Education Evaluation". International Journal of Assessment Tools in Education 8/1 (2021), 135-144. https://doi.org/10.21449/ijate.856143
Sayfa Oluşturulma Tarihi
25-04-2024 5:43

İLETİŞİM

TÜBİTAK ULAKBİM TR Dizin

Yüzüncüyıl, İşçi Blokları Mahallesi

Muhsin Yazıcıoğlu Caddesi No:51/C

06530 Çankaya / ANKARA +90 (312) 298 92 00

trdizin@tubitak.gov.tr

TÜBİTAK ULAKBİM Ulusal Akademik Ağ ve Bilgi Merkezi Cahit Arf Bilgi Merkezi © Tüm Hakları Saklıdır. Gizlilik Politikası Kullanım Şartları