The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test
Yıl: 2020 Cilt: 13 Sayı: 3 Sayfa Aralığı: 530 - 547 Metin Dili: İngilizce DOI: 10.30831/akukeg.621581 İndeks Tarihi: 19-01-2021
The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test
Öz: In this study logistic regression and Lord’s Chi Square methods were used to research the items that have DIF. The study utilized Peabody Picture Vocabulary Test (PPVT). The original form of the PPVT includes four options. Three different forms (A, B and C) were formed by removing one of the distractors respectively. The original form of PPVT was implemented in a group of 970 preschool children who were aged between 3 to 6. 757 of them took one of the forms. In each implementation, the order to the implementation of the original form and the form (A, B or C) was changed. The applications were conducted 15 days apart. In the first application, the original form was applied, while one of the devised forms (A, B or C) was used in the following application. In this way, the effect of order of application on responses was investigated. The gender variable constituted the reference and focus group of the study. The Logistic Regression and Lord's Chi-square methods did not give compatible results in DIF analysis. DIF was found in 15 items in the original form according to the logistic regression method and in nine items according to the Lord's Chi-square method. The three-option and four-option applications of the test revealed that DIF was determined in five items in different forms. It was observed that there was no compliance in different applications and analyses in other items with DIF.
Anahtar Kelime: Peabody Resim Kelime Testinde Çeldiricilerin Değişen Madde Fonksiyonuna Etkisi
Öz: Bu araştırmada değişen madde fonksiyonunun belirlenmesinde lojistik regresyon ve Lord’un Ki-kare yöntemleri karşılaştırılmıştır. Araştırmada Peabody Resim Kelime Testi (PRKT) kullanılmıştır. PRKT dört seçenekli maddelerden oluşmaktadır. Çeldiricilerin uygulamadaki etkisini görmek amacıyla farklı formlarda farklı bir çeldirici maddeden çıkarılarak üç seçenekli formlar oluşturulmuştur. PRKT 3-6 yaş arasında 970 çocuğa uygulanmış 757 uygulamadan elde edilen yanıtlar çözümlenmiştir. Uygulamalar 15 gün arayla gerçekleştirildi. Bir uygulamada önce original form uygulandı, diğer uygulamada oluşturulan formlardan biri (A, B veya C) uygulandı. Bu yolla yanıtlarda uygulama sırasının etkisi kontrol edildi. Cinsiyet değişkeni araştırmanın referans ve odak grubunu oluşturmuştur. DIF analizinde Lojistik Regreyon ve Lord’un Ki-kare yöntemi uyumlu sonuçlar vermedi. Araştırma bulgularına göre lojistik regresyon yöntemine göre orijinal formda 15, Lord’un Ki-kare yöntemine göre 9 maddede DIF belirlendi. Testin üç seçenekli ve dört seçenekli uygulamalarından elde edilen sonuçlarda farklı formlarda beş maddede uyumlu bir biçimde DIF belirlenmiştir. DIF belirlenen diğer maddelerde ise farklı uygulama ve analizlerde uyum olmadığı gözlenmiştir.
Anahtar Kelime: Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Erişime Açık
- Adebule, S. O. (2013). A study of differential item functioning in Ekiti State Unified Mathematics Examination for senior secondary schools. Journal of Education and Practice, 4(17), 43-46.
- Allalouf, A. (2003). Revising translated differential item functioning items as a tool for improving cross-lingual assessment. Applied Measurement in Education, 16(1), 55-73.
- Alordiah, C. O., & Agbajor, H. T. (2014). Bias in test items and implication for national development. Journal of Education and Practice, 5(9), 10-13.
- Ascalon, M. E., Meyers, L. S., Davis, B. W., & Smits, N. (2007). Distractor similarity and item-stem structure: Effects on item difficulty. Applied Measurement in Education, 20(2), 153-170.
- Banks, K. (2009). Using DDF in a post hoc analysis to understand sources of DIF. Educational Assessment, 14(2), 103-118.
- Başusta, N. B. (2013). PISA 2006 fen başarı testinin madde yanlılığının kültür ve dil açısından incelenmesi (Yayımlanmamış yüksek lisans tezi). Hacettepe Üniversitesi, Ankara, Türkiye.
- Camilli, G., Shepard, L. A., & Shepard, L. (1994). Methods for identifying biased test items (Vol. 4). Sage.
- Chalmers, R. P., Counsell, A., & Flora, D. B. (2016). It might not make a big DIF: Improved differential test functioning statistics that account for sampling variability. Educational and Psychological Measurement, 76(1), 114-140.
- Doğan, N., & Öğretmen, T. (2010). Değişen madde fonksiyonunu belirlemede Mantel‐Haenszel, Ki‐kare ve Lojistik Regresyon tekniklerinin karşılaştırılması [The comparision of Mantel–Haenszel, Chi-Square and Logistic Regression Techniques for identifying Differential item functioning]. Eğitim ve Bilim, 33(148), 100-112.
- Dunn, L. M., & Dunn, L. M. (1959). Manual for the Peabody Picture Vocabulary Test-revised. Circle Pines, MN: American Guidance Service.
- Erdem, B. (2015). Ortaöğretime geçişte kullanılan ortak sınavların değişen madde fonksiyonu açısından kitapçık türlerine göre farklı yöntemlerle incelenmesi (Yayımlanmamış yüksek lisans tezi). Hacettepe Üniversitesi, Ankara, Türkiye.
- Forsyth, R. A., & Spratt, K. F. (1980). Measuring problem solving ability in mathematics with multiple-choice items: The effect of item format on selected item and test characteristics. Journal of Educational Measurement, 17(1), 31-43.
- French, B. F., & Maller, S. J. (2007). Iterative purification and effect size use with Logistic Regression for differential item functioning detection. Educational and Psychological Measurement, 67(3), 373-393.
- Gierl, M., Khaliq, S. N., & Boughton, K. (1999). Gender differential item functioning in mathematics and science: Prevalence and policy implications. Paper presented at the annual meeeting of the Canadian Society for the Study of Education, Sherbrooke, Quebec.
- Gök, B., Kellecioğlu, H., & Doğan, N. (2010). Değişen madde fonksiyonunu belirlemede Mantel–Haenszel ve Lojistik Regresyon tekniklerinin karşılaştırılması [The comparision of Maentel-Haenszel and Logistic Regression techniques in determinin differential item functioning]. Eğitim ve Bilim, 35(156), 3-16.
- Green, B. F., Crone, C. R., & Folk, V. G. (1989). A method for studying differential distractor functioning. Journal of Educational Measurement, 26(2), 147-160.
- Hambleton, R. K., & Traub, R. E. (1974). The effects of item order on test performance and stress. The Journal of Experimental Education, 43(1), 40-46.
- Harasym, P. H., Leong, E. J., Violato, C., Brant, R., & Lorscheider, F. L. (1998). Cuing effect of" all of the above" on the reliability and validity of multiple-choice test items. Evaluation & the health professions, 21(1), 120-133.
- Hidalgo, M. D., & LÓPez-Pina, J. A. (2004). Differential Item Functioning detection and Effect Size: A Comparision between Logistic Regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64(6), 903-915.
- Holland, P. W., & Wainer, H. eds. (1993). Diferential item functioning. Hillsdale, N.J.: Erlbaum.
- Hughes, H. H., & Trimble, W. E. (1965). The use of complex alternatives in multiple choice items. Educational and Psychological Measurement, 25(1), 117-126.
- Ikeda, E. (1995). Raters differential item functioning and Mantel-Haenszel procedures Applied to an item analysis of write-answer type test. Japanese Journal Of Educational Psychology, 43(3), 343-350.
- Jensen, A. R. (1980). Bias in mental testing. New York: Free Press.
- Jodoin, M. G., & Gierl, M.J. (2001). Evaluating Type I Error and Power Rates using an Effect Size Measure with the Logistic Regression procedure for DIF detection. Applied Measurement in Education, 14(4), 329-349.
- Kalaycıoğlu, D. B., & Kelecioğlu, H. (2011). Öğrenci Seçme Sınavı’nın madde yanlılığı açısından incelenmesi [Item bias analyses of the University Entrance Exam]. Eğitim ve Bilim, 36(161), 3-13.
- Karakaya, İ., & Kutlu, Ö. (2012). Seviye belirleme sınavındaki Türkçe alt testlerinin madde yanlılığının incelenmesi [An investigation of item bias n Turkish sub Tests in Level Determination Exam]. Eğitim ve Bilim, 37(165), 348-362.
- Katz, J., Demir, N., Önen, F., Uzlukaya, A., & Uludağ, A. (1972). Türkçe konuşan çocuklar için Peabody resim kelime testi resim dizisi (Peabody Picture-Vocabulary Test). Ankara Rehberlik ve Araştırma Merkezi. Ankara.
- Köse, İ. A. (2015). PISA 2009 öğrenci anketi alt ölçeklerinde (Q32-Q33) bulunan maddelerin değişen madde fonksiyonu açısından incelenmesi [Investigation of items in PISA 2009 Student Questionnaire Subscales (Q32-Q33) in terms of differential item functioning]. Kastamonu Eğitim Dergisi, 23(1), 227-240.
- Kurnaz, F. B., & Kelecioğlu, H. (2008). Investigation of Peabody Picture Vocabulary Test from the point of item bias. World Applied Sciences Journal, 3(2), 231-239.
- Lord, F. M. (1980). Applications of item response theory to practical testing problems. Routledge.
- Love, T. E. (1997). Distractor selection ratios. Psychometrika, 62(1), 51-62.
- Maij-de Meij, A. M., Kelderman, H., & Van Der Flier, H. (2010). Improvement in detection of differential item functioning using a mixture item response theory model. Multivariate Behavioral Research, 45(6), 975-999.
- Magis, D., Beland, S., Tuerlinckx, F., & De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42, 847-862.
- Maller, S. J. (2001). Differential item functioning in the WISC-III: Item parameters for boys and girls in the national standardization sample. Educational and Psychological Measurement, 61(5), 793-817.
- Meyer, J. P., & Wise, S. (2006). Including item response time in a distractor analysis via Multivariate Kernel Smoothing. National Council on Measurement in education (NCME).
- Narayanon, P., & Swaminathan, H. (1996). Identification of items that show nonuniform DIF. Applied Psychological Measurement, 20(3), 257-274.
- Osterlind, S. J. (1983). Test item bias (Vol. 30). Sage.
- Öner, N. (1997). Türkiye’de kullanılan psikolojik testler: Bir başvuru kaynağı. İstanbul: Boğaziçi Üniversitesi Yayınları.
- Özgüven, İ. E. (1994). Psikolojik testler. Psikoloji Danışma Rehberlik ve Eğitim Merkezi (PDREM) Yayınları.
- Penfield, R. D. (2008). An Odds Ratio approach for assessing differential distractor functioning effects under the Nominal Response Model. Journal of Educational Measurement, 45(3), 247-269.
- Penfield, R. D. (2010). Modeling DIF effects using distractor-level invariance effects: Implications for understanding the causes of DIF. Applied Psychological Measurement, 34(3), 151-165.
- Petersen, M. A., Groenvold, M., Bjorner, J. B., Aaronson, N., ... & Sullivan, M. (2003). Use of differential item functioning analysis to assess the equivalence of translations of a questionnaire. Quality of Life Research, 12(4), 373-385.
- Ryan, K. E., & Chiu, S. (2001). An examination of item context effects, DIF, and gender DIF. Applied Measurement in Education, 14(1), 73-90.
- Roznowski, M., & Reith, J. (1999). Examining the measurement quality of tests containing differentially functioning items: Do biased items result in poor measurement?. Educational and Psychological Measurement, 59(2), 248-269.
- Sehmitt, A. P., & Dorans, N. J. (1990). Differential item functioning for minority examinees on the SAT. Journal of Educational Measurement, 27(1), 67-81.
- Stump, T. E., Monahan, P., & Mchorney, C. A. (2005). Differential item functioning in the short portable mental status questionnaire. Research on Aging, 27(3), 355-384.
- Suh, Y., & Talley, A. E. (2015). An empirical comparison of DDF detection methods for understanding the causes of DIF in multiple-choice items. Applied Measurement in Education, 28(1), 48-67.
- Temiz, G. (2002). Okulöncesi eğitimin çocuğun dil gelişimine olan etkisi (Yayımlanmamış doktora tezi). Selçuk Üniversitesi, Konya, Türkiye.
- Terzi, R., & Suh, Y. (2015). An odds ratio approach for detecting DDF under the nested logit modeling framework. Journal of Educational Measurement, 52(4), 376-398.
- Terzi, R., & Yakar, L. (2018). Differential item distractor functioning analyses on Turkish High School Entrance exam. Journal of Measurement and Evaluation in Education and Pshchology, 9(2), 136-149.
- Thissen, D., Steinberg, L., & Fitzpatrick, A. R. (1989). Multiple‐choice Models: The distractors are also part of the item. Journal of Educational Measurement, 26(2), 161-176.
- Tian, F., Pang, X. L., & Boss, M. W. (1994). The effects of sample size and criterion variable on the identification of DIF by the Mantel-Haenszel and Logistic Regression procedures. In Annual Meeting of the American Educational Research Association, New Orleans, LA.
- Tunç, E. B., & Kutlu, Ö. (2018). İki ve çok kategorili puanlanan maddelerde değişen madde fonksiyonlarının karşılaştırılması [Comparision of differential item functioning for two-category scored and multi-category scored items]. Başkent University Journal of Education, 5(1), 40-50.
- Van de Vijver, F. J. R. (2018). Towards an integrated framework of bias in noncognitive assessment in international large-scale studies: Challenges and prospects. Educational Measurements: Issues and Practice,37(4), 49-56.
- Washington, J. A., & Craig, H. K. (1999). Performances of at-risk, African American preschoolers on the Peabody Picture Vocabulary Test-III. Language, Speech, and Hearing Services in Schools, 30(1), 75-82.
- Washington, J. A. (1996). Issues in assessing the language abilities of African American children. In A. G. Kamhi, K. E. Pollock, & J. Harris (Eds.), Communication development and disorders in African American children: Research, assessment, and intervention (pp. 35-54). Baltimore, MD: Paul H. Brookes.
- Wiberg, M. (2007). Measuring and detecting differential item functioning in criterion-referenced licensing test: A theoretic comparison of methods. Retrieved from: http://umu.diva-portal.org/smash/record.jsf?pid=diva2%3A146028&dswid=1318.
- Yıldırım, A. (2017). PISA 2009 okuma becerileri alanındaki maddelerin tek değişkenli ve çok değişkenli eşleştirme yöntemi ile değişen madde fonksiyonlarının incelenmesi (Yayımlanmamış doktora tezi). Ankara Üniversitesi, Eğitim Bilimleri Enstitüsü, Ankara.
- Yıldırım, H., & Büyüköztürk, Ş. (2018). Using the Delphi Technique and focus-group interviews to determine item bias on the Mathematics Section of the Level Determination Exam for 2012. Educational Sciences: Theory & Practice, 18(2), 447-470.
- Yurdugül, H. (2003). Ortaöğretim Kurumları Seçme ve Yerleştirme Sınavının madde yanlılığı açısından incelenmesi (Yayımlanmamış doktora tezi). Hacettepe Üniversitesi, Ankara, Türkiye.
- Zwick, R. (1991). Effects of item order and context on estimation of NAEP reading proficiency. Educational Measurement: Issues and Practice, 10(3), 10-16.
- Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa: National Defense Headquarters.
| APA | KURNAZ F, YILDIZ H (2020). The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. , 530 - 547. 10.30831/akukeg.621581 |
| Chicago | KURNAZ Fatma Betül,YILDIZ Hüseyin The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. (2020): 530 - 547. 10.30831/akukeg.621581 |
| MLA | KURNAZ Fatma Betül,YILDIZ Hüseyin The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. , 2020, ss.530 - 547. 10.30831/akukeg.621581 |
| AMA | KURNAZ F,YILDIZ H The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. . 2020; 530 - 547. 10.30831/akukeg.621581 |
| Vancouver | KURNAZ F,YILDIZ H The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. . 2020; 530 - 547. 10.30831/akukeg.621581 |
| IEEE | KURNAZ F,YILDIZ H "The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test." , ss.530 - 547, 2020. 10.30831/akukeg.621581 |
| ISNAD | KURNAZ, Fatma Betül - YILDIZ, Hüseyin. "The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test". (2020), 530-547. https://doi.org/10.30831/akukeg.621581 |
| APA | KURNAZ F, YILDIZ H (2020). The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. Kuramsal Eğitimbilim Dergisi, 13(3), 530 - 547. 10.30831/akukeg.621581 |
| Chicago | KURNAZ Fatma Betül,YILDIZ Hüseyin The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. Kuramsal Eğitimbilim Dergisi 13, no.3 (2020): 530 - 547. 10.30831/akukeg.621581 |
| MLA | KURNAZ Fatma Betül,YILDIZ Hüseyin The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. Kuramsal Eğitimbilim Dergisi, vol.13, no.3, 2020, ss.530 - 547. 10.30831/akukeg.621581 |
| AMA | KURNAZ F,YILDIZ H The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. Kuramsal Eğitimbilim Dergisi. 2020; 13(3): 530 - 547. 10.30831/akukeg.621581 |
| Vancouver | KURNAZ F,YILDIZ H The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test. Kuramsal Eğitimbilim Dergisi. 2020; 13(3): 530 - 547. 10.30831/akukeg.621581 |
| IEEE | KURNAZ F,YILDIZ H "The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test." Kuramsal Eğitimbilim Dergisi, 13, ss.530 - 547, 2020. 10.30831/akukeg.621581 |
| ISNAD | KURNAZ, Fatma Betül - YILDIZ, Hüseyin. "The Effects of Distractors to Differential Item Functioning in Peabody Picture Vocabulary Test". Kuramsal Eğitimbilim Dergisi 13/3 (2020), 530-547. https://doi.org/10.30831/akukeg.621581 |
