The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data
Yıl: 2019 Cilt: 0 Sayı: 46 Sayfa Aralığı: 290 - 306 Metin Dili: İngilizce DOI: 10.9779/pauefd.546797 İndeks Tarihi: 29-12-2020
The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data
Öz: The objective of this study is to investigate the relation between the number of items and attributes and to analyze the manner in which the different rates of missing data affect the model estimations based on the simulation data. A Q-matrix contains 24 items, and data are generated using four attributes. A dataset of n = 3000 is generated by associating the first, middle, and final eight items in the Q-matrix with one, two, and three attributes, respectively, and 5%, 10%, and 15% of the data have been randomly deleted from the first, middle, and final eight-item blocks in the Q-matrix, respectively. Subsequently, imputation was performed using the multiple imputation (MI) method with these datasets, 100 replication was performed for each condition. The values obtained from these datasets were compared with the values obtained from the full dataset. Thus, it can be observed that an increase in the amount of missing data negatively affects the consistency of the DINA parameters and the latent class estimations. Further, the latent class consistency becomes less affected by the missing data as the number of attributes associated with the items increase. With an increase in the number of attributes associated with the items, the missing data in these items affect the consistency level of the g parameter (guessing) less and the s parameter (slip) more. Furthermore, it can be observed from the results that the test developers using the cognitive diagnosis models should specifically consider the item–attribute relation in items with missing data.
Anahtar Kelime: Kayıp Veri Varlığında DINA Model Madde-Özellik İlişkisinin Parametre Kestirimine Etkisi
Öz: Bu araştırmanın amacı, farklı oranlarda kayıp veri varlığında madde-özellik sayısı ilişkisinin, DINA model kestirimlerini nasıl etkilediğini incelediğini simülasyon verileri üzerinden incelemektir. Verilerin üretimlesinde dört özellik ve 24 maddeden oluşan bir Q matris kullanılmıştır. Q matrixteki ilk, orta ve son 8 madde sırasıyla 1, 2 ve 3 özellikle ilişkilendirilerek 3000 kişilik bir veri seti üretilmiş ve bu verilerde yer alan her 8 maddelik bloktan sırası ile %5, %10 ve %15 veri rassal silinmiştir. Ardından, bu veri setlerine MI yöntemi ile imputasyon yapılmıştır. Bu işlemler, her bir koşul için 100 kez tekrarlanmıştır. Bu veri setlerinden elde edilen kestirimler, kayıpsız veri setinden elde edilen değerler ile karşılaştırılmıştır. Araştırmanın bulguları kayıp veri miktarındaki artışın, DINA model parametre ve örtük sınıf kestirimlerindeki tutarlılığı olumsuz yönde etkilediğini göstermiştir. Maddenin ilişkili olduğu özellik sayısı arttıkça örtük sınıf uyumu kayıp veriden daha az etkilenmiştir. Maddenin ilişkili olduğu attribute sayısı arttıkça bu maddelerde gözlenen kayıp veri, testin g parametresi uyum düzeyini daha az, s parametresini daha çok etkilemiştir. Araştırmanın sonuçları özellikle CDM modellerini kullanan test geliştiricilerinin kayıp veri gözlenen maddelerde, madde-özellik ilişkisini göz önünde bulundurmaları gerektiğini göstermektedir.
Anahtar Kelime: Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Erişime Açık
- Başokçu, T. O. , Kalkan, Ö. K., & Öğretmen, T. (2016, September). DINA Modele Dayalı Madde Parametre Kestiriminde Kayıp Veri Ele Alma Yöntemleri Etkisinin İncelenmesi. [An Investigation of the Effect of Missing Data Handling Methods on Parameter Estimation Based on DINA Model]. Paper presented at the Fifth International Congress on Measurement and Evaluation in Education and Psychology, Antalya, Turkey. p134-135. Abstract retrieved from http://epod2016.akdeniz.edu.tr/_dinamik/333/53.pdf
- Bennett, D. A. (2001). How can i deal with missing data in my study? Australian and New Zealand journal of public health, 25(5), 464-469.
- Buuren, S. V., & Groothuis-Oudshoorn, K. (2011). mice: Multivariate imputation by chained equations in R. Journal of statistical software, 45(3), 1-67.
- Chen, J. (2017). A Residual-Based Approach to Validate Q-Matrix Specifications. Applied Psychological Measurement, (135), 014662161668602. https://doi.org/10.1177/0146621616686021
- de La Torre, J. (2008). An Empirically Based Method of Q Matrix Validation for the DINA Model: Development and Applications. Journal of educational measurement, 45(4), 343-362.
- de La Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of educational and behavioral statistics, 34(1), 115-130.
- de La Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179-199.
- de La Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69(3), 333-353.
- de la Torre, J., & Douglas, J. A. (2008). Model evaluation and multiple strategies in cognitive diagnosis: An analysis of fraction subtraction data.Psychometrika, 73(4), 595-624.
- de La Torre, J., Hong, Y., & Deng, W. (2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47(2), 227- 249.
- de la Torre, J., & Lee, Y. S. (2010). A Note on the Invariance of the DINA Model Parameters. Journal of Educational Measurement, 47(1), 115–127. https://doi.org/10.1111/j.1745-3984.2009.00102.x
- Dong, Y., & Peng, C. Y. J. (2013). Principled missing data methods for researchers. SpringerPlus, 2(1), 222.
- Doornik, J. A. (2018). An Object-Oriented Matrix Programming Language Ox 8. Timberlake Consultants.
- Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah. NJ: Erlbaum.
- Enders, C. K. (2010). Applied missing data analysis. Guilford Press.
- Finch, H. (2008). Estimation of item response theory parameters in the presence of missing data. Journal of Educational Measurement, 45(3), 225-245.
- Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37(6), 359–374. https://doi.org/10.1016/0001-6918(73)90003-6.
- Graham, J. W., Taylor, B. J., Olchowski, A. E., & Cumsille, P. E. (2006). Planned missing data designs in psychological research. Psychological methods, 11(4), 323.
- Haertel, E. H. (1989). Using Restricted Latent Class Models to Map the Skill of Achievement Structure Items. Journal of Educational Measurement, 26, 333–352. http://dx.doi.org/10.1111/j.1745- 3984.1989.tb00336.x
- Hartz, S. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Unpublished doctoral dissertation, Department of Statistics, University of Illinois, Urbana-Champaign.
- Henson, R., & Douglas, J. (2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29(4), 262-277.
- Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74(2), 191.
- Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258-272.
- Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data. Wiley. New York.
- Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64(2), 187-212.
- Nichols, P. D., Chipman, S. F., & Brennan, R. L. (2012). Cognitively diagnostic assessment. Routledge.
- Peugh, J. L., & Enders, C. K. (2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of educational research, 74(4), 525-556.
- R Core Team. (2018). R: A language and environment for statistical computing [Computer Software]. Vienna, Austria: R Foundation for Statistical Computing.
- Robitzsch, A., Kiefer, T., George, A. C., Uenlue, A., & Robitzsch, M. A. (2018). Package ‘CDM’.
- Rupp, A. A., & Mislevy, R. J. (2007). Cognitive Foundations of Structured Item Response Models. In Leighton, J., Gierl, M. (Eds.). Cognitive diagnostic assessment for education: Theory and applications, 205-241. New York: Cambridge University Press.
- Rupp, A. A., & Templin, J. (2008). The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model. Educational and Psychological Measurement, 68(1), 78-96.
- Schlomer, G. L., Bauman, S., & Card, N. A. (2010). Best practices for missing data management in counseling psychology. Journal of Counseling psychology, 57(1), 1.
- Sen, S., & Bradshaw, L. (2017). Comparison of Relative Fit Indices for Diagnostic Model Selection. Applied Psychological Measurement, 014662161769552. https://doi.org/10.1177/0146621617695521
- Sijtsma, K., & Van der Ark, L. A. (2003). Investigation and treatment of missing item scores in test and questionnaire data. Multivariate Behavioral Research, 38(4), 505-528.
- Sorrel, M. A., Olea, J., Abad, F. J., de la Torre, J., Aguado, D., & Lievens, F. (2016). Validity and Reliability of Situational Judgement Test Scores. Organizational Research Methods, 19(3), 506– 532. https://doi.org/10.1177/1094428116630065
- Stekhoven, D. J. (2016). MissForest: nonparametric missing value imputation using random forest. R package version 1.4.
- Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Allyn & Bacon/Pearson Education.
- Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of educational measurement, 20(4), 345-354. http://dx.doi.org/10.1111/j.1745- 3984.1983.tb00212.x
- Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological methods, 11(3), 287.
- van der Linden, W. J., & Hambleton, R. K. (2013). Item Response Theory: Brief History, Common Models, and Extensions. In Handbook of Modern Item Response Theory. https://doi.org/10.1007/978-1-4757-2691-6_1
- von Davier, M. (2005). A general diagnostic model applied to language testing data (ETS Research Report RR-05-16). Princeton, NJ: Educational Testing Service.
- Winship, C., Mare, R. D., & Warren, J. R. (2002). Latent class models for contingency tables with missing data. Applied latent class analysis, 408.
- Zhang, B., & Walker, C. M. (2008). Impact of missing data on person—model fit and person trait estimation. Applied Psychological Measurement, 32(6), 466-479.
| APA | Kalkan Ö, basokcu o (2019). The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. , 290 - 306. 10.9779/pauefd.546797 |
| Chicago | Kalkan Ömür Kaya,basokcu oguz The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. (2019): 290 - 306. 10.9779/pauefd.546797 |
| MLA | Kalkan Ömür Kaya,basokcu oguz The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. , 2019, ss.290 - 306. 10.9779/pauefd.546797 |
| AMA | Kalkan Ö,basokcu o The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. . 2019; 290 - 306. 10.9779/pauefd.546797 |
| Vancouver | Kalkan Ö,basokcu o The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. . 2019; 290 - 306. 10.9779/pauefd.546797 |
| IEEE | Kalkan Ö,basokcu o "The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data." , ss.290 - 306, 2019. 10.9779/pauefd.546797 |
| ISNAD | Kalkan, Ömür Kaya - basokcu, oguz. "The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data". (2019), 290-306. https://doi.org/10.9779/pauefd.546797 |
| APA | Kalkan Ö, basokcu o (2019). The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi, 0(46), 290 - 306. 10.9779/pauefd.546797 |
| Chicago | Kalkan Ömür Kaya,basokcu oguz The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi 0, no.46 (2019): 290 - 306. 10.9779/pauefd.546797 |
| MLA | Kalkan Ömür Kaya,basokcu oguz The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi, vol.0, no.46, 2019, ss.290 - 306. 10.9779/pauefd.546797 |
| AMA | Kalkan Ö,basokcu o The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi. 2019; 0(46): 290 - 306. 10.9779/pauefd.546797 |
| Vancouver | Kalkan Ö,basokcu o The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi. 2019; 0(46): 290 - 306. 10.9779/pauefd.546797 |
| IEEE | Kalkan Ö,basokcu o "The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data." Pamukkale Üniversitesi Eğitim Fakültesi Dergisi, 0, ss.290 - 306, 2019. 10.9779/pauefd.546797 |
| ISNAD | Kalkan, Ömür Kaya - basokcu, oguz. "The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data". Pamukkale Üniversitesi Eğitim Fakültesi Dergisi 46 (2019), 290-306. https://doi.org/10.9779/pauefd.546797 |
