學刊論文
Exploring the Variables That Influence the Performance of the DIF-free-then-DIF Strategy in Assessing Differential Item Functioning

DOI:10.6129/CJP.201812_60(4).0003
中華心理學刊 民107,60卷,4期,251-265
Chinese Journal of Psychology 2018, Vol.60, No.4, 251-265


Chi-Chen Chen(Institute of Education, National Sun Yat-sen University;Assessment Research Center, National Sun Yat-sen University);Yeh-Tai Chou(Research Center for Psychological and Educational Testing, National Taiwan Normal University);Ching-Lin Shih(Institute of Education, National Sun Yat-sen University;Center for Teacher Education, National Sun Yat-sen University;Assessment Research Center, National Sun Yat-sen University)

Conventional differential item functioning (DIF) assessment methods tend to yield an inflated type I error rate and a deflated power rate when the tests contain many DIF items that favor the same group. To control type I error rates in DIF assessments under similar conditions, the DIF-free-then-DIF (DFTD) strategy is proposed. The DFTD strategy consists of two steps: (1) selecting a set of items that is most likely to be DIF-free, and (2) assessing DIF for other items using the designated items as anchors. To explore the variables that influence the performance of the DFTD strategy in assessing DIF, a series of  simulation studies was implemented in this study. Three multiple indicators, multiple causes (MIMIC) methods, namely the standard MIMIC method (M-ST), the M IMIC method with scale purification (M-SP), and the iterative MIMIC method (M-IT), were used to select four items as an anchor set before implementing the DFTD strategy. The results of the analysis of variance showed significant differences among M-IT, M-SP, and M-ST in identifying DIFfree items, with M-IT performing better than M-SP, and M-SP performing better than M-ST. The analysis also found that the main effects of DIF patterns, DIF percentages, sample sizes, and item response theory (IRT) models, as well as their interactions, were significant in terms of their accuracy in identifying the DIF-free items. Based on the results, the M-SP and M-IT methods are recommended for use in identifying DIF-free items, especially when there are many DIF items in a test. The same set of variables significantly determined the power rates of these methods in assessing DIF. However, the type I error rates in the DIF assessments were significantly influenced by the DIF patterns, DIF percentages, and sample sizes. Based on the results of this study, it is recommended that R500/F500, as well as data fits two-parameter logistic model (2PLM), be adopted when applying the DFTD strategy with t he MIMIC method in assessing DIF. 

Keywords: MIMIC, DIF-free-then-DIF strategy, scale purification, item response theory, confirmatory factor analysis

登入
會員登入
更新驗證碼