Open Conference Systems, 50th Scientific meeting of the Italian Statistical Society

Font Size: 
Posterior Predictive Assessment for Item Response Theory Models: A Proposal Based on the Hellinger Distance
Mariagiulia Matteucci, Stefania Mignani

Last modified: 2018-05-17

Abstract


Bayesian posterior predictive assessment has received considerable attention for investigating specific aspects of fit of item response theory models. In fact, this approach is easy to apply within Markov chain Monte Carlo estimation, it is flexible and free from distributional assumptions. In its classical implementation, the method is based on graphical analysis and the estimation of posterior predictive p-values to investigate the degree to which observed data are expected under the model, given a discrepancy measure. In this work, we propose to quantify the distance between the realized and the predictive distributions of the discrepancy measure based on the Hellinger distance. The results show that this measure is able to provide clear recommendations about the investigated aspects of model fit.

References


  1. Bernini, C., Matteucci, M., Mignani, S.: Investigating heterogeneity in residents’ attitudes toward tourism with an IRT multidimensional approach. Qual. Quant. 49, 805-826 (2015).
  2. Gelman, A., Meng, X.L., Stern, H.S.: Posterior predictive assessment of model fitness via realized discrepancies. Stat. Sin. 6, 733-807 (1996).
  3. Hellinger, E.: Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen. Journal für die reine und angewandte Mathematik (in German) 136, 210–271 (1909).
  4. Levy, R., Svetina, D.: A generalized dimensionality discrepancy measure for dimensionality assessment in multidimensional item response theory. Br. J. Math. Stat. Psychol. 64, 208-232 (2011).
  5. Levy, R., Mislevy, R.J., Sinharay, S.: Posterior predictive model checking for multidimensionality in item response theory. Appl. Psychol. Meas. 33, 519-537 (2009).
  6. Rubin, D.B.: Bayesianly justifiable and relevant frequency calculations for the applies statistician. Ann. Stat. 12, 1151-1172 (1984).
  7. Sheng. Y., Wikle. C.: Bayesian IRT models incorporating general and specific abilities. Behaviormetrika 36, 27-48 (2009).
  8. Sinharay, S.: Bayesian item fit analysis for unidimensional item response theory models. Posterior predictive assessment of item response theory models. Br. J. Math. Stat. Psychol. 59, 429-449 (2006).
  9. Sinharay, S., Johnson, M.S., Stern, H.S.: Posterior predictive assessment of item response theory models. Appl. Psychol. Meas. 30, 298-321 (2006).
  10. van der Linden, W. J.,  Hambleton, R.K. Handbook of Modern Item Response Theory. Springer-Verlag, New York (1997).
  11. Wu, H., Yuen, K.V., Leung, S.O.: A novel relative entropy-posterior predictive model checking approach with limited information statistics for latent trait models in sparse 2k contingency tables. Comput. Stat. Data Anal. 79, 261-276 (2014).

Full Text: PDF