Title: A New Stopping Criterion for Rasch Trees Based on the Mantel-Haenszel
Effect Size Measure

Author: Mirka Henninger

Abstract:
Rasch trees from the model-based recursive partitioning framework are a
data-driven approach for detecting differential item functioning (DIF). They
search for optimal cutpoints in covariates and can therewith identify
previously unknown DIF groups. Rasch trees use a statistical significance test
to determine whether and in which variable a split should be made. Like for
any significance test, small differences are more likely to be detected in
larger samples. For Rasch trees, this means that they are more likely to flag
even small item parameter differences, grow larger and split the sample into
more subgroups when the sample size is large. We propose to use an effect size
measure, namely the Mantel-Haenszel odds ratio, to support the evaluation of
whether a split in a Raschtree is based on a meaningful difference in item
parameters. For this purpose, we have implemented this effect size as an
additional stopping criterion in Rasch trees. The criterion not only allows
Rasch trees to stop from growing when DIF effects are non-substantial, but it
also allows users to identify which items show DIF for each split. In a
simulation study, we show that the effect size further reduces type-1 error,
and avoids splits in Rasch trees in large samples when DIF effects are
non-substantial. The talk also discusses how the stopping rule is implemented
in the statistical software R.