Title: Optimizing Statistical Power in Complex Study Designs in Psychology: An
Application of Machine Learning

Authors: Rudolf Debelak, Felix Zimmer

Abstract:
The planning of an empirical study in psychology always includes the planning
of the sample collection. This requires the balancing of several factors: On
the one hand, the data collection should be as inexpensive as possible, and on
the other hand, the collected data set must be large enough to reliably detect
suspected effects. Furthermore, planning the data collection may involve
complex decisions, such as determining the number of groups and individuals
per group in a multilevel design. These decisions, in turn, affect the cost
and power of statistical tests. In this talk, we present a new method that
supports finding an optimal study design using machine learning methods.
Possible applications are a) finding a design with fixed power that is as
inexpensive as possible, and b) finding a design with as high power as
possible given a fixed cost budget. The new method is demonstrated using some
scenarios from classical statistics, multilevel modeling, and item response
theory. In summary, we find that the application of the new method is
promising and more efficient than the evaluation of possible study designs by
hand. The proposed method is implemented in the R package mlpwr.