The mixed mode (MM) designs are adopted by NSI both to contrast declining response and coverage rates and to reduce the cost of the surveys. However, MM introduces several issues that must be addressed both at the design phase, by defining the best collection instruments to contain the measurement error, and at the estimation phase, by assessing and adjusting the bias effects (mode effect). The accuracy of the estimates, in fact, has to be protected from possible increase in the total survey error due to extra measurement error introduced by the additional data collection modes.Mixed mode simultaneously generates selection and measurement effects: while selection effects refer to coverage and non-response issues, measurement effects refer to mode features that can influence the answers to survey questions. As the selection and measurement effects are confounded, methods to disentangle the two effects are needed to obtain unbiased estimates of measurement error. Generally, this complex inferential process can be facilitated with experimental survey designs which, being expensive, are rarely used.

The focus of this work is the evaluation of the impact, in terms of quality of the estimates, of switching from traditional PAPI mode to a mixed mode design (sequential web/PAPI) for the ISTAT survey “Aspects of daily life†in the 2017 edition. On this occasion a parallel single mode PAPI design was planned by ISTAT to assess total and specific mode effects.

Firstly, to evaluate if the MM design reduces the selection error with respect to the single mode (SM) design, comparative analyses between the two samples are performed using socio-demographic variables (benchmark variables from registers) and target variables of the survey. The benchmark variables are analyzed individually or jointly; in the latter case they are used in the study of the representativeness of the response. While for the benchmark variables absolute selection error is calculated in each samples, for the target variables the selection error is evaluated in relative terms assuming a single mode PAPI survey as a reference. Starting from the consideration that the mode effect is not unvarying for all the survey variables, univariate and multivariate analyses are used for the main target variables with the aim of evaluating respectively the total effect and the impact of the MM design on the multivariate data structure. Finally, different methods to estimate separately selection and measurement effects of some survey variables are applied: the instrumental variable approach and the propensity score method. The results of the analyses show that mixed mode design has an important impact both on the composition of the respondents sample and on several indicators. If a better coverage of the population is reached, nevertheless the quality of the estimates seems affected by the measurement effects that cannot always be easily evaluated nor correctly adjusted. The approaches to disentangle selection and measurement effects are based on very stringent hypotheses. Moreover, the validity of the models is strictly connected to the availability of auxiliary variables, coming from register or from the survey itself, capable of explaining different bias effects.