Open Conference Systems, 50th Scientific meeting of the Italian Statistical Society

Font Size: 
Dealing with Data Evolution and Data Integration: An approach using Rarefaction
Luca Del Core, Eugenio Montini, Clelia Di Serio, Andrea Calabria

Last modified: 2018-06-01

Abstract


Heterogeneity and unreliability of data negatively influence the effectiveness and reproducibility of the results in all fields involving sampling techniques. Heterogeneity is mainly due to technological advances which imply improvements in measurements resolution. Unreliability or under-representativeness in data may be due to machine/software or human variances/errors, or other unidentifiable external factors. In the era of big data, technological evolution, and continuous data integration, scientists are increasingly facing with the problems of how to (1) identify and filter-out unreliable data, and (2) harmonize samples gauged with different platforms improved over time. This work is aimed at developing a new statistical framework to address both issues, showing results in real case scenarios.

Full Text: ZIP