Open Conference Systems, 50th Scientific meeting of the Italian Statistical Society

Font Size: 
Direct Individual Differences Scaling for Evaluation of Research Quality
Michele Gallo, Nickolay Trendafilov, V Simonacci

Last modified: 2018-06-04

Abstract


The eValuation of Research Quality (VQR) is one the most important assessment processes achieved by the National Agency for the Evaluation of Uni- versities and Research Institutes (ANVUR). Its main task is to provide information on the status of the Italian research system by assessing the performance of univer- sities in various scientific areas. The entities measured are made up of researchers, assistants, first and second band professors, fixed-term professors and researchers, technology and research executives. To achieve this, â€research products†as jour- nal contributions, volume contributions, and other types of scientific products are considered. The basic evaluation criteria were defined by groups of experts (GEV) according to the specific characteristics of each subject area and through a synthetic statement on the products.

In this framework differences between GEV groups on a differential set of qual- ity judgment can be explained in terms of compositional dissimilarity matrices. In literature the INDSCAL (Individual Differences Scaling) model is used to study the individual differences in three-way data by doubly centered set of matrices of squared dissimilarity measures between a range of stimuli. A direct approach is here preferred, defined DINDSCAL (Direct INDividual Differences SCALing), in order to directly analyze simultaneous slices of dissimilarity matrices organized as com- positional data.

The compositional aspect of the data helps to understand easily, which is the re- search product with the highest assessment compared to the remaining ones, irre- spective of the role and the type of institutions to which researchers belong. Addi- tionally, the DINDSCAL algorithm underlines the main divergencies made by each GEV group in terms of research output classification.


Full Text: PDF