The main aim of this study was to compare two rating systems relating to the responses of individuals in the Computerized Emotional Perception Test using Photographs: consensus agreement and agreement with criteria defined by experts based on emotional intelligence theory and Item Response Theory. Participants comprised 55 subjects of both sexes, aged between 18 and 74, whose protocols were rated by both rating methods. Although a high correlation between the two rating methods was observed, the expert agreement criteria yielded significantly higher item-total correlations than consensus agreement. The results suggest that the development of evaluation tools for emotional intelligence related skills should be based primarily on expert agreement criteria, and shows how the Item Response Theory should be helpful in this process of investigation and selection of criteria.
Emotional intelligence; Test reliability; Item response theory
