Development and validation of a questionnaire to evaluate overt aggression and reactions to peer aggression

Desenvolvimento e validação de um questionário para avaliar comportamentos agressivos e reações à agressão entre pares

PSYCHOLOGICAL ASSSESSMENT

Development and validation of a questionnaire to evaluate overt aggression and reactions to peer aggression

Desenvolvimento e validação de um questionário para avaliar comportamentos agressivos e reações à agressão entre pares

Paola Gremigni^I; Bruno Figueiredo Damásio^II; Juliane Callegaro Borsa^II

^IUniversidade de Bologna, Bologna, Itália

^IIUniversidade Federal do Rio Grande do Sul, Porto Alegre, Brasil

^{Endereço para correspondência}

ABSTRACT

This study presents the psychometric properties of the Peer Aggressive and Reactive Behaviors Questionnaire (PARB-Q), a self-report questionnaire to measure child overt aggression (PA) and reaction to peer aggression (RPA). Participants were 587 Italian elementary school children, aged 7-10 years (51.5% female). Exploratory and confirmatory factor analyses presented a one-factor solution for the PA scale and a three-factor solution for the RPA scale (reactive aggression, seeking teacher support and internalizing reaction), presenting adequate reliability and goodness-of-fit indexes for both scales. Criterion validity presented satisfactory evidence. The PARB-Q appears to be a psychometrically sound tool to evaluate aggressive behavior and reactions to peer aggression in elementary school children.

Keywords: Aggressive behavior, reactive aggression, children, validation, self-report questionnaire.

RESUMO

Este estudo apresenta as propriedades psicométricas do Questionário de Comportamentos Agressivos e Reativos entre Pares (Q-CARP), instrumento de autorrelato que avalia comportamentos agressivos infantis diretos (ECA) e reações frente à agressão de pares (ERA). Participaram do estudo 587 crianças italianas, estudantes do ensino fundamental, com idade entre 7-10 anos (51,5% meninas). Análises fatoriais exploratórias e confirmatórias apresentaram uma solução unifatorial para a escala ECA e uma solução de três fatores para a ERA (reações agressivas, busca de apoio do professor e reações internalizadas), apresentando bons índices de consistência interna e de ajuste para ambas as escalas. A validade de critério apresentou índices satisfatórios. O Q-CARP apresenta-se como uma ferramenta útil para avaliar comportamentos agressivos e diferentes reações em crianças escolares.

Palavras-chave: Comportamento agressivo, reação agressiva, crianças, validação, questionário autorrelato.

Peer aggression in schools is a common problem of interaction between children (Hubbard, McAuliffe, Morrow, & Romano, 2010) and a major area of investigation, as 9-54% of school-age children from 25 nations were found to be involved in aggressive acts (Nansel et al., 2004). Aggression is defined as any behavior intended to hurt, harm, or injure another person (Coie & Dodge, 1998). Childhood aggression has potentially serious consequences for both aggressors and victims and has been shown to predict negative health outcomes in adulthood (Reijntjes et al., 2011; Temcheff, Serbin, & Martin-Storey, 2011).

Most research on peer aggression has focused on different forms and functions of aggression (Bushman & Huesmann, 2010). Forms refer to how the aggressive act is expressed, such as physical versus verbal and direct/overt versus indirect/covert (i.e., relational aggression). Overt aggression occurs when the aggressor directly attacks the victim, physically (e.g., hitting, kicking, stabbing, or shooting) or verbally (e.g., yelling, screaming, swearing, name calling). Although physical aggression tends to decrease after early childhood, some children show chronic aggression into adolescence and beyond (Underwood, Beron, & Rosen, 2009). Studies indicate that in elementary years children use more direct and overt forms of aggression, and then, over time, direct aggression decreases and social aggression increases (e.g., Björkqvist, 1994; Underwood et al., 2009).

A large body of research points to the existence of sex differences in aggressive behaviors (Tapper & Boulton, 2005). In general, boys use more overt aggression than girls (Card, Stucky, Sawalani, & Little, 2008; Lim & Ang, 2009); however, the literature is not consensual about different aggression forms according to gender (Björkqvist, 1994; Kistner et al., 2010). Children's aggressive behaviors also change with development in relation to improvements in social skills and individuals may follow different trajectories across the lifespan (Smith, Shu, & Madsen, 2001). Aggressive behaviors emerge in the first years of school, and continue through middle childhood and adolescence. Hence, age is an important variable, although not much attention has been paid to it (Kristensen & Smith, 2003).

Peer aggression is a common form of situational factor that may precipitate different reactions in elementary schools children (Craig, Pepler, & Blais, 2007). Among the strategies used by the victims in response to peers verbal or physical agression, are reacting aggressively, telling someone in order to get help, crying and internalizing, and ignoring or distraction (Kristensen & Smith, 2003; Tapper & Boulton, 2005). These strategies may have different consequences. Studies have found that girls were more likely than boys to respond to victimization with helplessness or internalizing behavior or by telling someone about victimization experiences (Craig et al., 2007; Hunter & Boyle, 2004). Boys, instead, were more likely than girls to respond aggressively to a provocation (Craig et al., 2007; Kristensen & Smith, 2003). Gender-specific attitudes about aggression and seeking help may also change with age (Kochenderfer-Ladd & Pelletier, 2008). Younger children were found to respond to victimization by internalizing (i.e., crying and running away) and telling someone about the peer aggression more often than older children (Kristensen & Smith, 2003).

Avoidance strategies, physical and verbal agressive responses are associated with higher levels of victimization. Seeking social support is an effective strategy for girls predicting lesser levels of victimization. In the other hand, seeking social support is associated with higher levels of victimization in boys (Houbre, Tarquinio, & Lanfranchi, 2010; Shelley & Craig, 2010). Thus, results of the literature suggest taking into account age and gender differences when studying aggressive behaviors or responses to them.

In order to evaluate child aggressive behaviors several instruments were developed (Farmer & Aman, 2010). Most of them are responded by peers or teachers (Little, Henrich, Jones, & Hawley, 2003). Peer nomination measures ask children to nominate peers who exhibit certain behavior. This method has the limitation that only children who use aggressive behaviors most frequently are nominated, and not every child receives a score. Peer rating measures ask children to rate every peer in the classroom on the frequency of aggressive behaviors. The disadvantages of using this method are that it is a time consuming process, and children may be rating only classmates they are close to. Teacher reports ask teachers to rate each child in their classroom on the frequency of aggressive behaviors. Teachers rating are said to be affected by the intentionality and stability of various disruptive behaviors (Phillips & Lonigan, 2010; Wrobel, Lachar, & Wrobel, 2005). Furthermore, most teacher reports predict behaviors observed based on adult-child interactions (De Los Reyes, Alfano, & Beidel, 2010).

Self-report measures ask children to report the frequency of their behaviors. The use of self-report scales in children is a relatively new phenomenon, but it seems to provide valuable information about children's feelings and behaviors, as elementary school children have a good self-perspective (Frick, Barry, & Kamphausthe, 2010). Taking into account that all ways to measure children behaviors have advantages and disadvantages, in this study we decided to develop a self-report measure, in view that many aggressive actions often take place away from the sight of teachers (Hyman et al., 2006). According to the authors' knowledge, there are no brief, user-friendly, cost-effective, reliable, and valid self-report questionnaires for the assessment of both children overt aggression and reaction to overt aggression by peers.

Current Study

The objective of this study was twofold: (a) to validate the Peer Aggressive and Reactive Behaviors Questionnaire (PARB-Q), a new empirically-based self-report scale developed to assess verbal and physical overt aggression and reactive behaviors to peer aggression in elementary school children, and (b) to evaluate if there were differences on the PARB-Q scores between children of different gender and age. In particular, it was hypothesized, based on the literature, that males were more likely than females to act and react aggressively, while females were expected to react by seeking social support and internalizing more than males. Lastly younger children were expected to respond to peer aggression by internalizing and seek social support more often than older children.

Method

Participants and Procedures

Participants were 587 Italian elementary school children attending second to fifth grade from a total of 29 classes of six schools in northeast and central Italy, selected by convenience. Children were aged 7-10 years (M = 8.70 years, SD = 1.07 years). This age range was chosen because, in Italy, children begin elementary school at the age of six and attend it for five years, but children in first grade were excluded because of their limited reading skills at the time of data collection. As many as 69 children (11.7%) failed to indicate their gender; most of them (62%) were fifth graders. Among respondents 51.5% were females. All the schools participating in the study were public schools of small urban and sub-urban districts.

This study was carried out in accordance with the Declaration of Helsinki and was aproved by University of Bologna Institutional Review Board. The data were collected after obtaining consent from school boards and informed consent from parents. Questionnaires were administered collectively in classrooms and took about 10-15 min to be completed. A subsample of 60 randomly chosen children completed also another questionnaire along with the PARB-Q and filled out the PARB-Q again after a 6-week interval. Five among all teachers participating in the study were randomly selected and asked to evaluate their students (n = 102; 5 classes) on social skills. All of them completed the requested questionnaires. It should be noted that in Italian primary schools, each teacher is in charge of a class that includes about 20 students.

Development of the Peer Aggressive and Reactive Behaviors Questionnaire (PARB-Q)

The PARB-Q is formed by two scales and by a total of 20 items with a 4-point response format (from "never" to "almost every day"). An initial pool of about 100 items was formulated by a team of three clinical and school psychologists, on the basis of observational and psychometrical studies available in the literature. A focus group was then conducted with 7 elementary school teachers to discuss the initial pool of the items and help researchers to select them as well as to identify appropriate language for the questionnaire design. Teachers expressed their main interest in assessing children's overt aggression and responses to overt aggression by peers, because they frequently observed these behaviors and their negative impact on both peer interactions and academic performance, as confirmed by empirical studies (e.g., Buhs, Ladd, & Herald, 2006). Another important teachers' concern was to obtain a very brief and easy to handle questionnaire to be used during everyday school activity. In order to develop a brief questionnaire, the focus group helped to select a total of 20 items, which were then reviewed by an independent elementary school teacher and three children for linguistic adequacy to the target group.

The PARB-Q is conceived as two separate scales measuring children's aggression and reactions to peer aggression, respectively. The first 8-item scale, named Peer Aggression Scale (PA), asks children to report the daily frequency of their overt verbal and physical aggressive behavior toward peers (e.g., "how often do you say bad things to your classmates?"; "how often do you kick or hit your classmates?"). Within this scale, there are also three control items (e.g., tell jokes, get happy, and watching cartoons on TV), which are not considered for the scoring. The second 12-item scale, named Reaction to Peer Aggression Scale (RPA) presents children with three hypothetical situations and asks them to report the frequency of their habitual use of three specific strategies in response to each situation. All three types of stimulus represent overt aggressive behavior (i.e., verbal aggression, physical aggression, and attack on property) to reduce ambiguity of the situational cues. The strategies proposed for each stimulus represent reactive aggression, either verbally or physically ("I yell at him/her"; "I hit or kick him/her"); seeking the teacher's help ("I tell the teacher about it"), and internalized reaction ("I cry about it or sulk").

Other Measures

The Childhood Social Adjustment Capacity Indicators Questionnaire, (CSACIQ, child self-report form; Caprara, Pastorelli, Barbaranelli, & Vallone, 1992) consists of three subscales representing three indicators of a child's ability to interact in social contexts: emotional instability (EI), pro-social behavior (PB), and aggressive behavior (A). In the present study we used only the A scale, a 14-item self-report measure of children's tendency to commit acts of physical and verbal aggression. The reliability of the CSACIQ-A, assessed with Cronbach's alpha, in this study was .85.

The Teacher Skill-streaming Checklist (TSC; McGinnis & Goldstein, 1997) assesses teachers' perception of the social abilities of school-age children. The complete TSC includes descriptions of a broad range of social skills. In this study we used the 9-item subscale that evaluates children's strengths in the skill alternatives to aggression (SAA), such as using self-control, responding adequately to teasing, dealing with someone else's anger, keeping out of fights, dealing with an accusation, and negotiating. The reliability of the SAA scale in this study was alpha = .94.

Data Analyses

Two random halves were created with 293 and 294 observations, respectively. Exploratory Factor Analysis (EFA) was used on the first half of the sample as an exploratory first step to identify the internal structure of the PARB-Q. Confirmatory Factor Analysis (CFA) was then used on the other half of the sample to examine whether the structure identified in the EFA worked in a different sample. Both analyses were performed on the Peer Aggression Scale (PA) scale and the Reaction to Peer Aggression (RPA) scale of the PARB-Q, separately.

EFA was performed on each scale using Principal Axis Factoring (PAF) analysis with subsequent orthogonal rotation (Varimax rotation with Kaiser normalization) to maximize independence of factors and obtain a more parsimonious and replicable solution. PAF is recommended in case that the assumption of multivariate normality of data is violated (Fabrigar, Wegener, MacCallum, & Strahan, 1999). The number of extracted factors was confronted with a Monte-Carlo Parallel Analysis, with 95% confidence interval (CI), and 1,000 random correlation matrices, as the primary factor retention criterion (Patil, Singh, Mishra, & Donovan, 2008). CFA was performed using the robust maximum likelihood extraction method (i.e., with corrections for data non-normality, Satorra & Bentler, 2001). No correlated errors were specified and the pattern of factor co-variances was specified on the basis of the exploratory model. The goodness of fit indexes used were: model chi-square, Root Mean Square Error of Approximation (RMSEA) and its associated confidence interval, Standardized Root Mean Square Residual (SRMR), Comparative Fit Index (CFI), and Tucker-Lewis Index (TLI). These indexes are among the least sensitive to sample size, model misspecification and parameter estimates (Hu & Bentler, 1999). We also reported the relative chi-square (χ²/df) as an attempt to make the model chi-square less dependent on sample size. Values of χ²/df < 2; SRMR <.08; RMSEA <.06 with the upper CI interval <.08; CFI and TLI >.90, are indicative of an acceptable model (Hu & Bentler, 1999).

The PARB-Q reliability was assessed using Cronbach's alpha in the total sample (n = 587) for both the scales and each factor. Confidence intervals about the reliability estimates and the statistical significance test against a hypothesized reliability value (i.e., H₀: α = .70) were also calculated, as suggested by Fan and Thompson (2001). An Intraclass Correlation Coefficient (ICC) using 2-way, single measures, random-effects model that address both systematic and random error was calculated to determine stability of the scores over a 6-week interval in a subsample of 60 subjects. An ICC minimal acceptable level was .80 (Nunnally & Bernstein, 1994). Criterion validity was analyzed by performing Person's r correlations between the PARB-Q and the CSACIQ-A and SAA scales in subsamples of 60 and 102 participants, respectively.

Potential gender and age effects on the PARB-Q in the total sample (n = 587) were investigated using age as a categorical independent variable (level of significance p <.05). First, the homogeneity of variances was tested with Levene's test, and then analyses of variance (ANOVA) were performed, based on Welch's test in case of non-equality of variances in the subgroups of participants. In the case of age groups, planned constrasts were then performed.

Results

Factor Structure of the PARB-Q

EFA performed on the first half of the sample (n = 293), with the parallel analysis, presented a one-factor solution for the PA scale including all five items, with loadings >.60. Total explained variance was 56.77% (see Table 1).

For the RPA scale, the EFA as well as the parallel analysis presented a three factor solution explaining 55.88% of the total variance. All items loaded higher than .55 on one factor and lesser than .20 on the other factors (see Table 2). Based on the content of items, the factors were named Reactive Aggression (RA, 6 items), Seeking Teacher Support (STS, 3 items), and Internalizing Reaction (IR, 3 items).

There was a moderate inter-factor correlation for STS H IR (r = .22, p <.001) and a low albeit significant correlation for RA H RI (r = .13, p <.001), while RA and STS were uncorrelated (r = -.08, p >.05). A strong correlation was found between PA and RA (r = .72, p <.001), while significant but low correlations were found for PA H STS (r = -.12, p <.001) and PA H IR (r = .17, p < .001).

Concerning the CFA performed on the second half of the sample (n = 294), in the case of the RPA scale, pattern of factor co-variances was specified as follows: IR correlated with both RA and STS, RA uncorrelated with STS. The goodness-of-fit indexes confirmed the factor structure emerged form EFAs showing good interpretability of the models (see Table 3). Standardized factor loadings (regression weights) ranged from .66 to .76 for the PA scale and from .58 to .88 for the RPA factors.

Criterion Validity

Although this study focused mainly on assessing the PARB-Q factorial structure, a preliminary assessment of criterion validity was also performed. In a sub-sample of 60 children, the self-reported CSACIQ-A scores showed significant positive correlations with PA (r = .79, p < .01), RA (r = .75, p <.01), and IR (r = .44, p <.01), but not with STS (r = .06, p =.73). In a sub-sample of 120 children evaluated by their teachers, the TSC-SAA showed significant negative correlations with PA (r = -.53, p <.01), RA (r = -.55, p <.01), and IR (r = -.25, p <.01), and a positive correlation with STS (r = .26, p <.01).

Reliability

Coefficients of internal consistency were significantly larger than the minimum expected value of .70 for both scales and all factors, with alphas ranging between .75 and .86 (see Table 4).

Test-retest reliability of the PARB-Q over a 6-week period, performed in a sub-sample of 60 children, was acceptable, with all ICC values above .80. In particular, the PA scale had an ICC = .87 (95% CI: .81-.91). For the overall RPA scale ICC was .85 (95% CI: .78-.89), and for each RPA factor ICCs were as follows: .87 (95% CI: .82-.91) for RA; .85 (95% CI: .77-.90) for STS, and .86 (95% CI: .80-.91) for IR.

Gender and Age Differences

Concerning gender and age differences on the PARB-Q scores, in all analyses the interaction gender H age was not significant, the main effects being independent of each other. Descriptive statistics are presented in Table 5.

PA and RA variances were not homogeneous between groups (Levene's test F_(7,479)= 4.31, p <.001 and F_(7,479)= 9.59, p <.001, respectively); comparisons were performed with Welch's test and contrasts were then run not assuming the homogeneity of variances. In both PA and RA, males scored significantly higher than females (PA: Welch's test_(1,435)= 40.88, p <.001; RA: Welch's test_(1,392)= 44.47, p <.001). Regarding age, significant differences emerged on both PA and RA (PA: Welch's test_(3,188)= 22.31, p <.001; RA: Welch's test_(3,171)= 6.31, p <.001). Planned contrasts (not assuming equal variances) showed that children aged 7 reported less aggressive behavior (PA: t₍₅₈₃₎= -8.02, p <.001) and less aggressive reaction (RA: t₍₅₈₃₎= -4.03, p <.001) than those aged 8, 9 and 10 years taken together.

STS and IR showed homogeneous variances (Levene's test F_(7,479) = .59, p =.77 and F_(7,479)= 1.58, p =.14, respectively); therefore, ANOVA was used to test differences between groups. Females scored higher than males on IR scores (F₍₁₎= 10.76, p =.001), while differences on STS score were not statistically significant (F₍₁₎= 3.47, p =.06). Age effects were also significant on both STS (F₍₃₎= 8.96, p <.001) and IR scores (F₍₃₎= 2.84, p =.04). Younger children scored higher than older children on STS, with significant planned contrasts of children aged 7 and 8 together against those aged 9 and 10 together (STS: t₍₅₈₃₎ = 3.89, p <.001). On IR, children aged 8 scored higher than children aged 7, 9 and 10 together (t₍₅₈₃₎ = 3.84, p <.001).

Discussion

This study aimed to validate the Peer Aggressive and Reactive Behaviors Questionnaire (PARB-Q), a newly developed self-report questionnaire to assess overt aggression and reaction to peer overt aggression in elementary school children.

The PARB-Q presents advantages compared to other existing instruments: it brings together a brief scale that assesses both the aggressive behavior and reactions to the suffered aggression. Instruments that measure aggressive behaviors do not generally take into account the reactions to peer aggression (Dodge & Coie 1987; Kempes, Matthys, Maassen, & Van Engeland, 2006; Little et al., 2003).

Both exploratory and confirmatory factor analyses suggest adequacy of the factorial structure. Adequate alpha and test-retest reliability were also observed in both scales and all factors. Preliminary results of criterion validity analyses were consistent with the literature and clarified the characteristics of the distinct behaviors examined. Results of this study indicated that elementary school children tend to react with reactive aggression, internalizing or seeking teacher support, independent of types of aggressive provocation by peers (e.g., physical, verbal, or attack on property). These types of reaction are inter-correlated, thus most accurately conceptualized as continuous dimensions, rather than as categories into which children are exclusively placed.

Reactive aggression was positively associated with the tendency to act aggressively. This is consistent with recent studies suggesting that proactive and reactive aggression tend to co-occur, with most aggressive children displaying some degree of both behaviors (Hubbard et al., 2010). Reactive aggression was also negatively associated with social abilities, including self-control of reactions to provocation. In other studies, school-age children manifesting high levels of aggressive interactions with peers also showed lower levels of self- and anger regulation (Little et al., 2003; McAuliffe, Hubbard, Rubin, Morrow, & Dearing, 2006). The positive association found between internalizing response and aggressiveness is consistent with evidence provided by studies on the co-occurrence between externalizing and internalizing problems at different developmental stages (Lilienfeld, 2003).

Seeking teacher support was positively associated with social abilities. Seeking social support has been found to be effective for decreasing the level of victimization, while externalizing, internalizing, and other strategies have been found to increase victimization and stress (Houbre et al., 2010; Newman, Holden, & Delville, 2011).

Regarding gender and age differences, our findings were consistent with those found in the literature. Boys showed more aggressive behaviors and aggressive reactions than girls as it was found in other studies (Lim & Ang, 2009; Monks, Palermiti, Ortega, & Costabile, 2011). Younger children were more likely to seek support in order to get help than did older children as reported by other researchers (Whitney & Smith, 1993).

Limitations

Overall, the PARB-Q show satisfactory validity and reliability indexes; nevertheless, some limitations should be evidenced. The present study did not present paired-sample by age, and it can be seen as a limitation for group comparisons, as it may have reduced the sensitivity in observing differences related to the stage of development. Another limitation is the lack of socio-demographic informations that have an influence on aggressive behaviors (e.g., familiar characteristics, socioeconomic status, contextual factors; Hubbard et al., 2010; Tremblay Gervais, & Petitclerc, 2008). Although the PARB-Q is a psychometrically sound tool, further studies are needed to address criterion validity and test-retest stability, both procedures involving larger samples of children. Finally, the main limitation was the lack of consideration of gender and age invariance in the dimensionality of the PARB-Q. Therefore, we need to collect a wider sample of observations to test measurement invariance.

Conclusion

The PARB-Q is an informative, inexpensive and psychometrically sound tool. Nevertheless, as a newly developed instrument, it requires further studies to investigate its effectiveness in different samples and countries, taking into consideration socio-demographic, economic, and family aspects. Despite the limitations, it is a promising measure for use in gathering information on overt aggressive and reactive behavior patterns reported by elementary school children. This information may help to develop interventions to reduce aggression in school, to promote change on behavioral self-regulation and encourage pro- social attitudes.

References

Björkqvist, K. (1994). Sex differences in physical, verbal, and indirect aggression: A review of recent research. Sex Roles, 30(3-4), 177-188.

Buhs, E. S., Ladd, G. W., & Herald, S. L. (2006). Peer exclusion and victimization: Processes that mediate the relation between peer group rejection and children's classroom engagement and achievement. Journal of Educational Psychology, 98, 1-13.

Bushman, B. J., & Hueasmann, R. (2010). Aggression. In S. T. Fiske, D. T. Gilbert, & G. Lindzey, Handbook of social psychology (5^th ed., pp. 833-863). Newark, NJ: John Wiley & Sons.

Caprara, G. V., Pastorelli, C., Barbaranelli, C., & Vallone, R. (1992). Indicatori della Capacità di Adattamento Sociale in Età Evolutiva. Firenze, Italia: Organizzazioni Speciali.

Card, N. A., Stucky, B. D., Sawalani, G. M., & Little T. D. (2008). Direct and indirect aggression during childhood and adolescence: A meta-analytic review of gender differences, intercorrelations, and relations to maladjustment. Child Development, 79(5), 1185-1229.

Coie, J. D., & Dodge, K. A. (1998). Aggression and antisocial behavior. In W. Damon & N. Eisenberg (Eds.), Handbook of child psychology: Social, emotional, and personality development (5^th ed., pp. 779-862). Toronto, Canada: John Wiley & Sons.

Craig, W., Pepler, D., & Blais, J. (2007). Responding to bullying: What works? School Psychology International, 28, 465-477.

De Los Reyes, A., Alfano, C. A., & Beidel, D. C. (2010). The relations among measurements of informant discrepancies within a multisite trial of treatments for childhood social phobia. Journal of Abnormal Child Psychology, 38(3), 395-404.

Dodge, K. A., & Coie, J. D. (1987). Social-information-processing factors in reactive and proactive aggression in children's peer groups. Journal of Personality and Social Psychology, 53(6), 1146-1158.

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299.

Fan, X., & Thompson, B. (2001). Confidence intervals around score reliability coefficients, please: An EPM guidelines editorial. Educational and Psychological Measurement, 61, 517-532.

Farmer, C., & Aman, M. (2010). Psychometric properties of the Children's Scale of Hostility and Aggression (C-SHARP). Research in Developmental Disabilities, 31, 270-280.

Frick, P. J., Barry, C. T., & Kamphaus, R. W. (2010). Clinical assessment of child and adolescent personality and behavior (3^rd ed.). New York: Springer.

Houbre, B., Tarquinio, C., & Lanfranchi, J. B. (2010). Expression of self-concept and adjustment against repeated aggressions: The case of a longitudinal study on school bullying. European Journal of Psychology of Education, 25, 105-123.

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

Hubbard, J. A., McAuliffe, M. D., Morrow, M. T., & Romano, L. J. (2010). Reactive and proactive aggression in childhood and adolescence: Precursors, outcomes, processes, experiences, and measurement. Journal of Personality, 78, 95-118.

Hunter, S. C., & Boyle, J. M. (2004). Appraisal and coping strategy use in victims of school bullying. British Journal of Educational Psychology, 74, 83-107.

Hyman, I., Kay, B., Tabori, A., Weber, M., Mahon, M., & Cohen, I. (2006). Bullying: Theory, research and intervention. In C. M. E. Evertson & C. S. E. Weinstein (Eds.), Handbook of classroom management: Research, practice, and contemporary issues (pp. 855-884). Mahwah, NJ: Lawrence Erlbaum.

Kempes, M., Matthys, W., Maassen, G., & Van Engeland, H. (2006). A parent questionnaire for distinguishing between reactive and proactive aggression in children. European Child and Adolescent Psychiatry, 15, 38-45.

Kistner, J., Counts-Allan, C., Dunkel, S., Drew, C. H., David-Ferdon, C., & Lopez, C. (2010). Sex differences in relational and overt aggression in the late elementary school years. Aggressive Behavior, 36(5), 282-291.

Kochenderfer-Ladd, B., & Pelletier, M. E. (2008). Teachers' views and beliefs about bullying: Influences on classroom management strategies and students' coping with peer victimization. Journal of School Psychology, 46(4), 431-453.

Kristensen, S. M., & Smith, P. K. (2003). The use of coping strategies by Danish children classed as bullies, victims, bully/victims, and not involved, in 82 response to different (hypothetical) types of bullying. Scandinavian Journal of Psychology, 44, 479-488.

Lilienfeld, S. (2003). Comorbidity between and within childhood externalizing and internalizing disorders: Reflections and directions. Journal of Abnormal Child Psychology, 31, 285-291.

Lim, S. H., & Ang, R. P. (2009). Relationship between boys' normative beliefs about aggression and their physical, verbal, and indirect aggressive behaviors. Adolescence, 44(175), 635-650.

Little, T. D., Henrich, C. C., Jones, S. M., & Hawley, P. H. (2003). Disentangling the ''whys'' from the ''whats'' of aggressive behaviour. International Journal of Behavioral Development, 27, 122-133.

McAuliffe, M. D., Hubbard, J. A., Rubin, R. M., Morrow, M. T., & Dearing, K. F. (2006). Reactive and proactive aggression: Stability of constructs and relations to correlates. Journal of Genetic Psychology, 167, 365-382.

McGinnis, E., & Goldstein, A. P. (1997). Skillstreaming the elementary school child: New strategies and perspectives for teaching prosocial skills (2^nd ed.). Champaign, IL: Research Press.

Monks, C. P., Palermiti, A., Ortega, R., & Costabile, A. (2011). A cross-national comparison of aggressors, victims and defenders in preschools in England, Spain and Italy. The Spanish Journal of Psychology, 14(1), 133-144.

Nansel, T. R., Craig, W., Overpeck, M. D., Saluja, G., Ruan, W. J., & The Health Behaviour in School-Aged Children Bullying Analyses Working Group. (2004). Crossnational consistency in the relationship between bullying behaviors and psychosocial adjustment. Archives of Pediatrics and Adolescent Medicine, 158, 730-736.

Newman, M. L., Holden, G. W., & Delville, Y. (2011). Coping with the stress of being bullied consequences of coping strategies among college students. Social Psychological and Personality Science, 2, 205-211.

Nunnally, J. C., & Bernstein I. H. (1994). Psychometric Theory (3^rd ed.). New York: McGraw-Hill.

Patil, V. H., Singh, S. R., Mishra, S. K., & Donovan, D. T. (2008). Efficient theory development and factor retention criteria: Abandon the 'eigenvalue greater than one' criterion. Journal of Business Research, 61(2), 162-170.

Phillips, B. M., & Lonigan, C. J. (2010). Child and informant influences on behavioral ratings of preschool children. Psychology in the Schools, 47(4), 374-390.

Reijntjes, A., Kamphuis, J. H., Prinzie, P., Boelen, P. A, van der Schoot, M., & Telch, M. J. (2011). Prospective linkages between peer victimization and externalizing problems in children: A meta-analysis. Aggressive Behavior, 37, 215-222.

Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507-514.

Shelley, D., & Craig, W. M. (2010). Attributions and coping styles in reducing victimization. Canadian Journal of School Psychology, 25(1), 84-100.

Smith, P. K., Shu, S., & Madsen, K. (2001). Characteristics of victims of school bullying developmental changes in coping strategies. In J. Juvonen & S. Graham (Eds.), Peer arrassment in schools (pp. 332-351). New York: Guilford Press.

Tapper, K., & Boulton, M. J. (2005). Victim and peer group responses to different forms of aggression among primary school children. Aggressive Behavior, 31, 238-253.

Temcheff, C. E., Serbin, L. A., & Martin-Storey, A. (2011). Predicting adult physical health outcomes from childhood aggression, social withdrawal and likeability: A 30-year prospective, longitudinal study. Interantional Journal of Behaviorl Medicine, 18, 5-12.

Tremblay, R. E., Gervais, J., & Petitclerc, A. (2008). Prévenir la violence par l'apprentissage à la petite enfance. Montreal, Canada: Centre D'Excellence pour Le Développement dês Jeunes Enfants.

Underwood, M. K., Beron, K. J., & Rosen L. H. (2009). Continuity and change in social and physical aggression from middle childhood through early adolescence. Aggressive Behavior, 35, 357-375.

Whitney, I., & Smith, P. K. (1993). A survey of the nature and extent of bullying in junior/middle and secondary schools. Educational Research, 35, 3-25.

Wrobel, N. H., Lachar, D., & Wrobel, T. (2005). Self-report problem scales and subscales and behavioral ratings provided by peers: Unique evidence of test validity. Assessment, 12(3), 255-269.

Recebido: 11/04/2012

1ª revisão: 14/07/2012

Aceite final: 14/08/2012

Publication Dates

• Publication in this collection
  19 July 2013
• Date of issue
  2013

History

• Received
  11 Apr 2012
• Accepted
  14 Aug 2012
• Reviewed
  14 July 2012