Agreement among evaluators in the Nursing Activities Score application

This is a methodological study, carried out in an ICU of a public and university hospital in southern Brazil. The researchers were trained to use the NAS, and after that, a researcher considered a reference standard (RR), two research assistants (RA1 and RA2) and nursing assistants (NUR) applied the instrument independently, considering the same patients. Agreement was tested using the intraclass correlation coefficient (ICC) for the final value of NAS, and kappa coefficient, for analysis of the NAS 23 items.

Results

In the final mean of NAS, there was a strong agreement among RR and RA1 (ICC=0.92; 95%CI: 0.89-0.95) and substantial agreement among RR and RA2 (ICC=0.78; 95%CI: 0.64-0.87) and RR and NUR (ICC=0.75; 95%CI: 0.62-0.84). Agreement regarding NAS items was considered perfect or almost perfect in 14 of the 23 items scored by the pairs of evaluators formed by RR with RA1 and RA2, and in two of the 23 items scored by the pair formed by RR with NUR. Agreement was considered poor or relative, with at least one pair of raters, in seven of the 23 items.

Conclusion

Despite the good agreement both in the general mean and in most of the NAS items, divergences were observed, especially in the items of greater subjectivity of the instrument.

Workload; Personnel downsizing; Intensive care units; Critical care; Nursing care

Variables	Patients n(%)
Male	32(57.1)
Age (years) (Mean±SD)	58.3±17.3
SAPS 3 (n= 50) (Mean±SD)	64.2±15.1
Hospitalization type
Clinical	50(89.3)
Surgical	6(10.7)
Reason for ICU admission
Neurological	14(25)
Sepsis	12(21.4)
Respiratory	10(17.9)
Cardiological	8(14.3)
Post-operatory	6(10.7)
Others	6 (10.7)
Previous diseases
Hypertension	31(55.4)
Cancer	17(30.4)
DM	15(26.8)
CKD	11(19.6)
Stroke	8(14.3)
HF	5(8.9)
COPD	4(7.1)
CAD	3(5.4)
Median ICU hospitalization days (P25 - P75)	5(2.2 – 13.7)
Death in the ICU	12(21.4)

Researchers	Number of assessments	NAS value %	ICC (95%CI)
Reference researcher	101	79.5±20.7	0.92 (0.89 – 0.95)
Research assistant 1		79.8±23.9
Reference researcher	61	81.3±18.8	0.78 (0.64 – 0.87)
Research assistant 2		78.6±20.5
Reference researcher	88	79.6±21.2	0.75 (0.62 – 0.84)
Nursing assistants		79.7±20.4

NAS subitems	RA 1 n= 101 k (95%CI)	RA 2 n= 61 k (95%CI)	NUR n= 88 k (95%CI)
1. Monitoring and control	0.5(0.36-0.63)	0.47(0.29-0.64)	0.32(0.16-0.47)
2. Laboratory investigations: biochemical and microbiological	0.87(0.73-1)	1	0.67(0.44-0.89)
3. Drug therapy, except vasoactive drugs	1	1	1
4. Hygiene procedures	0.38(0.21-0.54)	0.18(0.04-0.41)	0.08(0.03-0.19)
5. Care with drains: all (except gastric tube)	0.82(0.68-0.95)	0.85(0.71-0.99)	0.46(0.21-0.7)
6. Mobilization and positioning	0.22(0.08-0.35)	0.43(0.22-0.63)	0.05(0.02-.019)
7. Support and care to patients and their families	0.48(0.05-0.9)	0.31(0.16-0.79)	0.11(0.03 -0.19)
8. Management and administrative tasks	0.41(0.26-0.55)	0.34(0.11-0.56)	0.27(0.08-0.45)
Ventilatory support 9. Respiratory support 10. Care with artificial airways 11. Treatment of pulmonary function	0.94(0.87-1) 1 0.91(0.83-0.99)	1 1 0.93(0.84-1)	0.63(0.44-0.81) 0.79(0.66-0.91) 0.62(0.45-0.78)
Cardiovascular support 12. Vasoactive drug therapy 13. Intravenous restitution of major fluid loss 14. Left atrium monitoring, pulmonary artery catheter 15. Cardiopulmonary resuscitation in the last 24 hours	0.96(0.9-1) 0.52(0.16-0.88) 1 1	1 0.57(0.23-0.9) 1 1	0.72(0.58-0.87) 0.05(0.01-0.22) 0.64(0.31-0.96) *
Renal support 16. Hemofiltration techniques, dialysis techniques 17 Quantitative measure of diuresis	1 1	1 1	0.78(0.62-0.93) 0.42(0.08-0.76)
Neurological support 18. Intracranial pressure monitoring	1	1	1
Metabolic support 19. Treatment of metabolic acidosis/alkalosis 20. Total parenteral nutrition 21. Enteral nutrition	0.70(0.51-0.89) 1 0.98(0.95 – 1)	0.76(0.53-0.98) 1 1	0.25(0.01-0.51) 0.82(0.51-1) 0.61(0.44-0.78)
Specific interventions 22. Specific interventions in the ICU 23. Specific interventions out of the ICU	0.72(0.58-0.85) 0.91(0.79-1)	0.67(0.47-0.83) 0.93(0.77-1)	0.36(0.16-0.55) 0.48(0.22-0.74)

Escola Paulista de Enfermagem, Universidade Federal de São Paulo R. Napoleão de Barros, 754, 04024-002 São Paulo - SP/Brasil, Tel./Fax: (55 11) 5576 4430 - São Paulo - SP - Brazil
E-mail: actapaulista@unifesp.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Corresponding author: Érica Batassini E-mail: ebatassini@hcpa.edu.br