Reliability of Cognitive Tests of ELSA-Brasil, the Brazilian Longitudinal Study of Adult Health

Batista, Juliana Alves; Giatti, Luana; Barreto, Sandhi Maria; Galery, Ana Roscoe Papini; Passos, Valéria Maria de Azeredo

doi:10.1590/S1980-57642013DN74000003

ABSTRACT

Cognitive function evaluation entails the use of neuropsychological tests, applied exclusively or in sequence. The results of these tests may be influenced by factors related to the environment, the interviewer or the interviewee. Objectives: We examined the test-retest reliability of some tests of the Brazilian version from the Consortium to Establish a Registry for Alzheimer's disease. Methods: The ELSA-Brasil is a multicentre study of civil servants (35-74 years of age) from public institutions across six Brazilian States. The same tests were applied, in different order of appearance, by the same trained and certified interviewer, with an approximate 20-day interval, to 160 adults (51% men, mean age 52 years). The Intraclass Correlation Coefficient (ICC) was used to assess the reliability of the measures; and a dispersion graph was used to examine the patterns of agreement between them. Results: We observed higher retest scores in all tests as well as a shorter test completion time for the Trail Making Test B. ICC values for each test were as following: Word List Learning Test (0.56), Word Recall (0.50), Word Recognition (0.35), Phonemic Verbal Fluency Test (VFT, 0.61), Semantic VFT (0.53) and Trail B (0.91). The Bland-Altman plot showed better correlation of executive function (VFT and Trail B) than of memory tests. Conclusions: Better performance in retest may reflect a learning effect, and suggest that retest should be repeated using alternate forms or after longer periods. In this sample of adults with high schooling level, reliability was only moderate for memory tests whereas the measurement of executive function proved more reliable.

Key words:
cognitive assessment; reliability; cohort studies.

RESUMO

A avaliação da cognição utiliza testes neuropsicológicos, aplicados isoladamente ou em sequência. Os resultados destes testes podem sofrer influências relacionadas ao ambiente, ao entrevistador ou ao entrevistado. Objetivos: Investigar a confiabilidade teste-reteste de alguns testes da versão brasileira do Consortium to Establish a Registry for Alzheimer's disease. Métodos: O ELSA-Brasil é uma coorte multicêntrica de servidores (35-74 anos) de instituições públicas de seis estados. Os mesmos testes foram aplicados, em ordens distintas, pelo mesmo entrevistador treinado e certificado, com intervalo aproximado de 20 dias, entre 160 participantes (51% homens, média de idade de 52 anos). O Coeficiente de Correlação Intraclasse (CCI) foi usado na verificação da confiabilidade das medidas, e um gráfico de dispersão evidenciou padrões de concordância entre teste e reteste. Resultados: Foram observados maiores escores em todos os retestes, assim como menor tempo para completar o Teste de Trilhas B (Trilhas B). Os valores de CCI para cada teste foram: memória imediata de palavras (0,56), evocação (0,50), reconhecimento (0,35), Teste de Fluência Verbal fonêmica (TFV, 0,61), TFV semântica (0,53) e Trilhas B (0,91). O gráfico de Bland-Altman mostrou melhor correlação dos testes de função executiva (TFV e Trilhas B) que dos testes de memória. Conclusões: O melhor desempenho nos retestes parece refletir efeito de aprendizado e sugere que o reteste seja aplicado após períodos mais longos ou com formas alternativas dos testes. Nesta população com predominância de adultos jovens e de alta escolaridade, a confiabilidade foi moderada para testes de memória e maior para testes de função executiva.

Palavras-chave:
avaliação cognitiva; confiabilidade; estudos de coortes.

Texto completo disponível apenas em PDF.

Full text available only in PDF format.

REFERENCES

¹
Lezak MD, Howieson DB, Loring DW. Neuropsychological assessment. 4 ed. New York, NY: Oxford University Press; 2004.
²
Szklo M, Nieto FJ. Quality Assurance and Control. In: Sklo M, Nieto FJ (eds). Epidemiology beyond the basics. 2nd edition. Sudbury, MA, USA: Jones and Bartlett Publishers; 2007:297-348
³
Huley SB, Martin JN, Cummings SR. Planning the measurements: accuracy and precision In: Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB (eds).Designing clinical research. 3rd edition, Philadelphia, PA, USA. Linppicott Willians and Wilkings; 2006:55-67.
⁴
Aquino EM, Barreto SM, Benseñor IM, et al. Brazilian Longitudinal Study of Adult Health (ELSA-Brasil): objectives and design. Am J Epidemiol 2012;175:315-324.
⁵
Morris JC, Heyman A, Mohs RC, et al. The Consortium to Establish a Registry for Alzheimer's Disease (CERAD): Part I. Clinical and neuropsychological assessment of Alzheimer's disease. Neurology 1989; 39:1159-1165.
⁶
Fillenbaum GG, Belle G, Morris JC, et al. CERAD (Consortium to Establish a Registry for Alzheimer's disease): The first 20 years. Alzheimer's Dement 2008;4:96-109.
⁷
Brucki SMD, Malheiros SMF, Okamoto IH, Bertolucci PHF. Dados normativos para o teste de fluência verbal categoria animais em nosso meio. Arq Neuropsiquiatr 1997;55:56-61.
⁸
Brucki SMD, Rocha MSG. Category fluency test: effects of age, gender and education on total scores, clustering and switching in Brazilian Portuguese-speaking subjects. Braz J Med Biol Res 2004;37:1771- 1777.
⁹
Foss MP, Vale FAC, Speciali JG. Influência da escolaridade na avaliação neuropsicológica de idosos. Arq Neuropsiquiatr 2005;63:119-126.
¹⁰
Passos VMA, Giatti L, Barreto SM, et al. Verbal fluency tests reliability in a Brazilian multicentric study, ELSA-Brasil. Arq Neuropsiquiatr 2011; 69:814-816.
¹¹
Dean AG, Arner TG, Sunki GG, et al. Epi Info(tm), a database and statistics program for public health professionals. Centres for Disease Control and Prevention, Atlanta, Georgia, USA; 2007.
¹²
Stata Statistical Software: Release 10.College Station, Texas: Stata Corporation; 2007.
¹³
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74.
¹⁴
Harrison JE, Buxton P, Husain M, Wise R. Short test of semantic and phonological fluency: Normal performance, validity and test-retest reliability. Brit J Clin Psychol 2000;39:181-191.
¹⁵
Lee JH, Lee KU, Lee DY, et al. Development of the Korean Version of the Consortium to Establish a Registry for Alzheimer's Disease Assessment Packet (CERAD-K): Clinical and Neuropsychological Assessment Batteries. J Gerontol 2002;57:47-53.
¹⁶
Wagner S, Helmreich I, Dahmen N, Lieb K, Tadic A. Reliability of Three Alternate Forms of the Trail Making Tests A and B. Arch Clin Neuropsychol 2011;26:314-321.
¹⁷
Bertolucci PHF, Okamoto IH, Neto JT, Ramos LR, Brucki SMD. Desempenho da população brasileira na bateria neuropsicológica do Consortium to Establish a Registry for Alzheimer's disease (CERAD). Rev Psiq Clín 1998;25:80-83.
¹⁸
Salthouse TA. Selective review of cognitive aging. J Int Neuropsychol Soc 2010;16:754-760. 19, Charchat-Fichman H, Caramelli P, Sameshima K, Nitrini R. Declínio da capacidade cognitiva durante o envelhecimento. Rev Bras Psiq 2005; 27:79-82.
¹⁹
, Charchat-Fichman H, Caramelli P, Sameshima K, Nitrini R. Declínio da capacidade cognitiva durante o envelhecimento. Rev Bras Psiq 2005; 27:79-82.
²⁰
Christofoletti G, Oliani MM, Stella F, Gobbi S, Gobbi LTB. The influence of schooling on cognitive screening test in the elderly. Dement Neuropsychol 2007;1:46-51.

Publication Dates

Publication in this collection
Dec 2013

History

Received
29 May 2013
Accepted
15 Aug 2013

This is an open-access article distributed under the terms of the Creative Commons Attribution License

[1] ¹
Lezak MD, Howieson DB, Loring DW. Neuropsychological assessment. 4 ed. New York, NY: Oxford University Press; 2004.

[2] ²
Szklo M, Nieto FJ. Quality Assurance and Control. In: Sklo M, Nieto FJ (eds). Epidemiology beyond the basics. 2nd edition. Sudbury, MA, USA: Jones and Bartlett Publishers; 2007:297-348

[3] ³
Huley SB, Martin JN, Cummings SR. Planning the measurements: accuracy and precision In: Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB (eds).Designing clinical research. 3rd edition, Philadelphia, PA, USA. Linppicott Willians and Wilkings; 2006:55-67.

[4] ⁴
Aquino EM, Barreto SM, Benseñor IM, et al. Brazilian Longitudinal Study of Adult Health (ELSA-Brasil): objectives and design. Am J Epidemiol 2012;175:315-324.

[5] ⁵
Morris JC, Heyman A, Mohs RC, et al. The Consortium to Establish a Registry for Alzheimer's Disease (CERAD): Part I. Clinical and neuropsychological assessment of Alzheimer's disease. Neurology 1989; 39:1159-1165.

[6] ⁶
Fillenbaum GG, Belle G, Morris JC, et al. CERAD (Consortium to Establish a Registry for Alzheimer's disease): The first 20 years. Alzheimer's Dement 2008;4:96-109.

[7] ⁷
Brucki SMD, Malheiros SMF, Okamoto IH, Bertolucci PHF. Dados normativos para o teste de fluência verbal categoria animais em nosso meio. Arq Neuropsiquiatr 1997;55:56-61.

[8] ⁸
Brucki SMD, Rocha MSG. Category fluency test: effects of age, gender and education on total scores, clustering and switching in Brazilian Portuguese-speaking subjects. Braz J Med Biol Res 2004;37:1771- 1777.

[9] ⁹
Foss MP, Vale FAC, Speciali JG. Influência da escolaridade na avaliação neuropsicológica de idosos. Arq Neuropsiquiatr 2005;63:119-126.

[10] ¹⁰
Passos VMA, Giatti L, Barreto SM, et al. Verbal fluency tests reliability in a Brazilian multicentric study, ELSA-Brasil. Arq Neuropsiquiatr 2011; 69:814-816.

[11] ¹¹
Dean AG, Arner TG, Sunki GG, et al. Epi Info(tm), a database and statistics program for public health professionals. Centres for Disease Control and Prevention, Atlanta, Georgia, USA; 2007.

[12] ¹²
Stata Statistical Software: Release 10.College Station, Texas: Stata Corporation; 2007.

[13] ¹³
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74.

[14] ¹⁴
Harrison JE, Buxton P, Husain M, Wise R. Short test of semantic and phonological fluency: Normal performance, validity and test-retest reliability. Brit J Clin Psychol 2000;39:181-191.

[15] ¹⁵
Lee JH, Lee KU, Lee DY, et al. Development of the Korean Version of the Consortium to Establish a Registry for Alzheimer's Disease Assessment Packet (CERAD-K): Clinical and Neuropsychological Assessment Batteries. J Gerontol 2002;57:47-53.

[16] ¹⁶
Wagner S, Helmreich I, Dahmen N, Lieb K, Tadic A. Reliability of Three Alternate Forms of the Trail Making Tests A and B. Arch Clin Neuropsychol 2011;26:314-321.

[17] ¹⁷
Bertolucci PHF, Okamoto IH, Neto JT, Ramos LR, Brucki SMD. Desempenho da população brasileira na bateria neuropsicológica do Consortium to Establish a Registry for Alzheimer's disease (CERAD). Rev Psiq Clín 1998;25:80-83.

[18] ¹⁸
Salthouse TA. Selective review of cognitive aging. J Int Neuropsychol Soc 2010;16:754-760. 19, Charchat-Fichman H, Caramelli P, Sameshima K, Nitrini R. Declínio da capacidade cognitiva durante o envelhecimento. Rev Bras Psiq 2005; 27:79-82.

[19] ¹⁹
, Charchat-Fichman H, Caramelli P, Sameshima K, Nitrini R. Declínio da capacidade cognitiva durante o envelhecimento. Rev Bras Psiq 2005; 27:79-82.

[20] ²⁰
Christofoletti G, Oliani MM, Stella F, Gobbi S, Gobbi LTB. The influence of schooling on cognitive screening test in the elderly. Dement Neuropsychol 2007;1:46-51.

Brasil