Is the Mini-Mental State Examination the best cognitive screening test for less educated people?

O mini-exame de estado mental é o melhor teste de rastreamento cognitivo para indivíduos com menor escolaridade?

ABSTRACT

Mini-Mental State Examination (MMSE) results are strongly influenced by educational level. The Brief Cognitive Screening Battery (BCSB) is an alternative assessment tool that provides more accurate results in individuals with less education.

Objective:

Our aim was to compare the MMSE and BCSB as screening tests.

Methods:

The MMSE and BCSB were assessed in 112 participants by two evaluators blind to the other test's result. Participants were classified according to their level of education. The influence of education level was analyzed using the Kruskal-Wallis and multiple comparison tests.

Results:

Scores of the MMSE (p < 0.0001) and the clock-drawing test (p < 0.0001) were influenced by education level but the delayed recall test score was not (p = 0.0804). The verbal fluency test (p = 0.00035) was influenced only by higher educational levels. It took three minutes less to apply the MMSE than to apply the BCSB (p < 0.0001).

Conclusions:

These findings suggest that the delayed recall test and the verbal fluency test of the BCSB are better than the MMSE and clock-drawing test as tools for evaluating cognition in people with limited education.

Keywords:
Dementia; cognition; cognition disorders; memory

Resumo

Os resultados do Mini-Exame de Estado Mental (MEEM) são consideravelmente influenciados pelo nível de escolaridade. A Bateria Breve de Rastreio Cognitivo (BBRC) é uma ferramenta de rastreamento cognitivo alternativa que fornece resultados mais acurados em indivíduos com menor nível de escolaridade.

Objetivo:

Comparar o MEEM e a BBRC como testes de rastreamento cognitivo.

Métodos:

112 participantes foram submetidos ao MEEM e a BBRC por 2 avaliadores cegos para o resultados do outro teste. Os participantes foram classificados de acordo o nível de escolaridade. A influência do nível de escolaridade foi analisada utilizando o teste de Kruskal-Wallis e o teste de comparações múltiplas.

Resultados:

As pontuações do MEEM (p < 0,0001) e do teste do desenho do relógio (p < 0,0001) foram influenciadas pelo nível de escolaridade, porém o teste de memória tardia não sofreu influência do nível de escolaridade (p = 0,0804). O teste de fluência verbal (p = 0,00035) foi influenciado apenas pelos níveis educacionais mais altos. A aplicação do MEEM levou 3 minutos a menos que a da BBRC (p < 0,0001).

Conclusão:

Os achados desse estudo sugerem que o teste de memória tardia e o teste de fluência verbal da BBRC são mais adequados que o MEEM e o teste do desenho do relógio como ferramentas para avaliação cognitiva em populações com menor nível educacional.

Palavras-chave:
Demência; cognição; transtornos cognitivos; memória

Mild cognitive impairment (MCI) is a condition that lies between the normal aging-associated cognitive decline and dementia. It involves a degree of cognitive decline that is not sufficient to impair an individual's daily living activities. It is estimated that the prevalence of MCI is 1020% in individuals aged 65 years or older and that it increases with age¹.

Within this context, various cognitive tests are available for the screening and diagnosis of MCI and early-stage dementia. The Mini-Mental State Examination (MMSE) is the most commonly used test². The MMSE has a sensitivity of 62.7% and a specificity of 63.3% when used to detect MCI in healthy individuals³. It is important, however, to take into account age, gender and education in the interpretation of MMSE scores, especially in populations with lower educational levels^4,5.

Another less commonly used instrument is the Brief Cognitive Screening Battery (BCSB). The BCSB consists of the following tests: identification and naming of 10 common objects from drawings (sensitivity: 50%; specificity: 96.66%); incidental recall (sensitivity: 90%; specificity: 83.33%), immediate recall and learning of these images; the clock-drawing test (sensitivity: 90%; specificity: 83.33%); semantic verbal fluency test based on naming animals (sensitivity: 96.66%; specificity: 93.33%); delayed recall (sensitivity: 93.33%; specificity: 96.66%) and recognition of the drawings used in the first test when presented among other drawings (sensitivity: 93.33%; specificity: 90%)^6,7.

Scores on the MMSE, the verbal fluency test and the clock-drawing test differ between literate and illiterate individuals without dementia. On the other hand, this difference is not observed on the BCSB delayed recall test scores⁸. The BCSB is a highly accurate method for the diagnosis of Alzheimer's disease in both illiterate and literate individuals⁹.

We hypothesized that education influences the performance of individuals more significantly on the MMSE than on the BCSB.

For the purpose of testing this hypothesis, the major objective of this study was to compare the influence of educational levels on the MMSE and BCSB scores.

METHODS

This was a cross-sectional observational study of 112 healthy individuals aged between 60 and 80 years, who attended outpatient clinics of the Hospital de Clínicas of the Federal University of Parana, located in Curitiba, Brazil. Participants were randomized by a simple randomization technique. Ten patients were chosen per day from a list of patients who attended the outpatient clinics that day. These selected patients were addressed, then were included or not in the study according to the inclusion and exclusion criteria and their acceptance of the consent terms. All the individuals performed both tests and each test was conducted by a different evaluator blind to the other test's results (the MMSE was always conducted by evaluator 1 and the BCSB was always conducted by evaluator 2). We chose this design to minimize measurement bias. Patients attending neurology and psychiatry outpatient clinics were excluded, as were those using barbiturates, benzodiazepines, anticholinesterases, antipsychotics and thyroid hormones, because these criteria could predict the presence of major cognitive declines or neurologic or psychiatric comorbidities that could impact cognitive functioning¹⁰. Therefore, these patients were excluded, to minimize the risk of including cognitively impaired individuals. However, we did not perform any diagnostic test for the exclusions, because we aimed to mimic the reality of a primary care context in which accurate diagnostic testing might not be available. The study was approved by the Human Research Ethics Committee of the Hospital de Clínicas of the Federal University of Parana.

Participants were categorized on the basis of their education (group 0: illiterate; group 1: 1–4 years of education; group 2: 5-8 years of education; group 3: more than eight years of education). The MMSE was administered to all participants according to the procedure described by Brucki et al.¹¹ and the BCSB was assessed as described by Vitiello et al.⁶.

Scores for the various components of the BCSB were calculated separately and only the scores for the delayed recall test, clock-drawing test and verbal fluency test were used in the analysis. These tests were chosen from the BCSB because they evaluate important cognitive domains impaired in MCI and dementia, such as memory in the delayed recall test, executive function and language in the verbal fluency test, and visuospatial skills in the clock-drawing test. They were chosen instead of the other tests because they had the highest sensitivity and specificity in the domain they evaluate. If we included all the tests in the analysis, the results would have been redundant and less reliable^12,13. However, the time taken to apply the whole BCSB was used in the application time analysis.

For the analysis we used cut-off points proposed for the Brazilian population. The cut-off points for the MMSE were as follows: 20 for group 0; 25 for group 1; 26 for group 2; 28 for group 3¹¹. The cut-off points for the BCSB tests analyzed in our study were: delayed recall test: 6; verbal fluency test: 13⁷. We did not analyze performance on the clock-drawing test based on cut-off points because participants who refused to perform that test were excluded from its analysis.

Statistical analysis was carried out using R Statistical Software, 2016. Test administration times were compared using the Mann-Whitney test. The effects of educational level were assessed using the Kruskal-Wallis test and pairwise multiple comparisons.

RESULTS

Table 1 shows the educational profile of the sample. Eighty participants (71%) were women. Median assessment times for the MMSE and BCSB were respectively 5 and 8 minutes (p < 0.0001).

Table 1
Distribution of participants by level of education.

Groups	Level of education
	Total	Relative (%)
Group 0	10	9
Group 1	44	39
Group 2	29	26
Group 3	29	26
Total	112	100

Table 2 presents a group comparison of median scores on the MMSE, delayed test, clock-drawing test and verbal fluency test. Regarding the MMSE scores, all the pairwise comparisons between groups were significant (p < 0.0001), except for the comparison between groups 1 and 2 (p = 0.1189).

Table 2
Median scores for the MMSE and for the BCSB selected tests for analysis (delayed recall test, clock-drawing test and verbal fluency test).

Test	Median score
	Group 0	Group 1	Group 2	Group 3	p-value
MMSE	21	23	23	26	< 0.0001
Delayed recall test	7	8	7	8	0.0804
Clock-drawing test	4	8	9	9	< 0.0001
Verbal fluency test	12	14	12	18	0.00035

• MMSE: Mini-mental state examination; BCSB: Brief cognitive battery.

Similarly, pairwise comparisons of scores on the clock-drawing test were also significant (p < 0.0001) except for the comparison of groups 1 and 2 (p = 0.082). Fifteen participants (13%) refused to draw the clock and were therefore excluded from this analysis. The distribution of refusals by education group was as follows, group 0: n = 6 (60%); group 1: n = 6 (14%); group 2: n = 2 (7%); group 3: n = 1 (3%).

Pairwise comparison revealed that the performance of group 3 on the verbal fluency test was different from that of all the other groups: vs. group 0: p = 0.0032; vs. group 1: p = 0.0003; vs. group 2: p = 0.0001. All the other groups had similar verbal fluency test scores: group 0 vs. group 1: p = 0.5596; group 0 vs. group 2: p = 0.9618; group 1 vs. group 2: p = 0.435.

Delayed recall test scores were similar in all groups (p = 0.0804).

DISCUSSION

The median MMSE scores in our sample are comparable to those reported by Brucki et al.¹¹: illiterates, MMSE = 20; 1–4 years of education, MMSE = 25; 5–8 years of education, MMSE = 26.5; over eight years of education, MMSE = 28. However, there is some debate about which cut-off points should be used for the MMSE. Brucki et al.¹¹ suggested that the cut-off points should be analyzed for each individual separately, whereas Bertolucci et al.¹⁴ suggested values of 13 (illiterates), 18 (1–8 years of education) and 26 (more than eight years of education), and Almeida et al.¹⁵ suggested 20 as a cut-off for illiterate individuals. In a study that compared illiterate and literate individuals, the median MMSE scores were 21 and 26.5 for illiterate and literate participants respectively⁸. In summary, the analysis of MMSE scores has substantially changed over time.

Education significantly influences MMSE scores and has been reported to be the most important determinant of variance in performance on the test^13,16. Therefore, most of the Brazilian studies on the MMSE performed between 1998 and 2013 used educational criteria to define cut-off points¹⁷. Our results also suggest that education is a decisive element in MMSE performance. Thus, the lack of standardization for the MMSE, shown by the significant variability of the scores among different educational levels, impairs its validity¹⁷.

As discussed previously, the weaknesses of the MMSE have gradually become more apparent. Therefore, the urge to provide an assessment of cognition better than the MMSE, which is valid for the entire Brazilian population, became even more evident. The BCSB has become the more appropriate option for that purpose. The tests that make up the BCSB have been studied since 1994. In fact, the accuracy of the selected tests for the BCSB is high, and comparable to that of the MMSE⁷.

Our study suggests that education more significantly influences the MMSE than the BCSB in our population. Our sample showed that the delayed recall test scores did not vary with levels of education. Nitrini et al.⁷ reported that illiterate and literate participants, and participants with little education and standard education, obtained similar scores on the BCSB delayed recall test. They concluded that, unlike the MMSE, the delayed recall test is not educationally biased⁸.

The effect of educational level on the verbal fluency test that we observed was due to the performance of the group with more than eight years of education. This is consistent with another study, which reported that individuals with more than eight years of education spoke approximately six more words than individuals with less education¹⁸. Another study also found a significant difference between the verbal fluency scores of illiterate and literate participants⁸. In our sample, participants with more than eight years of schooling spoke between four and six more words than participants in the other groups.

In our sample, educational level influenced performance on the clock-drawing test. Previous studies have also reported that individuals with more education perform better than those with less education^19,20. It should be noted, however, that our analysis of the clock-drawing test was impaired by the high rate of refusals to perform the test, especially in the illiterate group. In most cases the refusal was due to lack of knowledge and embarrassment about having their drawings evaluated. This indicates a limitation for the clinical use of the clock-drawing test in individuals with less education.

We found that the MMSE took three minutes less to assess than the BCSB. The median assessment time for the BCSB in our sample, eight minutes, is similar to the median times reported in other studies⁹. The three-minute difference in application time is not a barrier to the clinical use of the BCSB.

While the BCSB takes longer to apply, tests that minimize educational bias, such as the delayed recall test of the BCSB, are more appropriate to the Brazilian context. The heterogeneity of the Brazilian population makes it difficult to standardize scores on tests that are influenced by the educational level.

Besides the fact that our sample was selected from outpatient clinics from a tertiary health service, the proportion of each education group was compatible to the profile of users of the Brazilian public health system, as stated by Ribeiro et al.²¹: up to three years of education (groups 0 and 1): 41.9%; four to seven years (group 2): 24%; over eight years (group 3): 34.1%. The small number of illiterates in our sample made the observations in this group less reliable, but that number can be justified by the percentage of illiterate people in the Brazilian population, which was 8% in 2015 according to data from the Brazilian National Institute of Geography and Statistics²². Therefore, our sample possibly reflects the reality of the Brazilian population.

The main weaknesses of our study are the small number of illiterates and the sample selected from a tertiary hospital. As stated before, both weaknesses could impair external validity but the proportions of participants in each group were consistent with some data already published. Therefore, we suggest, although we cannot confirm, that our results could be representative of the Brazilian population. Also, we did not carry out any additional testing for illiterates who refused to perform the clock-drawing test, which significantly impaired that analysis.

Further studies are necessary to assess the need for new cut-off points for the clock-drawing and verbal fluency tests and to develop more appropriate tools to assess visuospatial functions in illiterate individuals.

References

• ¹
  Langa KM, Levine DA. The diagnosis and management of mild cognitive impairment: a clinical review. JAMA. 2014 Dec;312(23):2551-61. https://doi.org/10.1001/jama.2014.13806
  » https://doi.org/10.1001/jama.2014.13806
• ²
  Arevalo-Rodriguez I, Smailagic N, Roqué I Figuls M, Ciapponi A, Sanchez-Perez E, Giannakou A, et al. Mini-Mental State Examination (MMSE) for the detection of Alzheimer's disease and other dementias in people with mild cognitive impairment (MCI). Cochrane Database Syst Rev. 2015 Mar;(3):CD010783.
• ³
  Mitchell AJ. A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res. 2009 Jan;43(4):411-31. https://doi.org/10.1016/j.jpsychires.2008.04.014
  » https://doi.org/10.1016/j.jpsychires.2008.04.014
• ⁴
  Grigoletto F, Zappalà G, Anderson DW, Lebowitz BD. Norms for the Mini-Mental State Examination in a healthy population. Neurology. 1999 Jul;53(2):315-20. https://doi.org/10.1212/WNL.53.2.315
  » https://doi.org/10.1212/WNL.53.2.315
• ⁵
  Velayudhan L, Ryu SH, Raczek M, Philpot M, Lindesay J, Critchfield M, et al. Review of brief cognitive tests for patients with suspected dementia. Int Psychogeriatr. 2014 Aug;26(8):1247-62. https://doi.org/10.1017/S1041610214000416
  » https://doi.org/10.1017/S1041610214000416
• ⁶
  Vitiello AP, Ciríaco JG, Takahashi DY, Nitrini R, Caramelli P. [Brief cognitive evaluation of patients attended in a general neurological outpatient clinic]. Arq Neuropsiquiatr. 2007 Jun;65(2A):299-303. Portuguese. https://doi.org/10.1590/S0004-282X2007000200021
  » https://doi.org/10.1590/S0004-282X2007000200021
• ⁷
  Nitrini R, Lefèvre BH, Mathias SC, Caramelli P, Carrilho PE, Sauaia N, et al. [Neuropsychological tests of simple application for diagnosing dementia]. Arq Neuropsiquiatr. 1994 Dec;52(4):457-65. Portuguese. https://doi.org/10.1590/S0004-282X1994000400001
  » https://doi.org/10.1590/S0004-282X1994000400001
• ⁸
  Nitrini R, Caramelli P, Herrera Júnior E, Porto CS, Charchat-Fichman H, Carthery MT, et al. Performance of illiterate and literate nondemented elderly subjects in two tests of long-term memory. J Int Neuropsychol Soc. 2004 Jul;10(4):634-8. https://doi.org/10.1017/S1355617704104062
  » https://doi.org/10.1017/S1355617704104062
• ⁹
  Nitrini R, Caramelli P, Porto CS, Charchat-Fichman H, Formigoni AP, Carthery-Goulart MT, et al. Brief cognitive battery in the diagnosis of mild Alzheimer's disease in subjects with medium and high levels of education. Dement Neuropsychol. 2007 Jan-Mar;1(1):32-6. https://doi.org/10.1590/S1980-57642008DN10100006
  » https://doi.org/10.1590/S1980-57642008DN10100006
• ¹⁰
  Molinuevo JL, Rabin LA, Amariglio R, Buckley R, Dubois B, Ellis KA, et al. Implementation of subjective cognitive decline criteria in research studies. Alzheimers Dement. 2017 Mar;13(3):296-311. https://doi.org/10.1016/j.jalz.2016.09.012
  » https://doi.org/10.1016/j.jalz.2016.09.012
• ¹¹
  Brucki SM, Nitrini R, Caramelli P, Bertolucci PH, Okamoto IH. Sugestões para o uso do mini-exame do estado mental no Brasil. Arq Neuropsiquiatr. 2003 Sep;61(3B):777-81. https://doi.org/10.1590/S0004-282X2003000500014
  » https://doi.org/10.1590/S0004-282X2003000500014
• ¹²
  Petersen RC, Aisen PS, Beckett LA, Donohue MC, Gamst AC, Harvey DJ, et al. Alzheimer's Disease Neuroimaging Initiative (ADNI): clinical characterization. Neurology. 2010 Jan;74(3):201-9. https://doi.org/10.1212/WNL.0b013e3181cb3e25
  » https://doi.org/10.1212/WNL.0b013e3181cb3e25
• ¹³
  Fichman-Charchat H, Miranda CV, Fernandes CS, Mograbi D, Oliveira RM, Novaes R, et al. Brief Cognitive Screening Battery (BCSB) is a very useful tool for diagnosis of probable mild Alzheimer's disease in a geriatric clinic. Arq Neuropsiquiatr. 2016 Feb;74(2):149-54. https://doi.org/10.1590/0004-282X20150202
  » https://doi.org/10.1590/0004-282X20150202
• ¹⁴
  Bertolluci PH, Brucki SM, Campacci SR, Juliano Y. [The Mini-Mental State Examination in a general population: impact of educational status]. Arq Neuropsiquiatr. 1994 Mar;52(1):1-7. Portuguese. https://doi.org/10.1590/S0004-282X1994000100001
  » https://doi.org/10.1590/S0004-282X1994000100001
• ¹⁵
  Almeida OP. Mini mental state examination and the diagnosis of dementia in Brazil. Arq Neuropsiquiatr. 1998 Sep;56(3B):605-12. https://doi.org/10.1590/S0004-282X1998000400014
  » https://doi.org/10.1590/S0004-282X1998000400014
• ¹⁶
  Lourenço RA, Veras RP. Mini-Mental State Examination: psychometric characteristics in elderly outpatients. Rev Saude Publica. 2006 Aug;40(4):712-9. https://doi.org/10.1590/S0034-89102006000500023
  » https://doi.org/10.1590/S0034-89102006000500023
• ¹⁷
  Melo DM, Barbosa AJ. [Use of the Mini-Mental State Examination in research on the elderly in Brazil: a systematic review]. Cien Saude Colet. 2015 Dec;20(12):3865-76. Portuguese. https://doi.org/10.1590/1413-812320152012.06032015
  » https://doi.org/10.1590/1413-812320152012.06032015
• ¹⁸
  Passos VMA, Giatti L, Bensenor I, Tiemeier H, Ikram MA, Figueiredo RC, et al. Education plays a greater role than age in cognitive test performance among participants of the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil). BMC Neurol. 2015 Oct;15(1):191. https://doi.org/10.1186/s12883-015-0454-6
  » https://doi.org/10.1186/s12883-015-0454-6
• ¹⁹
  Fabricio AT, Aprahamian I, Sanches Yassuda M. Qualitative analysis of the Clock Drawing Test by educational level and cognitive profile. Arq Neuropsiquiatr. 2014 Apr;72(4):289-95. https://doi.org/10.1590/0004-282X20140004
  » https://doi.org/10.1590/0004-282X20140004
• ²⁰
  Gunten A, Ostos-Wiechetek M, Brull J, Vaudaux-Pisquem I, Cattin S, Duc R. Clock-drawing test performance in the normal elderly and its dependence on age and education. Eur Neurol. 2008;60(2):73-8. https://doi.org/10.1159/000131895
  » https://doi.org/10.1159/000131895
• ²¹
  Ribeiro MC, Barata RB, Almeida MF, Silva ZP. Perfil sociodemográfico e padrão de utilização de serviços de saúde para usuários e não-usuários do SUS - PNAD 2003. Cien Saude Colet. 2006;11(4):1011-22. https://doi.org/10.1590/S1413-81232006000400022
  » https://doi.org/10.1590/S1413-81232006000400022
• ²²
  Instituto Brasileiro de Geografia e Estatística – IBGE. Pesquisa nacional por amostra de domicílios: síntese de indicadores. Rio de Janeiro: Instituto Brasileiro de Geografia e Estatística – IBGE; 2016.

Publication Dates

• Publication in this collection
  00 00 2019
• Date of issue
  May 2019

History

• Received
  02 Oct 2018
• Reviewed
  02 Jan 2019
• Accepted
  01 Feb 2019