Validation of the Brazilian Portuguese version of the Rapid Eye Movement Sleep Behavior Disorder Screening Questionnaire (RBDSQ-BR)

ABSTRACT Introduction: A diagnosis of rapid eye movement sleep behavior disorder (RBD) currently requires confirmation with polysomnography (PSG). However, PSG may not be sufficiently available. In these situations, a clinical diagnostic measure might be useful. Objective: To validate the Brazilian Portuguese version of RBD screening questionnaire (RBDSQ) for patients with Parkinson’s disease (PD). Methods: Using detailed clinical interviews and PSG analysis (diagnostic gold standard), a convenience sample of 69 subjects was divided into the following subgroups: patients with PD and RBD (PD+RBD; n=50) and patients with PD alone (PD-RBD; n=19). Results: RBDSQ-BR showed adequate internal consistency (Cronbach’s α=0.809) and, except for item 8, adequate item-test correlation. The retest performed in a second sample (n=13, consecutive) showed high agreement for total score (intraclass correlation coefficient, ICC=0.863) and acceptable agreement for items 2, 3, 6.2, 6.3, 7, and 8 (K>0.60). The receiver operating characteristic (ROC) curve analysis had an area under the curve (AUC) of 0.728. A cut-off score of 4 enabled the correct diagnosis of 76.8% subjects and provided the best balance between sensitivity (84%) and specificity (57.9%), with a 2.0 likelihood ratio of a positive result (LR+) and a 0.3 likelihood ratio of a negative result (LR-). Items 2 and 6.2 had 84.2% specificity and 3.2 LR+. Combined items 1+2+6.2, 2+6.1, and 6.1+6.2 increased the specificity to 94.7%, with LR+ ranging from 6.1 to 7.6. Conclusions: RBDSQ-BR is a reliable instrument, which may be useful for RBD diagnosis of Brazilian patients with PD. The instrument is also valid and may help in a better selection of cases for a more detailed clinical evaluation or even PSG analysis.


INTRODUCTION
Rapid eye movement (REM) sleep behavior disorder (RBD) is characterized by the loss of atonia during REM sleep, due to excessive motor activity or even dream performance 1,2 . The prevalence of RBD among the general population is approximately 1.8% 3 , whereas it may reach 55.7% in patients with Parkinson disease (PD) 4 .
A diagnosis of RBD currently requires confirmation with polysomnography (PSG) 2,5 . However, PSG may not be sufficiently available, especially in the public health system 6 . In these situations, a clinical diagnostic measure might be useful 7,8,9,10,11 . The RBD screening questionnaire (RBDSQ) is a 13-item, self-administered, dichotomous instrument, which also allows for the input from the patient's companion. It was originally published in German and English 7 and subsequently validated in Japan 12 , China 13 , South Korea 14 , Turkey 15 , and Italy 16 . Its use has already been validated for patients with PD 17,18 .
The objective of the present study was to validate the Brazilian Portuguese version of the RBDSQ (RBDSQ-BR) for patients with PD.

METHOD
The current study had a cross-sectional, observational design. The sample of 82 patients with PD was selected from a cohort of subjects participating in a follow-up study in outpatients at the University Hospital of Ribeirão Preto School of Medicine of Universidade de São Paulo. The following inclusion criteria used were: of Brazilian ethnicity, ≥18 years of age, diagnosed with PD according to the United Kingdom Brain Bank criteria 19 (allowing the presence of family history of PD). They also needed to be available for PSG and to provide verbal and written consent to participate in the study. The inclusion periods were from February 2010 to November 2011 and from July to September 2014. The study was approved by the Research Ethics Committee under protocol numbers 2213/2009 and 13410/2009.

Questionnaire translation and cultural adaptation
The original author of the RBDSQ authorized the current study, which was performed according to previously published protocols 20,21 . The English questionnaire was initially translated by three bilingual Brazilian natives separately 84,2% e RV+ 3,2. Itens combinados 1+2+6,2, 2+6,1 e 6,1+6,2 aumentaram a especificidade para 94,7%, com RV+ variando de 6,1 até 7,6. Conclusões: O QT-TCSREM-BR é um instrumento confiável que pode ser útil para o diagnóstico do TCSREM em pacientes com DP no Brasil. O instrumento também é válido e pode auxiliar numa melhor seleção de casos a serem submetidos a uma avaliação mais detalhada ou até mesmo a uma análise de PSG. (i.e., one physician experienced in sleep disorders, one physician experienced in movement disorders, and one engineer). The three versions were then analyzed by a committee comprising bilingual physicians (three physicians experienced in movement disorders, one physician experienced in sleep disorders, and one physician experienced in both disorders, who were informed about the objective of the study, but were not involved in the translation) who prepared a single version. This version was administered to 10 subjects (patients or their companions) as a pre-test. The committee then conducted a revision of this version (RBDSQ-BR, see Online resource) toward identifying any inconsistencies.
Subsequently, independent back-translation was performed by two native English teachers (school teachers). These versions were reviewed by a committee of three professionals (one experienced in movement disorders, one experienced in sleep disorders, and one experienced in both), preparing a single, back-translated version that was sent to the original author along with its Brazilian version for review and consent.

Clinical evaluation and polysomnography
Of the 82 patients, 69 patients with clinical and PSG data were selected for analysis. The remaining 13 consecutive patients only had the RBDSQ-BR results available for analysis.
The evaluation started with patients filling the RBDSQ-BR with the help of their companion, immediately after the PSG or on the following day. The patient was then subjected to a detailed sleep evaluation by an experienced physician who used the diagnostic criteria from the 3 rd edition of the International Classification of Sleep Disorders 5 and was blinded to the results from the questionnaire and PSG. This detailed evaluation occurred within 30 days of the PSG. Evaluation of PD was performed by another experienced physician who was blinded to the questionnaire, sleep evaluation, and PSG results, using the Unified Parkinson's disease rating scale 22 , the Hoehn and Yahr Staging Scale (HY) 23 , and the Schwab & England Functional Scale (S&E) 24 . All evaluations were performed with the patients in the on state.
PSG was performed using a digital polygraph (Biologic Sleepscan VISION PSG, Natus Bio-logic Systems Inc., San Carlos, CA) using an extended 10-20 system electroencephalogram (EEG), electro-oculogram, surface electromyogram (chin, masseter muscles, finger extensors, and tibialis anterior muscle), nasal-cannula pressure transducer, thermocouple nasal/oral airflow sensor chest and abdominal respiratory inductive plethysmography band transducers, peripheral oximetry, electrocardiogram (ECG), snore and body position sensors, and synched audio/video. The analysis was performed by two experienced physicians blinded to the RBDSQ-BR results. All of the technical parameters used were performed in accordance with the AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology, and Technical Specification 25 .

Statistical analysis
Categorical variables were assessed using Fisher's exact test, while quantitative variables were evaluated using Student's t-tests or Mann-Whitney tests to compare the groups of patients with PD, with (PD+RBD), and without (PD-RBD) RBD. Normality assessment was performed using the Shapiro-Wilk test 26 .
RBDSQ-BR reliability was assessed by determining both the temporal stability (re-test) and homogeneity (internal consistency) using Spearman's rank correlation coefficient (p) for the re-test (21 days of interval) and the intraclass correlation coefficient (ICC) for the total score (type III, twofactor, mixed analysis of variance [ANOVA]) 27 , and Cohen's kappa coefficient (K) for each item. Item-test correlation was used to estimate the coefficient discrimination of the items, accepting values >0.3. Internal consistency was determined using the Cronbach's α reliability estimate 27,28,29,30 .
The validity of the RBDSQ-BR was assessed using receiver operating characteristic (ROC) curve and area under the ROC curve (AUC) analysis, with confidence intervals determined using the exact binomial method 31 . Fisher' s exact test was used to evaluate each item. Sensitivity, specificity, positive (PPV) and negative (NPV) predictive values, accuracy, and the likelihood ratios of positive (LR+) and negative (LR-) results were calculated for each cut-off score and item. Pre-test probability of RBD was 72.5% (50 PD+RBD subjects/69 total subjects) 32,33 .
P<0.05 were considered significant, with test power >80%. Microsoft Office Excel ® , IBM SPSS Statistics 19, and R 3.1.0 software were used for statistical analysis. Table 1 outlines the clinical data of the 69 patients. Only the presence of companions during evaluation and dose-equivalent levodopa levels, which were higher in the PD+RBD group, significantly differed between the groups.

DISCUSSION
This study determined the diagnostic value of the Brazilian Portuguese version of the RBDSQ to validate its use for patients with PD. All participants were subjected to the gold standard of RBD diagnosis (clinical interview+PSG) 5 . The original RBDSQ study 7 and other studies that have sought to assess the diagnostic value of the questionnaire 12,13,14,15,16,17,18 have also used this gold standard to evaluate participants (except healthy controls).
The sample consisted of 50 participants in the PD+RBD group and 19 participants in the PD-RBD group. As in a previous study conducted by Stiasny-Kolster et al. 18 , the sample had a slight predominance of elderly men, although they were slightly younger (i.e., 68 years versus 60.4±12 years, respectively). Although the disease duration was similar (approximately 8.5 years), PD was less advanced in the current study (i.e., 2.0 versus 3.0) according to the HY staging.
The original study by Stiasny-Kolster et al. 7 already had adequate internal consistency (0.885 Cronbach's α), which was also shown in other validation studies conducted in Japan 12 , China 13 , South Korea 14 , Turkey 15 , and Italy 16 . The first study, which focused on patients with PD in Japan, found an adequate internal consistency (0.73 Cronbach's α) 17 . The item-test correlation of all items in the original study 7 was satisfactory (>0.3); however, the South Korean and Italian studies reported inadequate item-test correlations for item 10 14,16 . Our study showed low discriminatory power for item 8. This item indicates an awakening while dreaming due to the motor activity reported in item 7 (see Online resource). If this relationship is not recognized, it can be assumed that item 8 is about remembering the dreams that occurred the night before. Considering the low schooling (Table 1), a misinterpretation could explain the low discriminatory power of this item in our sample.
Agreement analysis of the re-test total score was reported in some studies 12,13,14,15 and proved satisfactory, albeit with variable agreement between the instrument items. Miyamoto et al. 12  Ac: accuracy; 95%CI: 95% confidence interval; E: specificity; LR+: likelihood ratio of a positive result; LR-: likelihood ratio of a negative result; NPV: negative predictive value; PPV: positive predictive value; REM: rapid eye movement; S: sensitivity. a Answer "yes" in the combined items; b pre-test probability: 72.5% (50/69=0.725); c Fisher's exact test. *Significant (p<0.05). **LR+ whose confidence intervals excluded the value 1.  1, 2, 5, and 6.1 (K>0.60). Conversely, the South Korean study 14 only reported this for items 7 and 10. Thus far, no study had focused on a population of patients with PD for assessing the RBDSQ agreement of either its total score or its items. In our study, a cut-off score of 4 enabled the correct diagnosis of 76.8% subjects and provided the best balance between sensitivity (84%) and specificity (57.9%), with 2.0 LR+ and 0.3 LR-. A cut-off score of 3 also correctly diagnosed 76.8% individuals and provided increased sensitivity (90.0%) with 0.2 LR-(an approximately 5-fold reduction in the probability of having RBD), whereas a cut-off score of 7 provided increased specificity (78.9%), with a correct diagnosis of 62.3% subjects and 2.7 LR+. Thus, total scores of <3 in our study might be useful to exclude RBD (LR-from 0.5 to 0.2 may generate small-to-moderate changes in post-test probability) 33 . The separate analysis of each item in our study enabled us to increase its specificity. Both items 2 and 6.2 had a specificity of 84.2%, with 3.2 LR+ (an increase of slightly more than 3 times the probability) and may be regarded as of little (but not necessarily unimportant) utility for RBD diagnosis 33 . The use of combined items also enabled us to further increase the specificity. The combination of positive answers in items 1+2+6.2, 2+6.1, and 6.1+6.2 determined a specificity of 94.7%, rendering more robust LR+ (6.1 up to 7.6, indicating an approximately 6-to-8-fold increase in the probability of having RBD) with acceptable reliability.
The original study by Stiasny-Kolster et al. 7 indicated a higher value (cut-off score 5) of the balance between sensitivity and specificity (96% sensitivity and 56% specificity). The low specificity of the total score of the instrument was attributed to the presence of comorbidities associated with excessive motor activity during sleep, including restless leg syndrome (RLS), obstructive sleep apnea syndrome (OSAS), periodic limb movements, and narcolepsy. Those comorbidities would render positive answers in items indicating limb movements (sub-items 4, 5, 6.2, and 7), inflating the resulting score. Another explanation was the presence of other sleep and neurological disorders, which would have rendered positive answers in sub-items 9 and 10, increasing the final score. As in the present study, the analysis of separate items in the Stiasny-Kolster study also allowed an increase of specificity, with 85.3-91.1% specificity when items 5, 6.3, and 6.4 were answered positively.
In our study, both the PD+RBD and PD-RBD subgroups included patients with other sleep disorders (such as RLS, OSAS, and insomnia), at similar ratios between groups. Thus, the low specificity found in the current study may be explained using the same arguments. The RBDSQ-BR score, however, showed no potential to select those patients, except in the case of insomnia. Even in this case, the resulting AUC (0.625) was less than that related to RBD (0.728), which was the target of the questionnaire.
The RBDSQ versions that were validated in Far East Asia 12,13,14,15 confirmed the cut-off score of 5 reported in the original study 7 , with sensitivity ranging from 88.5 to 100%, albeit with markedly high specificities (more often >90%). A new validation in Europe (Italy) 16 indicated a higher value as the best cut-off score (i.e., 8), with sensitivity of 84.2% and lower specificity of 78.0%. The validation study in Japan reported item 5 as the most specific one (92.3-96.4%) when the items were analyzed separately 12 . Tari et al. 15 found higher specificities in the Turkish population, not only for item 5 (85.7-97.4%), but also for items 7 (94.9%) and 10 (93.6%). Curiously, the Italian study 16 not only had higher specificities in some items separately (83.2-83.5% for items 5, 6.3, and 6.4), but also higher sensitivities than the cut-off total score (90.8-92.1% for items 1, 3, and 6.1).
The samples of those studies, however, were different from ours in that they were sometimes highly heterogeneous. Few studies have included patients with PD 7,15 , and except for that by Wang et al. 13 , the number of subjects with PD in these studies was minimal. Further, the method of selecting participants was not homogeneous between studies. Moreover, although the RBDSQ allows the companion to help, not all studies included that participation 12,15 .
The study by Nomura et al. 17 from 2011 was the first with the primary objective of evaluating RBDSQ performance specifically in patients with PD. Using the version validated in Japan 12 , consecutive patients with PD and patients with RBD alone were evaluated. The ROC curve of patients with PD showed a value of 6 as the best cut-off score, with a sensitivity similar to that observed in our study (84.2%), albeit with a considerably higher specificity (96.2%), which was comparable to other studies conducted in Far East Asia. The increase of 1 point in the cut-off score was explained by the positivity necessarily present in item 10 of the instrument. A possible explanation for the high value of specificity may be the fact that apparently no other sleep disorders were identified in the patients. Otherwise, they would have tended to score several items of the instrument, as previously mentioned.
In 2015, Wang et al. 13 included a sub-sample of patients with PD, with and without RBD. The cut-off score of 6 was also the most adequate for those subjects, with a sensitivity and specificity of 90.9 and 91.9%, respectively, which determined the diagnostic accuracy in 91.52% of patients. Their results did not show other sleep disorders in those patients, which might explain the high specificity.
In the same year, Stiasny-Kolster et al. 18 evaluated consecutive patients with PD, with no help from the companion, and concluded that the "learning" effect resulting from a detailed clinical evaluation before filling in the questionnaire may have significantly affected the diagnostic power of the instrument, at least in the case of PD.
In our study, the patients filled in the questionnaire before the clinical evaluation, which may explain why the overall RBDSQ-BR performance was clearly similar to the poor performance of the group that filled the instrument first in the previous study 18 . Nonetheless, the performance of the patients in the current study cohort was relatively better, which may be explained by the fact that most of our participants were helped by their companions. However, it should be noted that a possible epidemiological survey in our setting would most likely include the companion alongside the patient with PD filling in the questionnaire.
A few limitations in the current study must be noted. First, there was a lack of a control group of healthy subjects and a group of subjects with RBD and without PD (RBD alone), which would have improved the evaluation of the questionnaire performance and increased the generalization of the findings. Second, despite the similarity in education, either the patients' or their companions', between the analysis groups, its effect on the instrument performance cannot be ruled out. Further, the effect of companion assistance on instrument accuracy could also not be ruled out. Finally, the imbalance between the sizes of the groups, i.e., the small n in the group of patients without RBD, may have affected the determination of a low cut-off total score compared with the cut-off scores of other studies on patients with PD.
In conclusion, the results of the current study demonstrated that RBDSQ-BR was a valid and reliable instrument and that it may be useful for diagnosing RBD in Brazilian patients with PD. The instrument may also help in improving the selection of cases for a more detailed clinical evaluation or even polysomnography.