Evaluation of the quality of information on the internet: websites on the use of the Frequency Modulation System

ABSTRACT

Purpose

To evaluate the quality of the most accessed websites aimed at instructing teachers on Frequency Modulation Systems in terms of readability, reliability, usability and comprehensiveness.

Methods

The websites contained in the first five pages of Google and Yahoo!, obtained by searching for keywords / free terms: "FM system", "child", "hearing impairment", "teacher", "remote microphones" and " guidelines" were selected and evaluated by three analysts using the Flesch Reading EaseTest, the Discern Questionnaire, the System Usability Scale and the comprehensiveness assessment.

Results

Fourteen websites were evaluated. Half of them were classified according to the degree of readability and reliability as "difficult / very difficult" and "poor", respectively. As for usability, most websites were classified as "good", with only one categorized as "best achievable". The analysis of the scope of the websites revealed that the themes covered in a less precise way were: the use of the FM system by the teacher and student, and the handling of the device with the “on / off” mode.

Conclusion

The websites available in the Brazilian Portuguese language with teachers’ guidance, regarding the FM System, presented restricted readability and almost half of the available content was classified as poor, demonstrating that the reliability for treatment choices through this information is not so safe.

Keywords:
Internet; Telemedicine; Educational technology; Speech, Language and Hearing Sciences; Self-help devices

RESUMO

Objetivo

avaliar a qualidade dos websites de maior acesso, voltados à orientação de professores sobre Sistemas de Frequência Modulada, em termos de legibilidade, confiabilidade, usabilidade e abrangência.

Métodos

os websites contidos nas primeiras cinco páginas do Google e Yahoo!, obtidos mediante busca das palavras-chave/termos livres "sistema FM", "criança", "deficiência auditiva", "professor", “microfones remotos” e "orientações", foram selecionados e avaliados por três juízas, por meio do Teste de Facilidade de Leitura de Flesch, do Discern Questionnaire, da System Usability Scale e da avaliação de abrangência.

Resultados

quatorze websites foram avaliados. Destes, metade foi classificada quanto ao grau de legibilidade e confiabilidade como “difícil/muito difícil” e “ruim”, respectivamente. Quanto à usabilidade, a maioria dos webistes foi classificada como “boa”, sendo apenas um categorizado como “melhor alcançável”. A análise da abrangência dos websites revelou que os temas contemplados de forma menos precisa foram: a utilização do Sistema FM pelo professor e aluno e o manuseio do aparelho com o modo “liga/desliga”.

Conclusão

os websites disponíveis no idioma português do Brasil, com orientações para professores a respeito do Sistema FM, apresentaram legibilidade restrita e quase metade dos conteúdos disponibilizados foi classificada como “ruim”, demonstrando que a confiabilidade para escolhas de tratamento por meio dessas informações não é tão segura.

Palavras-chave:
Internet; Telemedicina; Tecnologia educacional; Fonoaudiologia; Equipamentos de autoajuda

INTRODUCTION

Hearing-impaired children and adolescents, users of hearing aids and/or cochlear implants (CI), may present difficulties in situations of competing noise, such as, school settings, being adjustments necessary for such contexts. Those difficulties bring about barriers, which affect the capability of auditory speech perception. A facilitator for those individuals is the use of remote microphones, such as the Frequency Modulation System (FM)⁽¹⁾. FM system consists of a device used to reduce the noise effects, distance and reverberation at school settings, thus helping auditory speech perception. The device is wireless, by means of a transmitter with a microphone, which is kept by the teacher/conversation partner, and a receiver connected to the student’s hearing aid and/or CI, allowing him/her better understanding of the conversation partner’s speech⁽²⁾.

Normative ordinance number 1,274 from June 25, 2013⁽³⁾ ensures, by means of the Brazilian Unified Health System (SUS in Portuguese), the concession of that device to speech-impaired individuals who make use of hearing aids and/or CI. Among the referral criteria are: age ranging from 5 to 17 years and eleven months, and enrollment in basic or high school. After update, by means of Normative Ordinance number 3 from February 19, 2020⁽⁴⁾, concession of the FM System by SUS, was extended to hearing-impaired individuals of any ages, enrolled in all academic levels.

In general, teachers of regular schools do not know the FM function, functioning mode and handling⁽⁵⁾. Considering that such professionals are fundamental parts for the effective use of that device, it is essential that they have information and required skills for its effective use and handling⁽²⁾.

Information on websites is valuable for health⁽⁶⁾ and education⁽⁷⁾ professionals’ training, as well as guidance of hearing-impaired children’s family members^(7,8). However, only the provision of such information is not enough. The online content is fundamental to be from reliable sources, grounded in the medical Science with scientific evidence, in order to avoid the spread of misleading/outdated notions^(9-11).

In line with the information content, care should be taken towards the readability in the text transmission. Thus, it is fundamental the adequate use of words, phrases and technical terms, in order to facilitate reading and comprehension of the offered materials, considering the plurality of the readers, who are diverse in age, socioeconomic status and schooling^(8,12).

Specifically, in the Speech-Language Pathology field, few studies have been conducted to investigate the quality of information on the internet. Among them, study which searched for available information on the orofacial functions⁽¹³⁾, oropharyngeal dysphagia⁽¹⁴⁾, and adenotonsillectomy surgery⁽¹⁵⁾stands out. That is important, as the best rated websites by the users, and most frequently accessed, do not always provide proper information.

The current context of public health crisis and social distancing initiated in 2020, triggered by the COVID-19 pandemic (SARS-CoV-2), has resulted in a series of social changes and highlighted preexisting human behaviors, for example, the use of information and communication digital technologies. Increase in the population contact with several electronic devices and online media, associated to the difficulty of physical access to healthcare services, has caused higher independent search for the most varied subjects in order to meet the needs of patients, parents and professionals. They have been prevented from clearing their doubts and concerns with specialized professionals in physically routine follow-ups, temporarily halted^(16,17). This reality stresses the concern with the quality and readability of website materials.

Keeping in mind the scarcity of studies on the quality of websites in the Speech-Language Pathology field, as well as the research gap in the analysis of Brazilian FM-System websites, this study aimed at assessing the quality of the most accessed websites on the FM System for teachers, in terms of readability, reliability, usability ad scope.

METHODS

The current research was elaborated in two steps; the first step was concerned with the search for the most accessed websites to guide teachers on the use of the FM System, and the second step assessed the quality, ethical principles, readability, scope and usability of the websites. The study did not involve human beings, as it did not analyze the impact of websites on the public. Therefore, it dismissed the analysis by an Ethics Board and the application of the Free Informed Consent Form.

1st Step: search for websites

The search for websites was conducted during the first semester of 2020. The search tools adopted were Google and Yahoo! search engines, considered the most popular worldwide. In this step, the five first pages from each search engine were reviewed.

In order to access the websites, the following keywords/free terms were used: “FM System”, “child”, “hearing impairment”, “teacher”, “remote microphones” and “guidance”, according to the combinations described in Chart 1.

As inclusion criteria, websites, portals, and blogs organized in Brazilian Portuguese, which held information and guidance on the concept of the FM system, its benefits, use and handling by teachers.

Websites, whose information sources aimed at reporting facts or sharing personal experiences, documents (scientific articles, monographs, dissertations, theses and recommendations), You Tube videos, news websites and online group discussion forums, were excluded.

The search and selection of websites were held by three analysts, undergraduates from the Speech-Language Pathology course. Before the assessment, all of them attended previous training on the FM System by a speech pathologist with ten years’ experience in the field. In addition, the standard criteria, considered in the content analysis of the websites, were defined. Subsequently, a single checklist of websites was established, contemplating the inclusion and eligibility criteria. In cases of disagreement, a fourth speech-language pathologist analyst was included for the review and consensus achievement.

2ª Step: assessment of the websites

The selected websites were independently reviewed by the analysts, according to three protocols established by literature: Flesch Reading Ease Test⁽¹⁸⁾, which assesses the level of readability by means of the formula from the Flesch Reading Ease Index (FREI); the Discern Questionnaire (DQ)⁽¹⁹⁾(^{Appendix 1}), which assesses the content quality, and the System Usability Scale (SUS)⁽²⁰⁾(^{Appendix 2}), which measures the usability. Only the specific pages on the theme were considered for all analyses. Automated Flesch Reading Ease Index (FREI) calculates the level of readability, considering the average length of sentences, and the average number of syllables per word. It is one of the best-used readability formulas in literature⁽²¹⁾. Score 0 means low level of readability, while 100 points means that the text has high level of readability. Regarding rating, 0-30% is considered “very difficult” (superior level), 30-50% as “difficult” (high school/higher education); 50-60% “fairly difficult” (beginning high school); 60-70% “standard” (7^th and 8^th grades); 70-80% “fairly easy” (6^th grade); 80-90% “easy” (5^th grade), and 90-100% “very easy” (4^th grade)⁽²¹⁾.

The DQ comprises 16 questions. Questions 1 to 15 assess the reliability of the information in health treatment choices, while the 16^th question assesses the overall score for the quality of information. In order to calculate the test result, questions are scored from 1 to 5. Thus, the higher the score, the better the content evaluation online. Each question entails discriminated information to facilitate the assessment. That tool is divided in 5 different ratings, depending on the score: between 16-26, the response is rated “very poor”; between 27-38, “poor”; 39-50, “fairly good”; 51-62, “good”, and higher than 63 points, “excellent”⁽²²⁾.

SUS scale comprises ten statements, regarding the usability of the assessed object. Odd-numbered statements indicate the positive points, while the even-numbered statements indicate the negative points in the usability. In order to calculate the total average of the usability, the scores for each item, which may range between 0 and 5 are added, and they can be rated as “strongly disagree”, “disagree”, “neutral””, “agree” and “strongly agree”. The scores in SUS scale range from 0 to 100, being 0-25 considered “the worst achievable”; 25-39, “poor”; 39-52, “acceptable”; 52-74, “good”; 74-85, excellent”, and 85-100, “the best achievable”⁽²³⁾.

For the analysis of each website scope, the analysts observed whether its contents contemplated or not pertinent information on the teacher’s guidance about four previously selected core themes: 1- definition of the FM System; 2- importance of the use of the device in the classroom; 3- how to use it by the teacher and by the student: transmitter (microphone) and receiver; 4- handling of the transmitter by the teacher: on/off, option of “muted microphone”. Based on the agreement of the response patterns among the analysts, the following scores were attributed: 1 point, “very insufficient”; 2, “insufficient”; 3, “reasonable”; 4, “satisfactory” and 5 points, “very satisfactory”. The scope of the websites was defined from the average score of each assessed item.

Results were tabulated and submitted to descriptive quantitative analysis of the data, by means of the average and standard deviation. Subsequently, the comparison of the adopted parameters was held, in order to establish the overall ranking of quality among the websites guiding teachers on the FM System, available in Brazilian Portuguese.

RESULTS

From 500 websites found in the search platforms (250 Google and 250 Yahoo!), 480 were excluded and 20 websites (12 Google and eight Yahoo!) were selected. However, during the study, 6 websites got unavailable. Therefore, the sample comprised 14 websites. The flowchart, regarding the search and selection steps, is described in Figure 1.

Figure 1
Flowchart of the research conducted on the FM System

Table 1 shows the aspects of readability measured by means of the Flesch Reading Ease Index.

Table 1
Readability analysis of the websites available in Brazilian Portuguese on the Frequency Modulation System applying the Flesch Reading Ease Index (FREI)

WEBSITE	READABILITY SCORE BY THE FREI	RATING
1	53.73%	Fairly difficult
2	57.23%	Fairly difficult
3	70.29%	Fairly easy
4	37.03%	Difficult
5	33.40%	Difficult
6	44.93%	Difficult
7	45.47%	Difficult
8	51.36%	Fairly difficult
9	46.05%	Difficult
10	87.72%	Easy
11	59.06%	Fairly difficult
12	18.82%	Very difficult
13	58.30%	Fairly difficult
14	43.84%	Difficult

• Subtitle: FREI = Flesch Reading Ease Index

Data regarding the reliability, analyzed by the DQ, are shown in Tables 2 and 3.

Table 2
Average analysis among the analysts by means of the Discern Questionnaire

WEBSITE	Q1	Q2	Q3	Q4	Q5	Q6	Q7	Q8	Q9	Q10	Q11	Q12	Q13	Q14	Q15	Q16
1	5.00±0.00	5.00±0.00	5.00±0.00	3.33±2.08	2.33±2.31	3.33±2.08	2.33±115	3.33±2.08	2.67±1.53	4.67±0.58	3.00±2.00	2.33±2.31	2.33±2.12	3.50±1.73	2.00±0.00	4.00±0.58
2	3.00±0.00	3.00±1.00	3.67±0.58	1.00±0.00	1.00±0.00	3.00±1.73	3.00±2.00	1.00±0.00	3.00±1.00	3.67±0.58	1.00±0.00	1.00±2.31	2.33±1.53	2.67±1.73	2.00±0.58	3.33±0.58
3	4.67±0.58	4.33±0.58	4.67±0.58	3.33±2.08	5.00±0.00	4.67±0.58	2.00±1.00	3.33±2.08	3.33±0.58	4.67±0.58	1.00±0.00	1.00±2.31	2.33±2.00	3.00±1.73	2.00±0.00	4.00±0.00
4	4.00±1.00	4.00±1.00	4.00±1.00	1.00±0.00	1.00±0.00	3.67±0.58	1.00±0.00	1.00±0.00	2.33±1.15	4.33±0.58	1.00±0.00	1.00±2.31	2.33±1.15	1.67±1.15	1.67±0.00	3.00±0.00
5	4.00±1.73	4.00±1.73	4.00±1.73	3.67±1.53	2.33±2.31	4.00±1.73	1.67±1.15	1.33±0.58	3.67±0.58	4.67±0.58	1.00±0.00	1.00±2.31	2.33±0.71	4.50±2.31	2.33±1.53	3.67±1.15
6	4.00±1.73	4.00±1.73	4.33±1.15	1.00±0.00	3.67±2.31	4.00±1.73	2.33±2.31	2.00±1.73	3.67±1.53	4.67±0.58	1.00±0.00	1.00±1.53	2.33±0.00	5.00±2.31	3.67±1.00	4.00±1.15
7	3.00±1.00	3.33±1.15	3.33±1.15	1.00±0.00	1.00±0.00	2.33±2.31	2.00±1.73	2.00±1.73	3.00±1.00	3.67±1.53	1.00±0.00	1.00±1.15	1.67±2.08	3.33±2.31	2.33±1.00	3.00±1.53
8	4.00±1.00	4.67±0.58	4.33±0.58	1.00±0.00	1.00±0.00	4.67±0.58	1.00±0.00	1.67±1.15	4.33±0.58	5.00±0.00	1.00±0.00	1.00±2.31	2.33±1.15	1.67±1.15	1.67±0.00	4.00±0.58
9	3.33±1.53	3.67±1.15	2.67±2.08	1.00±0.00	2.33±2.31	3.00±2.00	1.00±0.00	1.00±0.00	2.33±1.53	2.67±1.15	1.00±0.00	1.00±0.58	1.33±0.00	1.00±0.00	1.00±0.58	2.33±1.00
10	3.33±0.58	3.67±1.15	3.00±1.00	1.00±0.00	2.33±2.31	3.67±2.31	1.00±0.00	2.00±1.73	3.00±1.00	3.67±0.58	1.00±0.00	1.00±1.15	1.67±0.58	1.33±0.00	1.00±0.58	2.33±1.15
11	5.00±0.00	5.00±0.00	5.00±0.00	5.00±0.00	2.33±2.31	5.00±0.00	5.00±0.00	1.00±0.00	5.00±0.00	5.00±0.00	2.00±1.73	1.33±2.08	3.33±2.31	3.67±1.73	2.00±0.00	5.00±0.00
12	3.33±1.53	3.33±1.53	3.00±1.73	2.33±2.31	3.67±2.31	2.67±2.08	1.00±0.00	2.33±2.31	3.00±1.00	2.67±2.08	1.00±0.00	1.00±0.00	1.00±0.58	1.33±0.00	1.00±1.00	2.00±1.00
13	4.33±1.15	4.33±1.15	4.00±1.00	1.00±0.00	1.00±0.00	3.67±1.53	2.33±2.31	2.33±2.31	2.67±0.58	4.00±0.00	1.00±0.00	1.00±1.73	2.00±2.31	3.67±2.08	2.67±0.58	2.33±1.15
14	4.67±0.58	5.00±0.00	4.67±0.58	1.00±0.00	3.67±2.31	4.33±0.58	1.67±1.15	1.00±0.00	2.67±0.58	4.67±0.58	1.00±0.00	1.00±2.31	2.33±1.73	2.00±1.73	2.00±0.58	3.33±1.00

• Subtitle: Q = Question

Table 3
Overall average analysis and rating by means of the Discern Questionnaire

WEBSITES	TOTAL DQ	RATING
1	54.15	Good
2	37.67	Poor
3	53.33	Good
4	37.00	Poor
5	48.17	Fair
6	50.67	Fair
7	36.99	Poor
8	43.34	Fair
9	30.66	Poor
10	35.00	Poor
11	60.66	Good
12	34.66	Poor
13	42.33	Fair
14	45.01	Fair

• Subtitle: DQ = Discern Questionnaire

In Table 4, the total score attributed to the usability by means of the SUS scale.

Table 4
Result of the analysis of the usability questionnaire by applying the System Usability Scale

WEBSITES	Q1	Q2	Q3	Q4	Q5	Q6	Q7	Q8	Q9	Q10	TOTAL	RATING
1	3.33 ± 0.71	4.00 ± 2.12	3.67 ± 0.71	2.33 ± 0.23	3.00 ± 0.71	3.33 ± 1.18	4.00 ± 0.71	3.33 ± 1.18	3.67 ± 0.71	1.00 ± 2.12	79.17	Excellent
2	2.67 ± 0.71	4.00 ± 2.12	3.67 ± 0.71	2.33 ± 0.23	3.00 ± 0.71	2.00 ± 0.71	3.33 ± 0.71	3.33 ± 1.18	3.33 ± 0.71	1.00 ± 2.12	71.67	Good
3	3.00 ± 0.71	4.00 ± 2.12	3.67 ± 0.71	2.33 ± 0.23	3.33 ± 0.71	3.00 ± 0.71	3.67 ± 0.71	3.00 ± 0.71	3.67 ± 0.71	1.00 ± 2.12	76.67	Excellent
4	2.00 ± 0.71	3.67 ± 1.65	2.67 ± 0.71	2.33 ± 0.23	3.00 ± 0.71	2.67 ± 0.24	3.67 ± 0.71	4.00 ± 2.12	3.33 ± 0.71	1.00 ± 2.12	70.83	Good
5	3.33 ± 0.71	3.33 ± 1.18	3.67 ± 0.71	2.33 ± 0.23	3.33 ± 0.71	2.67 ± 0.24	3.67 ± 0.71	4.00 ± 2.12	3.33 ± 0.71	1.33 ± 1.65	77.50	Excellent
6	3.33 ± 0.71	3.33 ± 1.18	2.00 ± 0.71	2.67 ± 0.23	3.67 ± 0.71	2.67 ± 0.24	4.00 ± 0.71	4.00 ± 2.12	4.00 ± 0.71	1.00 ± 2.12	76.67	Excellent
7	1.67 ± 0.71	4.00 ± 2.12	2.67 ± 0.71	2.33 ± 0.23	3.67 ± 0.71	2.67 ± 0.24	3.67 ± 0.71	3.33 ± 1.18	3.00 ± 0.71	0.67 ± 2.59	69.17	Good
8	2.67 ± 0.71	3.67 ± 1.65	4.00 ± 0.71	2.33 ± 0.23	3.33 ± 0.71	2.67 ± 0.24	3.67 ± 0.71	4.00 ± 2.12	3.67 ± 0.71	1.00 ± 2.12	77.50	Excellent
9	1.00 ± 0.71	3.67 ± 1.65	0.33 ± 0.71	2.00 ± 0.70	2.33 ± 0.71	3.00 ± 0.71	3.67 ± 0.71	4.00 ± 2.12	2.67 ± 0.71	0.67 ± 2.59	58.33	Good
10	1.33 ± 0.71	3.67 ± 1.65	1.33 ± 0.71	2.00 ± 0.70	2.33 ± 0.71	3.00 ± 0.71	3.00 ± 0.71	3.67 ± 1.65	2.67 ± 0.71	1.00 ± 2.12	60.00	Good
11	4.00 ± 0.71	4.00 ± 1.18	3.67 ± 0.71	3.67 ± 0.70	4.00 ± 0.71	2.67 ± 0.71	3.67 ± 0.71	4.00 ± 0.71	4.00 ± 0.71	1.33 ± 2.12	87.50	Best achievable
12	1.00 ± 0.71	3.67 ± 1.65	2.00 ± 0.71	2.33 ± 0.23	1.33 ± 0.71	3.67 ± 1.65	0.67 ± 0.71	3.67 ± 1.65	2.00 ± 0.71	1.67 ± 1.18	55.00	Good
13	0.67 ± 0.71	3.67 ± 1.65	2.00 ± 0.71	2.00 ± 0.70	1.33 ± 0.71	3.67 ± 1.65	2.00 ± 0.71	3.67 ± 1.65	2.00 ± 0.71	1.67 ± 1.18	56.67	Good
14	2.00 ± 0.71	3.67 ± 1.65	1.33 ± 0.71	2.33 ± 0.23	3.33 ± 0.71	2.67 ± 0.24	2.67 ± 0.71	3.67 ± 1.65	3.00 ± 0.71	1.33 ± 1.65	65.00	Good
OVERALL AVERAGE											70.12	Good

The analysis of the website scope is shown in Table 5.

Table 5
Result of the scope assessment average on the information in the websites

WEBSITES	DEFINITION	RELEVANCE IN THE CLASSROOM	USE BY THE TEACHER AND STUDENT	ON/OFF	OVERALL SCOPE
1	4.33 ± 0.58	4.33 ± 0.58	3.00 ± 1.00	1.00 ± 0.00	12.66
2	4.00 ± 0.00	2.33 ± 0.58	1.67 ± 1.15	1.33 ± 0.00	9.33
3	4.67 ± 0.58	4.67 ± 0.58	4.33 ± 0.58	1.00 ± 0.00	14.67
4	4.00 ± 0.00	2.67 ± 1.15	1.00 ± 0.00	1.00 ± 0.00	8.67
5	3.67 ± 2.31	5.00 ± 0.00	4.67 ± 0.58	1.00 ± 0.00	14.34
6	4.00 ± 1.73	4.67 ± 0.58	5.00 ± 0.00	1.00 ± 0.00	14.67
7	3.00 ± 1.00	2.33 ± 1.53	3.67 ± 0.58	1.00 ± 0.00	10.00
8	4.33 ± 0.58	4.00 ± 1.00	5.00 ± 0.00	1.00 ± 0.00	14.33
9	2.00 ± 1.00	2.67 ± 0.58	3.67 ± 2.31	1.00 ± 0.00	9.34
10	3.00 ± 0.00	2.33 ± 0.58	4.33 ± 1.15	1.33 ± 0.58	10.99
11	5.00 ± 0.00	5.00 ± 0.00	5.00 ± 0.00	5.00 ± 0.00	20.00
12	3.67 ± 1.15	2.33 ± 1.15	2.67 ± 1.53	1.00 ± 0.00	9.67
13	4.00 ± 0.00	3.33 ± 1.53	2.67 ± 2.08	1.00 ± 0.00	11.00
14	4.00 ± 0.00	4.33 ± 0.58	4.33 ± 0.58	1.00 ± 0.00	13.66

Chart 2 shows the overall score in the analysis of the websites, regarding the highest and lowest scores in the applied assessments.

DISCUSSION

Websites are tools used by various types of readers as a source of searching for certain information. During the past year, mainly due to the COVID-19 (SARS-CoV-2) pandemic, increase was observed in the use of media and the internet by adults⁽¹⁷⁾. That is occurring because new educational, healthcare and social strategies, among others, implemented by the current society of the “new normal”, have been changed in remote modalities⁽²⁴⁾. Therefore, the assessment of the information sources available online is fundamental so that they can be allied to health promotion and education. For example, the warrant of quality materials on websites allows the use and the indication of health professionals as complementary to the information provided in on-site appointments.

Despite the search conducted in two large search engines, from 500 websites, only 14 (2.8%) addressed the FM-System theme (Figure 1). In other studies, which used the same search tools regarding orofacial functions, 35 (10%) out of 350 websites met the inclusion criteria⁽¹³⁾. On the theme of the oropharyngeal dysphagia in the elderly, 19 (9.5%) websites out of 200 were identified during the search⁽¹⁴⁾. Regarding laypeople’s guidance on adenotonsillectomy surgery, 34 (34%) websites out of 100 met the inclusion criteria⁽¹⁵⁾. Such data corroborate the content scarcity available online about the use of the FM System.

As for the readability, 50% of the selected websites were rated “difficult/very difficult” (Table 1), that is, half of the websites require higher education on the part of the readers for their readability, which limits their access to information. Comparing to other studies that used the same type of assessment, a 47.8% level of difficulty was observed in the research on adenotonsillectomy surgery⁽¹⁵⁾, and 61.23% on the theme of orofacial functions, rating the materials as appropriated. However, the readability test presents some constraints, such as assessment lacks high level of depth, and only considers syllable and word count, not the featured content⁽²⁵⁾.

Most websites included in this study (42.85%) were rated “poor” (Table 3) when considering their reliability. This finding is in line with data obtained in another study, which observed that no websites were rated “excellent” or “very poor”. Among the justifications for that finding, the authors reported that in 60% of the websites, references were not provided, corroborating the low quality of the information⁽²²⁾.

The overall average scoring in the SUS scale was 70.0 points, and website rating was “good” (Table 4). The same rating was evidenced in a study using that protocol for the usability assessment of an information system on neonatal health, according to the users’ perception⁽²⁶⁾.

Concerning the scope, it was observed that in great part of the included websites, information was rated “insufficient” in the criteria related to the “way of using the FM device by the teacher and the student: position of the transmitter (microphone) and receiver”; “transmitter handling by the teacher: on/off, “muted microphone” option (Table 5). This result agrees with the above-mentioned study⁽¹⁴⁾on oropharyngeal dysphagia in the elderly, which rated 33.6% in the scope assessment.

Website 11 (Web11) was the best scored concerning reliability, usability and scope (Chart 2). It was designed by a higher education institution, which can explain the quality of the information provided. It was the only website rated “the best achievable” in the SUS scale, and the maximum scoring regarding the content scope. However, it did not achieve such high scoring regarding readability, being rated “fairly difficult” in this aspect.

Despite not replacing the guidance provided by a specialized speech-language pathologist, teachers cannot prevent from searching for online information on the use of the FM System. Thus, the offer of quality information is essential in order to help them in the inclusion of students who use that device in the classroom. In that sense, the assessment of online health information is extremely important, mainly nowadays, as it is a matter of public health to warrant trustworthy, safe websites for the users, whether a patient, a family member or a professional⁽²⁷⁾.

Further studies on the theme are deemed necessary, aiming at promoting discussions which foster the interest and concern on the part of website content creators, not only with the elaboration of reliable information material, based on scientific evidence, but also committed to getting the content across in an accessible way, regarding content readability, keeping in mind the diversity of the target public.

CONCLUSION

The websites available in Brazilian Portuguese with guidance for teachers on the FM System presented restricted readability, requiring high school/higher education level for their content understanding. Moreover, almost half of the contents available were rated “poor”, evidencing that the information reliability for treatment choices is not safe.

Most websites do not contemplate significant information on the use of the FM System by the teachers in the classroom.

Appendix 1 Discern Questionnaire (DQ) Questions

1. Is the text clear about the page aims?

2. Does the text achieve the proposed objectives?

3. Is the text information relevant?

4. Is the text clear what sources of information were used by the authors for the reported data?

5. Is the text clear when (date) the reported information was produced?

6. Is the text fair, balanced and unbiased?

7. Does the text provide other additional sources for further information?

8. Does the text mention uncertainties/lack of consensus regarding the treatment?

9. Does the text describe each treatment step by step (the technique)?

10. Does the text describe at least one benefit/advantage of the treatment?

11. Does the text describe at least one of the risks/disadvantages of the treatment?

12. Does the text describe what would happen if no treatment were used?

13. Does the text describe how the treatment choices affect the overall quality of life?

14. Is the text clear that there are other treatment choices?

15. Does the text offer suggestions of items or topics to be discussed with the doctor, or encourage the discussion with the doctor to make a shared decision on the treatment between the user and the doctor?

16. Based on the responses of the former questions, what is the overall quality of the publication as a source of information on the treatment?

Appendix 2 System Usability Scale (SUS) Questions - translated to Portuguese

1. I think I would like to use this product frequently.

2. I considered the product unnecessarily complex than it should be.

3. I thought the product easy to use.

4. I think that I would need the support of a technical person to be able to use this product.

5. I found the various functions in this product were very well integrated.

6. I thought that there was too much inconsistency in this product.

7. I would imagine most people would learn to use this product very quickly.

8. I found the product very cumbersome to use..

9. I felt very confident using this product.

10. I needed to learn a lot of things before I get going with this product.

11. Do you have any criticism and/or suggestions regarding the product? (optional).

Chart 1. Combination of search terms

Combination 1	FM System / child/ hearing impairment
Combination 2	FM System/ teacher/ guidance
Combination 3	FM System/ guidance
Combination 4	FM System
Combination 5	Remote microphones/ child/ hearing impairment

Chart 2. Overall analyses of the best three and the worst three scores in the four assessment tests on the information about the Frequency Modulation System

	FREI	DQ	SUS	Scope
Highest Score	Web10 (87.72%)	Web11 (60.66)	Web11 (87.50)	Web11 (20)
	Web3 (70.29%)	Web1 (54.15)	Web1 (79.17)	Web3/6(14.67)
	Web14 (57.23%)	Web3 (53.33)	Web5/8(77.50)	Web5(14.34)
Lowest Score	Web12 (18.82%)	Web9 (30.66)	Web12 (55.0)	Web4 (8.67)
	Web5 (33.40%)	Web12 (34.60)	Web13 (56.67)	Web2 (9.33)
	Web4 (37.03%)	Web10 (35.0)	Web9 (58.33)	Web9 (9.34)

Subtitle: FREI =Flesch Reading Ease Index; DQ = Discern Questionnaire; SUS = System Usability Scale; Web = website

• Study conducted at Centro Universitário Planalto do Distrito Federal – UNIPLAN – Brasília (DF), Brasil.
• Funding: None.

REFERÊNCIAS

• ¹
  Barreiros GE, Mendes BCA, Ficker LB, Novaes BCAC. Reabilitação auditiva: papel da família na adesão ao uso do sistema de transmissão sem fio na escola. Distúrb Comun. 2016;28(4):718-29.
• ²
  Carvalho DS, Pedruzzi CM. Uso do sistema de frequência modulada por escolares com perda auditiva. Distúrb Comun. 2019;31(1):12-21. http://dx.doi.org/10.23925/2176-2724.2019v31i1p12-21
  » http://dx.doi.org/10.23925/2176-2724.2019v31i1p12-21
• ³
  Brasil. Ministério da Saúde. Portaria nº 1274, de 25 de junho de 2013. Inclui o Procedimento de Sistema de Frequência Modulada Pessoal (FM) na Tabela de Procedimentos, Medicamentos, Órteses, Próteses e Materiais Especiais (OPM) do Sistema Único de Saúde. Diário Oficial da União [Internet]; 2013. [citado em 2021 Fev 20]. Disponível em: https://bvsms.saude.gov.br/bvs/saudelegis/gm/2013/prt1274_25_06_2013.html
  » https://bvsms.saude.gov.br/bvs/saudelegis/gm/2013/prt1274_25_06_2013.html
• ⁴
  Brasil. Ministério da Saúde/ Secretaria de Ciência, Tecnologia, Inovação e Insumos Estratégicos em Saúde. Portaria nº 3, de 19 de fevereiro de 2020. Torna pública a decisão de ampliar o uso do Sistema de Frequência Modulada Pessoal para indivíduos com deficiência auditiva de qualquer idade matriculados em qualquer nível acadêmico, no âmbito do Sistema Único de Saúde - SUS. Diário Oficial da União [Internet]; 2020. [citado em 2021 Fev 20]. Disponível em: https://www.in.gov.br/web/dou/-/portaria-n-3-de-19-de-fevereiro-de-2020-244302714
  » https://www.in.gov.br/web/dou/-/portaria-n-3-de-19-de-fevereiro-de-2020-244302714
• ⁵
  Fidêncio VLD, Costa CA, Sousa IS, Romão JMFE. Investigação do conhecimento de professores de escolas regulares de uma região do Distrito Federal sobre o sistema de frequência modulada. Audiol Commun Res. 2020;25:e2278. http://dx.doi.org/10.1590/2317-6431-2019-2278
  » http://dx.doi.org/10.1590/2317-6431-2019-2278
• ⁶
  Maximino LP, Zambonato TCF, Picolini-Pereira MM, Corrêa CC, Feniman MR, Blasca WQ. Development and evaluation of a blog about cleft lip and cleft palate and hearing. Int Arch Otorhinolaryngol. 2018;22(1):60-7. http://dx.doi.org/10.1055/s-0037-1603494 PMid:29371900.
  » http://dx.doi.org/10.1055/s-0037-1603494
• ⁷
  Corrêa CC, Silva RA, Blasca WQ. Elaboration and evaluation of contents about hearing health inserted in cybertutor. Int Arch Otorhinolaryngol. 2014;18(2):115-21. http://dx.doi.org/10.1055/s-0033-1358578 PMid:25992075.
  » http://dx.doi.org/10.1055/s-0033-1358578
• ⁸
  Chaves JN, Libardi AL, Agostinho-Pesse RS, Morettin M, Alvarenga KF. Telessaúde: avaliação de websites sobre triagem auditiva neonatal na Língua Portuguesa. CoDAS. 2015;27(6):526-33. http://dx.doi.org/10.1590/2317-1782/20152014169 PMid:26691616.
  » http://dx.doi.org/10.1590/2317-1782/20152014169
• ⁹
  Roshan A, Agarwal S, England RJ. Role of information available over the internet: what are the parents of children undergoing tonsillectomy likely to find? Ann R Coll Surg Engl. 2008;90(7):601-5. http://dx.doi.org/10.1308/003588408X318129 PMid:18701014.
  » http://dx.doi.org/10.1308/003588408X318129
• ¹⁰
  Strychowsky JE, Nayan S, Farrokhyar F, MacLean J. YouTube: a good source of information on pediatric tonsillectomy? Int J Pediatr Otorhinolaryngol. 2013;77(6):972-5. http://dx.doi.org/10.1016/j.ijporl.2013.03.023 PMid:23598152.
  » http://dx.doi.org/10.1016/j.ijporl.2013.03.023
• ¹¹
  Sorensen JA, Pusz MD, Brietzke SE. YouTube as na information source for pediatric adenotonsillectomyandeartube surgery. Int J Pediatr Otorhinolaryngol. 2014;78(1):65-70. http://dx.doi.org/10.1016/j.ijporl.2013.10.045 PMid:24315211.
  » http://dx.doi.org/10.1016/j.ijporl.2013.10.045
• ¹²
  Favoretto NC, Carleto NG, Arakawa AM, Alcalde MP, Bastos JRM, Caldana ML. Portal dos idosos: desenvolvimento e avaliação de um website com informações sobre o processo de envelhecimento e as principais alterações fonoaudiológicas que acometem os idosos. CoDAS. 2017;29(5):e20170066. http://dx.doi.org/10.1590/2317-1782/20172017066 PMid:29069273.
  » http://dx.doi.org/10.1590/2317-1782/20172017066
• ¹³
  Corrêa CC, Ferrari DV, Berretin-Felix G. Quality, range, and legibility in web sites related to orofacial functions. Int Arch of Otorhinolaryngol. 2013;17(4):358-62.
• ¹⁴
  Corrêa CC, Costa DR, Berretin-Felix G. Websites em português sobre disfagia orofaríngea no idoso. Rev Disturb Comun. 2018;30(1):140-6. http://dx.doi.org/10.23925/2176-2724.2018v30i1p140-146
  » http://dx.doi.org/10.23925/2176-2724.2018v30i1p140-146
• ¹⁵
  Piña SP, Corrêa CC, Carvalho LR, Weber SAT. Comprehensiveness, readability, and reliability of Brazilian websites available for lay people guidance on adenotonsillectomy. Braz J Otorhinolaryngol. 2021;87(1):66-73. http://dx.doi.org/10.1016/j.bjorl.2019.06.012 PMid:31439530.
  » http://dx.doi.org/10.1016/j.bjorl.2019.06.012
• ¹⁶
  Senhoras EM. Coronavírus e educação: análise dos impactos assimétricos. Revista UFRR. 2020;2(5):128-33.
• ¹⁷
  Malta DC, Szwarcwald CL, Barros MBA, Gomes CS, Machado IH, Souza PRB Jr, et al. A pandemia da COVID-19 e as mudanças no estilo de vida dos brasileiros adultos: um estudo transversal. Epidemiol Serv Saude. 2020;29(4):e2020407. http://dx.doi.org/10.1590/s1679-49742020000400026 PMid:32997069.
  » http://dx.doi.org/10.1590/s1679-49742020000400026
• ¹⁸
  Flesch R. How to write plain English. New York, NY: Harper and Row; 1979
• ¹⁹
  Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105-11. http://dx.doi.org/10.1136/jech.53.2.105 PMid:10396471.
  » http://dx.doi.org/10.1136/jech.53.2.105
• ²⁰
  Liang J, Xian D, Liu X, Fu J, Zhang X, Tang B, et al. Usability study of mainstream wearable fitness devices: feature analysis and system usability scale evaluation. JMIR Mhealth Uhealth. 2018;6(11):e11066. http://dx.doi.org/10.2196/11066 PMid:30409767.
  » http://dx.doi.org/10.2196/11066
• ²¹
  Lyra DH, Amaral CLF. Apreensibilidade e legibilidade de artigos científicos de um periódico nacional. Tekhne e Logos. 2012;3(3):90-101.
• ²²
  Olkun HK, Demirkaya AA. Evaluation of internet information about lingual orthodontics using DISCERN and JAMA Tools. Turk J Orthod. 2018;31(2):50-4. http://dx.doi.org/10.5152/TurkJOrthod.2018.17042 PMid:30112514.
  » http://dx.doi.org/10.5152/TurkJOrthod.2018.17042
• ²³
  Santos BM, Cordeiro MEC, Schneider IJC, Ceccon RF. Educação médica durante a pandemia da COVID-19: uma revisão de escopo. Rev Bras Educ Med. 2020;44(suppl.1):e0139.
• ²⁴
  Feitosa ANA, Nascimento IMG, Alencar RL Na, Alencar MT, Tavares MML, Assis EV. Tecnologias educacionais em tempos de pandemia: relato de experiência. Braz J Production Engineering. 2020;6(6):166-72.
• ²⁵
  Corrêa CC, Pauleto ARC, Ferrari DV, Berretin-Felix G. Website Babies Portal: development and evaluation of the contents regarding orofacial functions. J Appl Oral Sci. 2013;21(6):581-9. http://dx.doi.org/10.1590/1679-775720130267 PMid:24473726.
  » http://dx.doi.org/10.1590/1679-775720130267
• ²⁶
  Padrini-Andrade L, Balda RCX, Areco KCN, Bandiera-Paiva P, Nunes MV, Marba STM, et al. Avaliação da usabilidade de um sistema de informação em saúde neonatal segundo a percepção do usuário. Rev Paul Pediatr. 2019;37(1):90-6. http://dx.doi.org/10.1590/1984-0462/;2019;37;1;00019 PMid:30569950.
  » http://dx.doi.org/10.1590/1984-0462/;2019;37;1;00019
• ²⁷
  Moretti FA, Oliveira VE, Silva EMK. Acesso a informações de saúde na internet: uma questão de saúde pública? Rev Assoc Med Bras. 2012;58(6):650-8. http://dx.doi.org/10.1590/S0104-42302012000600008 PMid:23250092.
  » http://dx.doi.org/10.1590/S0104-42302012000600008

Publication Dates

• Publication in this collection
  30 Aug 2021
• Date of issue
  2021

History

• Received
  17 Mar 2021
• Accepted
  15 May 2021