Validity and reliability of the questionnaire that measures the perception of speech-language-hearing therapists regarding the use of information and communication technologies (ICTs) in speech-language-hearing evaluation and intervention processes

ABSTRACT

Purpose:

to study the validity and reliability of the questionnaire that measures the perception of speech-language-hearing therapists, regarding the use of information and communication technologies (ICTs) in evaluation and intervention processes.

Methods:

a descriptive, cross-sectional, observational, nonexperimental study. A total of 123 professionals from the Region of Valparaíso, Chile, with degrees in speech-language-hearing therapy, participated in the study. They answered the questionnaire that measures their opinion and knowledge on the use of ICTs.

Results:

the questionnaire had an adequate level of stability, as the Cronbach's α and Guttman's λ₆ coefficient values were respectively 0.85 and 0.95. When studying each item, adequate levels of stability were obtained. On the other hand, the corrected homogeneity index (CHI) yielded values lower than 0.2 for items A8, A9, A17, A18, A26, A27, A42, A43, and A46, which suggests not considering them due to a lack of correlation between these items and the overall questionnaire score. Regarding content validity, assessed with the Question Appraisal System, 100% of the questions presented no major semantic problems.

Conclusion:

the results of the study indicate that the ICT questionnaire is valid and reliable.

Keywords:
Reproducibility of Results; Surveys and Questionnaires; Speech, Language and Hearing Sciences; Information Technology

RESUMEN

Objetivo:

estudiar la validez y confiabilidad del cuestionario para medir la percepción de los profesionales fonoaudiólogos respecto al uso de las tecnologías de la información y comunicación (TICs) en los procesos de evaluación e intervención.

Métodos:

estudio descriptivo de corte transversal, observacional no experimental, participaron un total de 123 profesionales titulados de la carrera de fonoaudiología de la Región de Valparaíso, Chile, quienes contestaron el cuestionario que mide la opinión y conocimiento del uso de TICs.

Resultado:

el cuestionario exhibió un nivel de estabilidad adecuado, ya que los valores de los coeficientes de α de Cronbach y λ₆ de Guttman fueron 0,85 y 0,95, respectivamente y al estudiar cada ítem se obtuvieron niveles adecuados de estabilidad. Por otro lado, Índice de Homogeneidad Corregida (IHC), arrojó valores menores a 0,2 para los ítems A8, A9, A17, A18, A26, A27, A42, A43 y A46, lo cual sugiere no considerarlos porque indica que no existe correlación entre el ítem y la puntuación global del cuestionario. En cuanto a la validez de contenido, a través del sistema de evaluación de preguntas, el 100% de las preguntas no presentan problemas desde el punto de vista semántico.

Conclusión:

los resultados del estudio indican que el cuestionario TICs es válido y confiable.

Descriptores:
Reproducibilidad de los resultados; Encuestas y Cuestionario; Fonoaudiología; Tecnología de la Información

Introduction

Speech-language-hearing (SLH) pathology is the field of health responsible for assessments, diagnoses, intervention, promotion, and prevention regarding disorders that affect speech, language, voice, hearing, communication, swallowing, and oral-motor function¹. Since the 1990s, in Chile, the SLH practice has specialized in the field of education², which is explained by their connection with language and speech disorders in neurodevelopment^3-4. In the educational context, the fields of child language and speech are specifically ruled by ministerial decrees^5-6, which establish that SLH diagnoses must be made with standardized tests, designed for children 3 years to 7 years and 11 months old⁵.

Decree no. 1300 became effective in Chile in 2003⁵. It advocates the gradual implementation of the study plans and programs for students with special education needs (SEN) or specific language disorder (SLD)⁷, from special language schools and/or basic schools with inclusion projects approved by the Ministry of Education. The decree indicates that, for SLH therapists to rule out or confirm an SLD diagnosis, they must use three structured tests that have been adapted and standardized for the Chilean population by the Universidad de Chile (University of Chile), namely: (1) the Test for Auditory Comprehension of Language⁸ (TACL), (2) Test for the Evaluation of Phonological Simplification Processes⁹ (TEPROSIF-R, in Spanish), and the Screening Test of Spanish Grammar¹⁰ (STSG).

Technology has been gradually incorporated into the SLH professional practice, specifically in the intervention process¹¹. These technological tools are used as educational instruments that help improve linguistic skills in treatment. However, the use of this progress - i.e., the information and communication technologies (ICTs) - is a decision that must be made exclusively by each therapist. In this sense, it is not explicitly known how the SLH therapists’ who work with children perceive their effectiveness and appropriateness in the various age ranges of children and adolescents who need SLH attention.

ICTs have changed over time, due to the quick technological development. More specifically, the main representatives of such advancements are the smartphone, computers, social media, applications, and the Internet¹². Their current social impact is unquestionable, given the various degrees of protagonism they have had over time and in recent years^13-14. Thus, ICTs are described as the “revolutionary, impacting, and changing phenomenon that encompasses both technical and social aspects and that impregnates all human, work, educational, academic, leisure, and consumption activities”¹⁵.

The availability of an instrument that reveals the SLH therapists’ perception of technological elements used in their professional practice helps propose intervention plans that consider the use of ICTs as auxiliary tools in SLH sessions¹⁶.

There is no valid instrument in Chile with an adequate level of reliability to learn the perception of SLH therapists regarding the use of ICTs. Therefore, a questionnaire has been proposed to measure these professionals’ perceptions. This idea arose as an initiative from the investigation team that participated in an internal project at the Universidad Viña del Mar (Viña del Mar University), named “SLH professionals’ perspective of the use of new information and communication technologies (ICTs) in the SLH assessments in the field of child language and speech in the commune of Viña del Mar”, approved in 2019.

Hence, the research question asks: “Is this proposed instrument valid and reliable to assess the perception of the SLH therapists regarding the use of ICTs in the assessment and intervention processes?”. To answer this question, it is necessary to know first the perception of the therapists responsible for the prevention, assessment, habilitation, and/or rehabilitation of children, adolescents, and adults concerning the use and effectiveness of new ICTs in SLH sessions.

This investigation aims to study the content validity and reliability of a questionnaire that measures the perception of SLH professionals, regarding the use of ICTs in evaluation and intervention processes.

Methods

This investigation design did not require submission to the Ethics Committee because there was no intervention with the participants.

SLH therapists were invited to participate in this validity and reliability study of the ICT questionnaire. They voluntarily agreed to participate and signed the informed consent form, which was sent to them via e-mail. To ensure each participant’s confidentiality, the data were coded, and only the lead researcher had access to personal data.

This cross-sectional, observational, nonexperimental study has a quantitative investigation and descriptive design. The target population comprises SLH professionals who work in the field of education in the Region of Valparaíso, Chile. The inclusion/exclusion criteria were as follows: SLH professionals with a bachelor’s degree who had been working in the profession for at least one year in the context of education. SLH professionals who did not have a bachelor’s degree and had not been working in the said terms were excluded from the study.

Five experts participated in the content validity assessment, while 164 professionals who met the characteristics of the target population were considered for the reliability study. Of the total 164 subjects invited, 123 fully answered the questionnaire - hence, the response rate was 75%.

Validity

The content validity¹⁷ was conducted at the end of the ICT questionnaire construction process. The questionnaire construction was focused on the top-down theory, which is centered on developing questions according to the theoretical framework and literature review related to the topic that is to be measured^18-19. Then, based on the conceptual theoretical framework, three modules are established to assess three aspects related to the SLH assessment and intervention processes and the use of ICTs.

The construction of the ICT questionnaire had three steps, detailed below:

Step 1: Conceptualization of the Questionnaire Constructs.

Three modules were considered to find the perception of SLH professionals regarding the use of ICTs in assessment and intervention processes in children aged 0 to 6 years or older.

1. Module 1: Collecting the opinion of the professionals regarding the overall characteristics of the three existing instruments, namely: TACL⁸, TEPROSIF-R⁹, and STSG¹⁰.

2. Module 2: Surveying information on the use of the new ICTs in the SLH assessment and intervention processes.

3. Module 3: Measuring the SLH professionals’ level of knowledge of the current regulations.

Step 2: Operationalization of the Constructs.

Each module has a set of Likert-type questions. Module 1 has 27 items, module 2 has 14 items, and module 3 has six items. The Likert scale has five response options, whose lowest and highest scores, related to the categories “Strongly disagree” and “strongly agree”, correspond respectively to 1 and 5.

Lastly, modules 1 and 2 in the questionnaire have questions related to opinion, while module 3 has questions on knowledge. Also, in the section where the participants identify themselves, the questions address their professional responsibilities and practices. Figure 1 presents the structure of the ICT questionnaire.

Figure 1:
Structure of the questionnaire to find the perception of speech-language-hearing therapists regarding the use of information and communication technology in the speech-language-hearing assessment and intervention processes

Step 3: Assessment of the Questionnaire Items.

The Question Appraisal System (QAS) is a quantitative tool that assesses questionnaire items. Altogether, it has eight items to assess possible problems related to each question’s reading, instructions, clarity, underlying assumptions, knowledge, bias, response categories, and other problems in general, which an expert with knowledge on the topic may identify²⁰.

Figure 2 describes the question assessment process - i.e., the content validity¹⁷ -, which was carried out by a focus group²¹ based on the judgment of experts. The activity followed three stages: before, during, and after the focus group.

Figure 2:
Diagram of the process to assess questions potentially with problems, before, during, and after the focus group

• Before the focus group: The activities to be carried out in the focus group were scheduled in detail. Then, a qualitative convenience sample was defined to select the sampling unit, that is, the expert on the topic to be studied. The unit was selected considering SLH participants whose time of graduation varied and who had been working in the context of education, health, and/or projects. The literature suggests from seven to 10 participants²², though this study counted with five experts. The professionals whose expertise level was based on their practice in the abovementioned fields were later contacted via email or phone call. Moreover, the questions that would possibly conflict with the question-response cognitive process associated with the cognitive model of Tourangeau¹⁹ were selected in this stage.

• During the focus group: Following the focus group activity schedule, the experts were introduced to the importance of counting with a valid questionnaire¹⁹ to assess the perception of the SLH therapists regarding their assessment and intervention processes. Then, a brief presentation explained the use of the QAS and how the system assesses possible problems in the items that make up a questionnaire. The QAS can be used in two modalities: individually and/or in a group. In this study, the question assessment was made in a group.

• After the focus group: The results of the QAS assessment process were described. Then, the questions that received observations and/or suggestions in the QAS steps were analyzed and adjusted. Lastly, the self-administered questionnaire was adjusted in paper and digital format.

In summary, the content validity¹⁷ was conducted with a panel of experts²³ in an online focus group²¹ with five SLH professionals. The activity was carried out via Zoom platform²⁴ and phone calls, due to the health contingency circumstances.

Reliability

The questionnaire reliability study (internal consistency) used the Cronbach’s α and Guttman’s λ₆ coefficients^25-27. Values in these coefficients higher than 0.70 indicated adequate stability in the responses. The corrected homogeneity index (CHI) was also analyzed to study the relationship between a selected item and the other ones. Values lower than 0.2 indicated that the item should be assessed and/or excluded because of the lack of an important relationship with the other items.

Data Collection

The data were surveyed with the self-administered digital questionnaire, using the Google Forms platform. After receiving the participants’ information, the data were ordered, coded, and encrypted to protect each participant’s identification, complying with statistical confidentiality. Furthermore, the data were stored in .xls format and exported to the R-project statistical program for the reliability study.

Results

Validity

The content validity results from the experts’ focus group activity required no adjustments in the collected observations and suggestions for the module 1 questions (A1 - A27). Rather, the experts considered that they did not have any type of problems in terms of reading, instructions, clarity, assumptions, knowledge, bias, response categories, and so forth. Hence, they declared that 100% of the questions were clear and precise.

In module 2 (A28 - A41), besides assessing the potential problems in the questions, the experts suggested narrowing the age ranges regarding whom they questioned the reliability and improvements in the assessment and intervention processes and ICT use. Likewise, 100% of the questions were exempt from potential problems as studied with QAS. Lastly, in module 3 (A42 - A47) they suggested reviewing how questions A43 and A44 were presented, as they had clarity problems. In this module, the experts highlighted that the concept of knowledge is vague or not defined. Therefore, they suggested distinguishing the knowledge delivered at the academic institutions to which the interviewees belonged from the knowledge either self-taught or acquired in postgraduate programs. After adjusting the questions as suggested by the experts, the questionnaire was adjusted in paper and converted into digital format.

Reliability

The ICT questionnaire reliability study yielded a Cronbach’s α coefficient of 0.85 and Guttman’s λ₆ coefficient of 0.95, which indicates the questionnaire’s adequate internal consistency. On the one hand, the analysis of the coefficient values for each questionnaire item revealed that all items had high internal consistency, as the coefficient values ranged from 0.84 to 0.86 for Cronbach’s α and from 0.94 to 0.95 for Guttman’s λ₆ (see Table 1). Moreover, none of the internal consistency values presented in Table 1 surpassed the overall Cronbach’s α (0.85) or Guttman’s λ₆ coefficients (0.95).

Table 1:
Reliability of the items in the questionnaire on the use of information and communication technology, considering all questions

Questions	Coefficients			Comment
	Cronbach	Guttman	CHI
A1: The quality of the images in the TACL test is good	0.85	0.95	0.292	Reliable
A2: The size of the images in the TACL test is adequate	0.85	0.95	0.283	Reliable
A3: The presentation format in the TACL test is attractive to children	0.85	0.95	0.372	Reliable
A4: I rely on the results obtained with the TACL test	0.85	0.95	0.336	Reliable
A5: The content assessed in the TACL test is consistent	0.85	0.95	0.363	Reliable
A6: The presentation format of the TACL test seems adequate to me	0.85	0.95	0.356	Reliable
A7: The TACL test requires a digital version (a tablet or computer application)	0.85	0.95	0.480	Reliable
A8: The image content in the TACL test must be updated	0.85	0.95	0.181	Reliable
A9: The TACL test takes too long to administer	0.85	0.95	0.112	Reliable
A10: The quality of the images in the TEPROSIF-R test is good	0.85	0.95	0.294	Reliable
A11: The size of the images in the TEPROSIF-R test is adequate	0.85	0.95	0.363	Reliable
A12: The presentation format in the TEPROSIF-R test is attractive to children	0.85	0.95	0.420	Reliable
A13: I rely on the results obtained with the TEPROSIF-R test	0.85	0.95	0.331	Reliable
A14: The content assessed in the TEPROSIF-R test is consistent	0.85	0.95	0.317	Reliable
A15: The presentation format of the TEPROSIF-R test seems adequate to me	0.85	0.95	0.370	Reliable
A16: The TEPROSIF-R test requires a digital version (a tablet or computer application)	0.85	0.95	0.464	Reliable
A17: The image content in the TEPROSIF-R test must be updated	0.85	0.95	0.055	Reliable
A18: The TEPROSIF-R test takes too long to administer	0.86	0.95	0.050	Reliable
A19: The quality of the images in the STSG test is good	0.85	0.95	0.381	Reliable
A20: The size of the images in the STSG test is adequate	0.85	0.95	0.348	Reliable
A21: The presentation format in the STSG test is attractive to children	0.85	0.95	0.396	Reliable
A22: I rely on the results obtained with the STSG test	0.85	0.95	0.213	Reliable
A23: The content assessed in the STSG test is consistent	0.85	0.95	0.313	Reliable
A24: The presentation format of the STSG test seems adequate to me	0.85	0.95	0.243	Reliable
A25: The STSG test requires a digital version (a tablet or computer application)	0.84	0.95	0.492	Reliable
A26: The image content in the STSG test must be updated	0.85	0.95	0.132	Reliable
A27: The STSG test takes too long to administer	0.85	0.95	-0.024	Reliable
A28: I believe it is appropriate to use a tablet in speech-language-hearing assessment.	0.85	0.95	0.461	Reliable
A29: I believe it is appropriate to use a computer in speech-language-hearing assessment.	0.84	0.95	0.577	Reliable
A30: I believe it is appropriate to use a tablet in the speech-language-hearing intervention	0.84	0.95	0.541	Reliable
A31: I believe it is appropriate to use a computer in the speech-language-hearing intervention	0.84	0.94	0.601	Reliable
A32: I rely on the effectiveness of using technology to assess children 0 to 2 years old.	0.85	0.95	0.336	Reliable
A33: I rely on the effectiveness of using technology to assess children 2 to 4 years old.	0.84	0.95	0.546	Reliable
A34: I rely on the effectiveness of using technology to assess children 4 to 6 years old.	0.84	0.94	0.654	Reliable
A35: I rely on the effectiveness of using technology to assess children 6 years old or more	0.84	0.95	0.551	Reliable
A36: I rely on the effectiveness of using technology to conduct therapy in children 0 to 2 years old.	0.85	0.95	0.348	Reliable
A37: I rely on the effectiveness of using technology to conduct therapy in children 2 to 4 years old.	0.84	0.95	0.559	Reliable
A38: I rely on the effectiveness of using technology to conduct therapy in children 4 to 6 years old.	0.84	0.94	0.653	Reliable
A39: I rely on the effectiveness of using technology to conduct therapy in children 6 years old or more	0.84	0.95	0.572	Reliable
A40: I get better results in the assessment process by using technology	0.85	0.95	0.388	Reliable
A41: I get better results in therapy by using technology	0.84	0.95	0.465	Reliable
A42: I thoroughly know the current regulations that rule how speech-language-hearing therapists should proceed in the field of education	0.86	0.95	-0.040	Reliable
A43: I know the current regulations that rule how speech-language-hearing therapists should proceed in the field of education thanks to the training I had at university	0.85	0.95	0.026	Reliable
A44: I know the current regulations that rule how speech-language-hearing therapists should proceed in the field of education because I learned them on my own.	0.85	0.95	0.207	Reliable
A45: The speech-language-hearing assessment instruments proposed in the current regulations are sensitive and adequate	0.84	0.95	0.447	Reliable
A46: Conducting speech-language-hearing diagnoses in the educational contexts requires complementing the assessment with tests that are not in the current regulations	0.85	0.95	-0.015	Reliable
A47: Using technology in the educational/therapeutic context is both efficient and appropriate	0.85	0.95	0.446	Reliable

• Source: Developed by the authors
• Caption: CHI = Corrected Homogeneity Index

On the other hand, the CHI analysis revealed that it yielded values lower than 0.2 for items A8, A9, A17, A18, A26, A27, A42, A43, and A46. This suggests that they should be reassessed, as they lacked correlation with the overall score of the questionnaire. Moreover, a sensitivity analysis, excluding the said questions and considering only 37 items in the questionnaire, revealed that the Cronbach’s α coefficient increased from 0.85 to 0.88, the Guttman’s λ₆ coefficient increased from 0.95 to 0.96, and the CHI for all items was higher than 0.2, maintaining the stability of the questionnaire.

Discussion

There is no questionnaire in Chile to measure the perception of SLH professionals regarding the use of ICTs to understand their assessment and intervention processes using technological tools. However, this study objective of assessing the ICT questionnaire content validity and reliability may help understand the use and knowledge of ICTs in professional practice.

Given the above, concerning the discussion of results in the question/response validation process, the experts suggested reformulating the knowledge questions in module 3. Firstly, they pointed out that the response options in the questions do not seem to be the most adequate because the usual responses are “Yes”, “No”, and “Do not know/Not responded”. Secondly, the questions themselves had clarity problems with the term “knowledge”, concerning the current regulations of the consulted instruments.

Moreover, the study found that the internal consistency of the ICT questionnaire is adequate - i.e., it is reliable because the Cronbach’s α coefficient value of 0.85 was significant (p<0.005). A similar situation occurred with Guttman’s λ₆ coefficient, which was 0.95. Although there is no adequate value for reliability coefficients, at least in Streiner²⁸ different values are proposed according to the objective of the questionnaire. In this case, its objective is to learn the perception of professionals who work in different contexts related to the use and knowledge of the ICTs. Therefore, the authors indicate that values higher than 0.70 are adequate and indicate its reliability.

As for the CHI results, nine questions had values lower than the criteria set by the theory (<0.2). A more in-depth analysis of items A8, A9, A17, A18, A26, A27, which were meant to measure some characteristics of TACL, TEPROSIF-R, and STSG, revealed that they tended to gather information associated with their use rather than their characteristics. A suggestion for the second version is to separate the questions associated with the use from those related to the characteristics of the said assessment instruments.

Furthermore, the CHI suggested revising questions A42, A43, and A46, which belong to module 3, despite their high reliability coefficient scores. For instance, item A42: “I thoroughly know the current regulations that rule how SLH therapists should proceed in the field of education” suggests the response options “Yes”, “No”, and “Do not know/Not responded”, instead of the Likert-type presented as 1: “Strongly disagree” and 5: “Strongly agree”. In this scenario, the idea would be to reformulate the question or the response options. A similar situation occurred with question A43, which is quite like A42, as its response options are “Yes”, “No”, and “Do not know/Not responded”. Likewise, the suggestion is to reformulate either the question or the response options.

Lastly, concerning item A46, which asks about the knowledge of other instruments other than those under the current regulation, it coherently does not correlate with the other items, which explains the CHI values lower than 0.2.

Conclusion

The questionnaire to assess the perception of SLH professionals, regarding the use and knowledge of ICTs, to assist in the assessment and intervention processes, is valid and reliable. Specifically, concerning the content validity of the questions and response options with a panel of experts, no major semantic problems were found, in general. Nonetheless, they suggested some adjustments in the questions in module 3 associated with the professionals’ knowledge on the use of ICTs. In summary, it was suggested that in future research the investigation team assess module 3 and propose a second version of the ICT questionnaire to be administered nationwide.

REFERENCES

• ¹
  García Ubillo V, Herrera Lillo A, Sandoval Ramírez M. Léxico especializado para la iniciación fonoaudiológica. Valparaíso: Universidad de Playa Ancha; 2020.
• ²
  Torres F, Romero L, Álvarez C. Transferencia de aprendizajes desde el aula formal hacia la práctica clínica: una reflexión sobre la formación de fonoaudiólogos en Chile. Rev Chil Fonoaudiol. 2018;17:1-10.
• ³
  Vega YE, Torres AM, del Campo MN. Análisis del Rol del Fonoaudiólogo(a) en el Sector Salud en Chile. Cienc Trab. 2017;19(59):76-80.
• ⁴
  Torres RA, Vega RY, Del Campo RM. Autorreflexión sobre el desempeño profesional del fonoaudiólogo(a) en el sector educativo: una aproximación a la construcción del rol. Rev Chil Fonoaudiol. 2015;14:103-17.
• ⁵
  Gobierno de Chile. Decreto 1300. Aprueba planes y programas de estudios para alumnos con trastorno específico del lenguaje. Santiago: Ministerio de Educación; 2002.
• ⁶
  Escudero JC. Sistema escolar chileno: principales políticas públicas (1990-2017). QUAESTIO. 2020;22(1):117-44.
• ⁷
  Coloma CJ, Mendoza E, Carballo G. Desempeño gramatical y narrativo en niños con Trastorno Específico del Lenguaje. Círc. Lingüíst. Apl Comun. 2017;69:67-90.
• ⁸
  Pavez MM. Test para la Comprensión Auditiva del Lenguaje de E. Carrow aplicación en Chile. Santiago de Chile: Escuela de Fonoaudiología de la Universidad de Chile; 2004.
• ⁹
  Pavez MM, Maggiolo M, Coloma CJ. Test para evaluar procesos de simplificación fonológicas TEPROSIF-R. 3rd ed. Santiago de Chile: Ediciones Universidad Católica de Chile; 2008.
• ¹⁰
  Pavez MM. Test exploratorio de gramática española. 3rd ed. Santiago de Chile: Ediciones Universidad Católica de Chile; 2010.
• ¹¹
  Martínez R, Cisterna F. Las Tecnologías de Información y Comunicación en el desarrollo de la enseñanza del área trastornos del lenguaje, habla y deglución en el adulto, de la carrera de Fonoaudiología. Rev Educ Cienc Salud. 2015;12(1):29-34.
• ¹²
  Cidrim L, Magalhães Braga PH, Madeiro F. Desembaralhando: a mobile application for intervention in the problem of dyslexic children mirror writing. Rev. CEFAC. 2018;20(1):13-20.
• ¹³
  Dillon F, Freire I, Rojas D, Espinosa D. Educación y TIC: El uso de la teoría de redes en el análisis de relaciones sociales y académicas en contextos educativos. Rev Ibérica Sist E Tecnol Informação. 2020;E32:88-97.
• ¹⁴
  Laverde Robayo DM. Reflexiones sobre la utilidad de la realidad virtual en la práctica fonoaudiológica. Rev Colomb Rehabil. 2014;13(1):26-33.
• ¹⁵
  Grande M, Cañón R, Cantón I. Tecnologías de la información y la comunicación: Evolución del concepto y características. IJERI. 2016;(6):218-30.
• ¹⁶
  Rodríguez Y, Jiménez G. Competencia digital en Fonoaudiología: retos de formación profesional desde los avances tecnológicos. AELFA-IF. 2019;39(4):192-200.
• ¹⁷
  Taherdoost H. Validity and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research. IJARM. 2016;5(3):28-36.
• ¹⁸
  Lima MMO de, Cordeiro AA de A, Queiroga BAM de. Developmental Stuttering Screening Instrument: development and content validation. Rev. CEFAC. 2020;23(1):e9520.
• ¹⁹
  Groves R, Fowler F, Couper M, Lepkowski J, Singer E, Tourangeau R. Survey Methodology. 2nd ed. Wiley; 2011.
• ²⁰
  Willis GB, Lessler JT. Question Appraisal System QAS-99 [Internet]. Rockville, MD: Research Triangle Institute; 1999 [cited 2021 May 3]. Available from: http://www.websm.org/uploadi/editor/1364216022Willis_Lessler_1999_QAS_99.pdf
• ²¹
  Hamui-Sutton A, Varela-Ruiz M. La técnica de grupos focales. Inv Ed Med. 2013;2(5):55-60.
• ²²
  Onwuegbuzie AJ, Leech NL, Dickinson WB, Zoran AG. Un marco cualitativo para la recolección y análisis de datos en la investigación basada en grupos focales. Paradigmas. 2011;3(2):127-57.
• ²³
  Bernal-García MI, Salamanca Jiménez DR, Perez Gutiérrez N, Quemba Mesa MP. Validez de contenido por juicio de expertos de un instrumento para medir percepciones físico-emocionales en la práctica de disección anatómica. Educ Med. 2020;21(6):349-56.
• ²⁴
  Kenny AJ. Interaction in cyberspace: an online focus group. JAN. 2005;49(4):414-22.
• ²⁵
  Bonett DG, Wright TA. Cronbach's alpha reliability: Interval estimation, hypothesis testing, and sample size planning. J Organiz Behav. 2015;36(1):3-15.
• ²⁶
  Carmines EG, Zeller RA. Reliability and Validity Assessment (Sage University Paper Series on Quantitative Applications in the Social Sciences). Newbury Park, CA: SAGE; 1979.
• ²⁷
  Villasís-Keever MÁ, Márquez-González H, Zurita-Cruz JN, Miranda-Novales G, Escamilla-Núñez A, Villasís-Keever MÁ et al. El protocolo de investigación VII. Validez y confiabilidad de las mediciones. Rev Alerg Mex. 2018;65(4):414-21.
• ²⁸
  Streiner DL. Starting at the Beginning: An Introduction to Coefficient Alpha and Internal Consistency. J Pers Assess. 2010;80(1):99-103.

Publication Dates

• Publication in this collection
  03 Dec 2021
• Date of issue
  2021

History

• Received
  04 Aug 2021
• Accepted
  25 Oct 2021