Essential aspects in the design of data collection instruments in primary health research

ABSTRACT

Objective:

to investigate the literature production on the design of forms for research in the health area and describe the most relevant concepts and precepts of the topic.

Methods:

an integrative literature review in the PubMed and Scielo databases with the key words: survey, constructing, questionnaire, formulary, development and design in various combinations, including articles published in any language in the last ten years. The survey returned 1,480 articles, and after reading and critically reviewing the abstracts according to the objective of the study, 16 articles were selected for complete reading. Information regarding aspects that were most relevant to the objective of the study was analyzed, as well as its recurrence in the selected articles.

Results:

the reading of the 16 articles resulted in three categories, based on the recurrence of the themes: structure, validation and sampling.

Conclusion:

clarity in the formulation of the questions was the most valued aspect in the structure of the instrument. As for validation, the realization of pilot tests was considered fundamental. Finally, the method of administration and adaptation of the questionnaire to target population was considered fundamental.

Keywords:
Health Surveys; Surveys and Questionnaires; Methodology; Review

RESUMO

Objetivo:

verificar a literatura sobre a construção de formulários de pesquisas em saúde e descrever os conceitos e preceitos mais relevantes do tema.

Métodos:

revisão integrativa da literatura realizada nas bases Pubmed e Scielo com as palavras-chave: survey, constructing, questionnaire, formulary, development e design em diversas combinações, incluindo artigos em qualquer língua e publicados nos últimos dez anos. A pesquisa retornou 1480 artigos e após leitura e análise crítica dos resumos de acordo com o objetivo, 16 estudos foram selecionados para leitura na íntegra. Foi realizada a análise das informações dos aspectos mais relevantes para o objetivo do estudo e a sua recorrência nas pesquisas selecionadas.

Resultados:

dentre os 16 artigos foram encontradas três categorias baseadas na recorrência dos temas: estrutura, validação e amostragem.

Conclusão:

a clareza na formulação das perguntas foi o aspecto mais valorizado em relação à estrutura do instrumento, enquanto que para a validação, é fundamental realizar o teste-piloto; e ainda, a forma de administração e a adaptação do questionário deve ser considerada para a população à qual se aplica.

Descritores:
Inquéritos Epidemiológicos; Inquéritos e Questionários; Metodologia; Revisão

Introduction

Health research involving human beings consists of generating new knowledge through the use of the scientific method to identify and deal with health problems. Knowledge acquired through research is considered a global public good¹. The Committee for Health Research and Development, created in 1987 based on the premise of comprehensive analysis of health conditions and health research, pointed out that research is essential for health actions and health promotion, and it is also necessary to contribute to new ideas and alternative interventions².

In Latin America, health research is concentrated in some countries, that happen to be the largest economies: Mexico, Chile, Brazil and Argentina. These countries contribute with 90% of the total Latin American investment in research and development³. Among Latin American countries, Brazil has a high production in the areas of public health, biotechnology and pharmacy. Over the past two decades, the focus on reducing poverty, child and maternal mortality, and combating HIV, malaria and other diseases has given impetus to improvements in health care and investments in this area. The understanding is that investments in health research have a direct effect on meeting the aforementioned goals³.

Scientific research requires extensive planning, human and material resources, time and effort. The application of instruments such as questionnaires and forms is an effective way to collect population data with relative logistical practicality.

Health research instruments are essential tools to gather information from individuals thatrepresent a given population since health research has diverse objectives, from those focused on experiments (e.g. search for a new drug treatment, a new intervention), to those of observational nature (e.g. assessment of risk factors, assessment of quality of health care, description of indicators). Each objective of health research should be interconnected to the feasible dimensions. For example, the study of patient perceptions regarding health treatments is an important tool to identify strengths, weaknesses and unmet needs in health services as well as to identify changes in patient satisfaction over time^4,5.

For the development of scientific research with primary sources, it is necessary to formulate clear and functional instruments during the planning stage of the study. They should contemplate questions that will originate variables to enable a response directly associated with the objectives of the study. In addition to assessing what they are proposing, they should have stability, i.e., they should present similar results in test and retest when subjects are in the same health condition⁶. Some errors are recurrent in this field; some of them are related to the layout of the instrument: too small font size; too long instrument; selection of an opportunistic sampling that excludes certain groups that would be of interest to the research⁷.

The design of an instrument is an extremely important aspect to ensure that the data will be accurately collected and the results will be interpretable and generalizable⁶. The form of application (postal delivery, telephone, face-to-face or the Internet), taking into account the particularities of the study design⁶ appropriately targeted to the study population, as well as validation, are also questions to be evaluated for the creation of effective and efficient instruments. All this must be taken into account, since a biased tool sent to an unrepresentative sample will not provide useful information, but rather distort the possible actual results of the research.

The reality of health research, especially of observational nature, is that studies do not always present adequate rigor for the creation of research instruments, this fact is observed in publications of studies whose data collection tools are not validated and/or their interpretations are not clear, leading to unintelligible data or erroneous conclusions⁸.

Given the importance of planning in research, the objective of this study is to survey the literature on the design of health research forms and describe the most relevant concepts and precepts of the theme.

Methods

Thisis a study with data collection from secondary sources, through an integrative review of the literature performed in the Scielo and Pubmed databases. The keywords "survey", "constructing", "questionnaire", "formulary", "development" and "design" were used, resulting in 1,480 indexed articles with selection filter set for articles published from 2006 to 2016, in any language, and without design restriction. After reading and critically analyzing the titles and abstracts, taking into account the proximity to the objective of the present research, the articles that did not present in the description the context of design of questionnaires/instruments/forms and/or validation studies were excluded. At the end, 16 articles were selected. The selected articles were read in full length, and the information on journal and year of publication, objective, type of study involving the elaboration of the research instrument, aspects considered for the elaboration of the research instrument, bias of assessment, attention to the application of a pilot study to test the instrument of data collection, and influence of the type of population onthe application of instruments, were factors to be considered, among others. The content was categorized according to the aspects most relevant to the nature of this work and the frequency in which it appeared in the selected literature. Methodological and theoretical issues associated with the preparationof a health research instrument were evaluated and a list of guidelines for decision making at each stage of such development were presented (Figure 1).

Figure 1:
Summary of search strategies in Pubmed and Scielo

Literature Review

Main Findings

Among the 16 selected articles, three distinct categories were found based on the recurrence of the themes: structure, validation and sampling (population studied). Aspects related to the structure were described in 10 articles (62.5%); those related to the population (subjects of the research/sampling/form of administration) appeared in 10 (62.5%); aspects related to validation appeared in 8 articles (50%), with emphasis on pilot studies that were recurrent in 75% of articles that addressed validity. Each aspect wasthen discussed and the most relevant concepts were highlighted.

Structure

Before preparing a new instrument, it is recommended that a literature review be done to verify if there is no validated instrument for the theme in question; if a data collection tool (research tool) needs to be created, a search in the literature for the recommended scale and items (categories) more relevant to respond to the purpose of the study should be done. In quantitative studies, once the main research items are defined, the questions should be prepared so as to provide variables on each of these items, whereas in qualitative studies they should provide answers that are coherent with the guiding question without the need for systematized items^7,8.

An instrument with an adequate structure should consider the type of question (objective, subjective), the language used, which must be appropriate for the studied population and the order of the items. Questions should be formulated so as to ensure that each is easy to understand, free from information bias, and appropriate to the level of education and culture of the participants^7-9.

Given that the order of the questions influences the answers, the literature indicates that questions should be organized from the easiest to the most difficult; from the general to the specific; from the factual to the abstract. Clinical and demographic data should be left to the end of the instrument because they are easier and potentially intrusive^5,6,9,10. Controversial or emotive items should not be placed at the beginning of the instrument^5,9.

In addition to the questions, researchers should consider the best response format (open, closed, dichotomous, multiple choice, ordinal, among others). In cases of scale, the choice should be designed to contribute positively to data collection and to the purpose of the study^7,11.

Closed questions are easier to analyze and apply, but the downside is that they reduce the number of possible responses. This way of questioning can provide summary information and reduce the misinterpretation bias of what was written by the participants. In turn, open questions allow respondents to put their own answers that increase the range of possible answers and of individuality. With these type of questions, researchers can often measure the importance of a particular subject better, but the analysis and comparison is challenging. In general, an effort needs to be put to the formulation of objective questions, while in the case of open questions, the effort will be concentrated on data interpretation; the fact is that the choice for either of the options will depend on the objective and the study design^5,6,8,10.

Validation

Validity is the degree of congruence between what the instrument proposes to measure and what it actually measures; a valid tool should question what the researcher wants to ask and the questions should be formulated in such a way that the interviewee understands the purpose of the question.

There are four different dimensions in the validity of an instrument: apparent validity (logical dimension), content validity (face), concept or construct validity, and criterion validity¹².

Apparent validity (or logical dimension) refers to the extent to which it seems that the instrument measures what it proposes to measure in the view of specialists and of the subjects themselves. This validity must be verified at the time of structuring the instrument, because when the issues do not have logical validity, it is highly likely that individuals will refuse to answer the questions. However, in specific cases, in order to achieve apparent validity, the instrument must undergo a pilot test with the target population (e.g. if the target respondents are diabetic patients, so one of these respondents should comment on his understanding of the instrument). Any uncertainties and doubts should be clarified until the question becomes easy to understand. When instruments are extensively tested and correctly calibrated for their target, they can be considered accurate^5,6. In some cases, it may be interesting to formulate questions without logical validity, because in sensitive or controversial issues, the use of questions with too much apparent validity may cause the subjects decide not to respond or distort their response¹².

Content validity (or face validity) refers to the opinion of the expert on whether the items in the scale represent the domains or concepts proposed for the research. It is therefore considered that an instrument is valid if it considers all aspects related to the concept studied. For example, a teacher gives a course consisting of five classes, each with a different content, and he decides to evaluate how much the students learned through a test. In order for the content to be valid, the testmust contain a set of questions that is representative of the contents given in all the five classes, with the same number of questions about each class¹³. This dimension of validity is related to the composition of an instrument; it evaluates whether it contains a representative sample of the components of the construct to be measured. This validation involves the systematic analysis of the contents of the measuring instrument to determine whether the questions are relevant and the essential characteristics of the construct are represented in the appropriate proportions. Content validity assessment is based on judgments from different sources (review of medical literature, expert opinion, pilot studies)^8,9.

The validity of a construct (or concept validity) evaluates the extent to which the instrument reflects the theory of the phenomenon or of the concept to be measured. It ensures that the measurements resulting from the responses can be considered and used as a measure of the phenomenon studied. The measurement of the theoretical concepts requires the prior identification of the content of the instruments that they used and the elaboration of a conceptual model to help interpret the obtained results. That is, if a test that intends to measure speech changes, whether phonetic or phonological deviations, effectively measures such changes, as proposed by TERDAF¹⁴ or it has concept validity. Factor analysis is a statistical technique that can be used to determine the constructs or domains in the developing measure, that is, it groups responses according to the underlying factors^9,12.

Criterion validity is the type of validity that is commonly referred to when validating an instrument. It involves the following steps: identifying a relevant and reliable external criterion; obtaining a representative sample of the subject for the population in which the instrument will be used; applying the instrument and obtaining a score for each subject, and evaluating each of the individuals with the external reference value¹². For example, for an instrument that evaluates intelligence, a relevant criterion could be school performance¹³.

A valid instrument, whether questionnaire or form, should be simple, feasible and acceptable to patients, users and researchers (viability); reliable and accurate (reliability); appropriate to the research problem (content validity); reflecting the theory underlying the phenomenon or concept to be measured (construct validity); and able to measure changes, both in different individuals and in the responses of a single individual over time (sensitivity to changes)^8,12.

Sampling (research subjects/form of administration)

Once the research question is defined, the target population must be established⁷, which can have people, houses, organizations or other aggregates as a unit, in order to enable the researchers to achieve the planned objectives, including the design and application of the instrument. After this step, and according to the study design and structure of the research, the sampling method should be defined.

As for data collection, the research tools can be self-administered or interviews can be applied via facilitator (interviewer). Self-administered instruments are more widely used. They must have clear instructions and questions and follow a logical order to ensure a high response rate, which is lowerin this method when compared to other ones. However, respondents are more likely to respond sincerely without the presence of an interviewer. On the other hand, when an instrument is applied by an interviewer, the questions canbe more complex, as they can be clarified. However, the presence of an interviewer may pressure respondents to give what they deem to be "appropriate" rather than true answers. Furthermore, this last method consumes much more time and resources. The form of administration must be adapted to the target population, to the research resource structure, and to what is intended to be achieved with the application of the instrument^5,6.

Poor data collection is a barrier to high-quality research is¹⁵ because no statistical technique, no matter howsophisticated it is, can correct data whose collection was poorly planned. The data collected must be truly capable of responding to the question of interest and all relevant information must be recorded in a way that allows it to be easily used for further analysis. Poorly designed forms often have high levels of missing data. An obvious consequence is that this reduces the number of individuals that can be included in the study and consequently the possibility of an effect being detected, if it exists. The scriptfor collecting secondary data (for example, searching for information in medical records) should also be systematized and, as far as possible, follow the aspects described above.

Figure 2 describes the main themes described in each integrative review study.

Figure 2:
Description of the main findings of each article of the integrative review

Conclusion

As for the structure, keeping the clarity in the formulation of the questions was the most valued aspect, while in validation, the realization of pilot tests stood out. As for the population, the way of administration (interview, the internet, among others) and the adaptation of the instrument to that sample of subjects were outstanding aspects. Proper planning of the health research form can minimize possible biases and errors in the conduction of the scientific studies.

References

• ¹
  Canario JA, Lizardo J, Espinal R, Colomé M. Gaps in health research in the Dominican Republic. Rev Panam Salud Publica Pan Am J Public Health. 2016;39(4):179-85.
• ²
  Nuyens Y. No development without research a challenge for research capacity strengthening. Genève (Suisse): Global Forum of Health; 2005.
• ³
  Moloney A. Latin America faces hurdles in health research. The Lancet. 2009;374(9695):1053-4.
• ⁴
  Dell-Kuster S, Sanjuan E, Todorov A, Weber H, Heberer M, Rosenthal R. Designing questionnaires: healthcare survey to compare two different response scales. BMC Med Res Methodol [Internet]. 2014 Dec [cited 2018 Feb 25];14(1). Available from: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-14-96
  » http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-14-96
• ⁵
  Jones R. Development of a Questionnaire and Cross-Sectional Survey of Patient eHealth Readiness and eHealth Inequalities. Med 20. 2013;2(2):e9.
• ⁶
  Jenn NC. Designing A Questionnaire. Malays Fam Physician Off J Acad Fam Physicians Malays. 2006;1(1):32-5.
• ⁷
  Thwaites Bee D, Murdoch-Eaton D. Questionnaire design: the good, the bad and the pitfalls. Arch Dis Child - Educ Pract Ed. 2016;101(4):210-2.
• ⁸
  Rattray J, Jones MC. Essential elements of questionnaire design and development. J Clin Nurs. 2007;16(2):234-43.
• ⁹
  Bryson GL, Turgeon AF, Choi PT. The science of opinion: survey methods in research. Can J Anesth Can Anesth. 2012;59(8):736-42.
• ¹⁰
  Kazi AM, Khalid W. Questionnaire designing and validation. JPMA J Pak Med Assoc. 2012;62(5):514-6.
• ¹¹
  Song Y, Son Y-J, Oh D. Methodological issues in Questionnaire Design. J Korean Acad Nurs. 2015;45(3):323.
• ¹²
  Yébenes Prous MJG, Rodríguez Salvanés F, Carmona Ortells L. Validación de cuestionarios. Reumatol Clínica. 2009;5(4):171-7.
• ¹³
  Hutz CS, Bandeira DR, Trentini CM (ed). Psicometria. Porto Alegre: Artmed Editora; 2015.
• ¹⁴
  Goulart BNG de, Ferreira J. Speech disorder screening test for children. Pro-Fono R Atual. Cientif. 2009;21(3):231-6.
• ¹⁵
  Smith CJ. Designing a research study. Phlebol J Venous Dis. 2011;26(8):369-70.
• ¹⁶
  Doody O, Bailey ME. Setting a research question, aim and objective. Nurse Res. 2016;23(4):19-23.
• ¹⁷
  Jones T, Baxter M, Khanduja V. A quick guide to survey research. Ann R Coll Surg Engl. 2013;95(1):5-7.
• ¹⁸
  Bobrovitz N, Santana MJ, Ball CG, Kortbeek J, Stelfox HT. The development and testing of a survey to measure patient and family experiences with injury care. J Trauma Acute Care Surg. 2012;73(5):1332-9.
• ¹⁹
  Slattery EL, Voelker CCJ, Nussenbaum B, Rich JT, Paniello RC, Neely JG. A practical guide to surveys and questionnaires. Otolaryngol-Head Neck Surg. 2011;144(6):831-7.
• ²⁰
  Pittman J, Bakas T. Measurement and instrument design. J Wound Ostomy Continence Nurs. 2010;37(6):603-7.
• ²¹
  Burford B, Hesketh A, Wakeling J, Bagnall G, Colthart I, Illing J et al. Asking the right questions and getting meaningful responses: 12 tips on developing and administering a questionnaire survey for healthcare professionals. Med Teach. 2009;31(3):207-11.

• 1
  Research support source: National Council for Scientific and Technological Development - CNPq and Foundation for Research Support of the State of Rio Grande do Sul - FAPERGS.

Publication Dates

• Publication in this collection
  Sep-Oct 2018

History

• Received
  05 Mar 2018
• Accepted
  11 Sept 2018