Content and usability validation of the Retire with Health web software

Validación de contenido y usabilidad del web software Jubilarse con Salud

ABSTRACT

Objectives:

to validate the content and usability of the Retire with Health web software.

Methods:

a methodological study. The content validation was carried out by seven experts in the aging subject, using an instrument that evaluated the objectives, contents, relevance, and the environment of the web software. Usability was validated by seven experts in software development, referring to ergonomics, functionality, usability, and efficiency. The data were submitted to statistical analysis, described by calculating the Content Validity Index.

Results:

the global Content Validity Index found 0.97 for the usability aspects and 1.00 for the content. After incorporating all the experts’ suggestions, the Retire with Health web software was made available on the internet (www.aposentarsecomsaude.com.br).

Conclusions:

content validation and usability showed that the web software meets the objective of supporting reflection on retirement, with a high level of agreement among experts.

Descriptors:
Validation Study; Software; Hypermedia; Occupational Health; Retirement

RESUMEN

Objetivos:

validar el contenido y usabilidad del web software Jubilarse con Salud.

Métodos:

estudio metodológico. La validación de contenido fue realizada por siete especialistas del área de envejecimiento mediante instrumento que evaluó objetivos, contenidos, relevancia y ambiente del web software. La usabilidad fue validada por siete especialistas en desarrollo de software, que observaron ergonomía, funcionalidad, usabilidad y eficiencia. Datos sometidos a análisis estadístico, descriptos mediante cálculo del Índice de Validez de Contenido.

Resultados:

el Índice de Validez de Contenido global fue de 0,97 para los aspectos de usabilidad y de 1,00 para el contenido. Agregadas todas las sugerencias de los especialistas, el web software Jubilarse con Salud fue puesto a disposición en Internet (www.aposentarsecomsaude.com.br).

Conclusiones:

la validación de contenido y de usabilidad demostró que el web software satisface el objetivo de apoyar la reflexión sobre la jubilación, con elevado nivel de concordancia entre los especialistas.

Descriptores:
Estudio de Validación; Programas Informáticos; Hipermedia; Salud Laboral; Jubilación

RESUMO

Objetivos:

validar o conteúdo e a usabilidade do web software Aposentar-se com Saúde.

Métodos:

estudo metodológico. A validação de conteúdo foi realizada por sete experts da área de envelhecimento, por meio de instrumento que avaliou os objetivos, conteúdos, relevância e o ambiente do web software. A usabilidade foi validada por sete experts em desenvolvimento de software, quanto à ergonomia, funcionalidade, usabilidade e eficiência. Os dados foram submetidos à análise estatística, descritos por meio do cálculo do Índice de Validade de Conteúdo.

Resultados:

o Índice de Validade de Conteúdo global foi de 0,97 para os aspectos de usabilidade e de 1,00 para o conteúdo. Após incorporar todas as sugestões dos experts, o web software Aposentar-se com Saúde foi disponibilizado na internet (www.aposentarsecomsaude.com.br).

Conclusões:

Validação de conteúdo e usabilidade demonstraram que o web software atende ao objetivo de apoiar a reflexão sobre a aposentadoria, com elevado nível de concordância entre os experts.

Descritores:
Estudo de Validação; Software ; Hipermídia; Saúde do Trabalhador; Aposentadoria

INTRODUCTION

In recent years, there has been an intense development of software in the health and nursing areas. Facing the creation of these new technologies, the evaluation process becomes crucial, as a tool to guarantee the quality and adequate performance of the product offered to end-users⁽¹⁾.

The availability of validated and targeted software as a tool to support the retirement planning process is shown as a strategy that allows the dissemination of information to professionals both in their work environment and during leisure. In this way, it encourages workers to reflect on the value of preparing for retirement and the implications of their decision-making.

In this sense, it is worth noting that retirement can be understood from different perspectives, for some it is an expected moment, while others fear the changes and challenges to be faced. The decision to retire proves to be complex and influenced by several factors, which include considerations of health, family, work, leisure opportunities and income after work cut off. This is one of the most significant changes for someone^(2-4).

The Theory of Planned Behavior can be understood as a relevant conceptual model for understanding the specificities of human behavior. It assumes that behaviors are always preceded by intentions, which can be materialized or abandoned as the person creates his/her life history⁽⁵⁾, besides being controlled as individuals obtain information on them^(6-7).

This theoretical framework proves to be adequate to analyze retirement behavior and contribute to the creation of contents and resources necessary for planning retirement. The lack of planning can negatively influence adaptation to this stage of life, since getting ready for it contributes to well-being, not only instantly, but in the post-retirement period, which reinforces the need of managers to encourage the involvement of workers with actions to prepare for retirement⁽⁸⁾.

Thus, the use of web software that makes content available for thinking on retirement becomes an innovative strategy. It is understood by web software, an application that allows access through a web browser, hosted on a server, with extra functions, such as interactivity with the user, collecting and storing data⁽⁹⁾.

In this sense, the Retire with Health web software was developed by a team of nurse researchers, to collect and process data related to retirement planning, making available to users the possibility of carrying out a current diagnosis of the most fragile areas, by filling out the Inventory of Retirement Resources⁽¹⁰⁾ and then making available materials that instigate them to prepare for dismissal⁽¹¹⁾. Based on the theory of planned behavior, it is guided by planning the analysis of physical, financial, social, motivational, and emotional resources and assumes the support to facilitate the accomplishment of such behavior⁽⁶⁾.

Considering the potential contribution of the Retire with Health web software, as one of the tools to support the retirement planning process, it is required to validate the adequacy of its contents and usability. The validation of software becomes essential to assure the quality and check the consistency with the objectives and design principles proposed in its initial scope⁽¹²⁾.

OBJECTIVES

To validate the content and usability of the Retire with Health web software.

METHODS

Ethical aspects

Following the ethical precepts of Resolution 466/12, the study was carried out only after approval by the Research Ethics Committee Involving Human Beings, on May 13^th, 2016. The experts who agreed to participate in the study received the Informed Consent Term (ICF), having access to the content and usability validation instrument only after reading, acknowledging, and accepting the term, ensuring anonymity throughout the process.

Study design

An applied methodological study, aimed at validating the content and usability of the Retire with Health web software, prepared according to the assumptions of the Theory of Planned Behavior⁽⁶⁾.

Methodological procedures

The validation process of a new technology aims to promote the improvement of its functions and to ensure that the contents and tools used are adequate to the scope. In this sense, the experts assessed the support materials made available on the Retire with Health web software, after completing the Retirement Resources Inventory (RSI), which include texts created from the scientific literature, reports, video-lectures. recorded, all free access, categorized in Social; Physical; Financial; Emotional, Cognitive, and Motivational Aspects.

By analyzing the Lattes Platform curricula, seven experts in aging were invited. Those who met the criteria of a minimum degree of Doctor of nursing, health sciences and psychology were selected; with experience of orientation or carrying out researches related to aging and/or retirement, participation in groups or projects on to the theme. Then, the e-mail of the experts selected in the curricula, websites of the work institutions and published scientific articles were searched.

Along with content validation, the web software usability validation was carried out by seven experts in software development, identified through analysis of their curricula submitted to the Lattes Platform. Those who met the criteria for minimum titling of specialists in computer science, computer engineering with previous professional experience in software development were selected. Then, the e-mail of the experts selected in the curricula, websites of the work Institutions and published scientific articles was searched.

It is emphasized that because it is a content validation and usability of the new software process, there is no minimum sample of experts indicated, however, instrument validation theorists recommend using 6 to 20 judges⁽¹³⁾. Thus, in the present study 7 experts were included in the study for content analysis and 7 for usability. It was necessary to carry out only one evaluation set, because of the high score of agreement obtained both in content and usability validation.

Study scenario

The study scenario was composed of seven experts in aging, invited to validate the content of the Retire with Health web software and seven experts in software development, who carried out the usability validation.

Data collection and organization

The validation was carried out from May to July 2018, from the perspective of experts working in aging and the software development area.

The experts were contacted through email, with the invitation and the link to access the electronic form made available on the Google Forms^® platform, from which they were directed to the Informed Consent Form for Content and Technicians Experts, and only after accepting it, they accessed the validation instrument. A period of fifteen days was established for the expert to respond.

The content validation instrument was adapted from Sousa⁽¹⁴⁾, consisting of two parts: the first, comprehending the characterization of the experts, and the second included 13 questions regarding the validation of the web software content from four dimensions: objectives (three questions), content (six questions), relevance (two questions) and environment (two questions). These were scored by a Likert scale, ranging from 1 to 3 points, from totally inadequate (one point), somehow appropriate (two points), and totally adequate (three points).

Likewise, the web software usability validation instrument was also structured in two sections. The first, with the experts’ characterization and the second with 14 questions on the validation of the usability aspects of the web software, distributed in four dimensions: ergonomics (six questions), functionality (three questions), usability (three questions) and efficiency (two questions), also scored by a three-point Likert scale: totally inadequate (one point), somehow adequate (two points) and totally adequate (three points).

At the end of the content and usability validation instrument, a field was made available to the experts to insert suggestions for improving the web software.

Data analysis

The data from the content and usability validation instruments were inputted in an Excel® spreadsheet and subjected to statistical analysis using the Statistical Package for Social Sciences® (SPSS) version 20.0. The percentage of individual agreement attributed by the experts was calculated for each item of the validation instruments, using the following formula:

$\%\mathrm{of}\mathrm{agreement}=\frac{\mathrm{number}\mathrm{of}\exp \mathrm{erts}\mathrm{who}\mathrm{agreed}}{\mathrm{total}\mathrm{number}\mathrm{of}\exp \mathrm{erts}}x100$

After descriptive analysis, the Content Validity Index (CVI) was calculated, which is indicated to estimate the agreement of experts on different aspects or items assessed by a given instrument, by using the following formula:

$\mathrm{CVI}=\frac{\mathrm{number}\mathrm{of}\mathrm{responses}"\mathrm{somehow}\mathrm{adequate}"\mathrm{and}"\mathrm{totally}\mathrm{adequate}"}{\mathrm{total}\mathrm{number}\mathrm{of}\mathrm{responses}}$

The CVI of the dimensions of the content and usability validation instrument was also calculated, by adding the individual CVI of each item of the dimension and divided by the number of items in that dimension. Finally, the global CVI was found by adding the individual CVI, divided by the total items of the content and usability validation instrument⁽¹⁵⁾.

As recommended in the literature⁽¹⁶⁾, a CVI score equal to or greater than 0.78 was considered acceptable. Even for items with a score higher than 0.78, it was decided to incorporate all the suggestions made by the experts and the final version of the web software to support retirement planning was achieved.

RESULTS

From the content validation, a total agreement of 76.9% was reached among the experts who considered the contents of the Retire with Health web software totally adequate, while 21.3% considered the contents somehow adequate and none of the experts considered them totally inadequate. The experts’ answers reached a Content Validity Index of 1.00, both for the total aspects and for each of the dimensions of the web software content validation instrument, as shown in Table 1.

Table 1
Percentage of agreement among content validation experts regarding the Retire with Health web software, Brazil, 2018

Validation Items	Percentage of agreement			CVI*
	Totally Inadequate	Somehow Adequate	Totally Adequate
	%	%	%
1 Objectives
1.1 The objectives of web software are consistent with the practice of retirement planning.	-	-	100	1.00
1.2 The web software allows the understanding of the retirement planning theme.	-	42.9	57.1	1.00
1.3 The proposed objectives are adequate for its effectiveness.	-	14.3	85.7	1.00
2 Content
2.1 The contents of the web software correspond to its objectives.	0	28.6	71.4	1.00
2.2 The contents of the web software are enough to achieve the objectives proposed.	0	28.6	71.4
2.3. The contents of the web software accurately reach the scope of the theme.	0	42.9	57.1	1.00
2.4 The information presented is correct.	0	0	100	1.00
2.5 The language is appropriate to the target audience's different levels of knowledge.	0	28.6	71.4	1.00
2.6 The contents allow the understanding of the different dimensions of retirement planning.	0	28.6	71.4	1.00
3 Relevance
3.1 The contents allow the understanding of the different retirement planning dimensions.	0	14.3	85.7	1.00
3.2 The contents of the web software are relevant so that the worker can identify the areas of greatest weakness concerning retirement planning, as well as seek strategies for better performance	0	28.6	71.4	1.00
4 Environment
4.1 The environment is suitable for presenting the content.	0	28.6	71.4	1.00
4.2 The environment is suitable for understanding the retirement planning theme.	0	14.3	85.7	1.00
Total	-	23.1	76.9	1.00

• Note:
• ^*
  Content Validity Index.

Even with high percentages of agreement and CVI considered adequate, greater than 0.78, all the suggestions and considerations of the experts were incorporated into the final version of the web software (Chart 1).

Chart 1
Suggestions and considerations of the content judges for the improvement of the Retire with Health web software, Brazil, 2018

Dimensions	Suggestions/Considerations	Decision
Objectives	Check the possibility of including other reasons for retirement, which may not occur due to time, but due to other needs (taking care of a family member, emotional exhaustion, due to illness/disability, etc.).	Content to pursue a reflection on the reasons for retiring was included. Available as part of the "Emotional, Cognitive and Motivational Aspect" icon, accessed after filling the Inventory of Resources for Retirement.
Contents	I suggest that the system save the answers to the questionnaire with the date, so that the user can enter and go directly to the supporting texts for each dimension. The option of recording the responses with the date would also allow the user to respond after a while and compare the results, noting whether he has advanced in his difficulties.	It was included the option "Login" for the system to save the users' responses according to the date of completion. Also, all responses are forwarded via email to the user, along with the Informed Consent Form, and the user can access the responses' history and scores by clicking on the login icon on the web software homepage.
	The page is well designed, publishes important, regular, and current information. However, I suggest publishing more information about pre-retirement and its planning in all aspects involved, such as psychological, social, health promotion and financial.	New content was added to the News screen, aimed at retirement planning.
Relevance	The contents are very relevant, very well written, "user friendly".	No adjustments were made to the relevance dimension.
Environment	Think about giving the option of not revealing the person's name in the experiences' statements if they do not want to be identified. Congratulations on the project. I was curious to know what web software about retirement was, and I liked it.	The identification option on the experiences screen was readjusted for non-mandatory filling.
	Even knowing that retirement and aging are closely related, I think they could also bring images of women aged 60 and over, but with a youthful look. It captured my attention the predominance of images of white-haired women in the web software.	The images of the web software were changed, to relate them to healthy aging.

As for the web software usability validation, performed by experts in software development, a total agreement of 78.6% was reached among the experts who considered the aspects analyzed totally adequate and 19.4% somehow, being that only 2.0% of the experts considered them totally inadequate. The total Content Validity Index was 0.97, the functionality, usability and efficiency dimensions presented a CVI of 1.00 (Table 2).

Table 2
Experts' usability validation regarding the Retire with Health web software, Brazil, 2018

Validation Items	Percentage of agreement			CVI*
	Totally Inadequate	Totally Inadequate	Totally Inadequate
	%	%	%
1 Ergonomics
1.1 The user can Exchange from one screen to another quickly	14.3	-	85.7	0.85
1.2 The placement of the data is kept consistently from one screen to another.	14.3	-	85.7	0.85
1.3 Texts and style resources (eg underline, bold, italics) are used properly.	-	57.1	42.9	1.00
1.4 Controls and commands are clearly differentiated from the information presented on the screens.	-	14.3	85.7	1.00
1.5 Selected items become highlighted from the others.	-	429	57.1	1.00
1.6 Error messages are concise and to the point.	-	286	71.4	1.00
2 Functionality
2.1 The web software is suitable for the proposals for which it is intended.	-	143	85.7	0.85
2.2 The web software performs the proposed functions correctly.	-	-	100	0.85
2.3 The web software favors to generate positive results.	-	-	100	1.00
3 Usability
3.1 The web software is easy to use.	-	286	71.4	1.00
3.2 It is easy to learn the concepts and applications of web software.	-	143	85.7	1.00
3.3 The web software allows the worker to easily apply its content.	-	286	71.4	1.00
4 Efficiency
4.1 The response time of the web software is suitable for the worker to access the content available on different screens.	-	-	100	1.00
4.2 The organization of themes in different screens of the web software is suitable for a good understanding of the content, as well as the easy location of the desired theme.	-	429	57.1	1.00
Total	2,0	194	78.6	0.97

• Note:
• ^*
  Content Validity Index.

The experts made suggestions and considerations only to improve the dimensions of ergonomics and efficiency. After analysis and discussion among the researchers, it was decided to incorporate all suggestions into the final version of the web software (Chart 2).

Chart 2
Suggestions and considerations of experts in software development to improve the usability aspects of the Retire with Health web software, Brazil, 2018

Dimensions	Suggestions/Considerations	Decision
Ergonomics	There are some coding errors on the pages.	Adjustments and tests were carried out to ensure proper working.
	Adjusting the icon that indicates the progress on the "Interview" screen. Use a "JS" to make the Input visible only when necessary. Example: if radio button equal to OTHER, visible Input.
Functionality	No suggestions were made.	No adjustments were made.
Usability	No suggestions were made.	No adjustments were made.
Efficiency	The great number of questions, inform how many questions make up the interview stage.	The information on the number of questions in the Sociodemographic and Occupational Questionnaire and Inventory of Resources for Retirement was inserted in the initial Interview screen was inserted.

After making the adjustments and incorporating the experts’ suggestions, the final version of the Retire with Health web software was obtained. This was made available to users for free, through the link www.aposentarsecomsaude.com.br.

DISCUSSION

The Retire with Health web software showed agreement percentages and satisfactory CVI for all evaluated items, both concerning content and usability aspects, which demonstrates its validity for use by workers who seek to prepare for the retirement.

It was observed that the experts considered the contents of the web software adequate with concerning the objectives, approach to the theme, relevance and environment, thus allowing users to be encouraged to reflect on the importance of retirement planning, from different perspectives, as social, emotional, financial, physical and others. Workers who adopt strategies to prepare for retirement in advance have a better ability to decide and adapt to the new life routine after leaving work⁽¹⁷⁾.

The materials made available on the web software refer to an expanded approach to preparing for retirement, which includes the need for financial preparation, family support, the influence of interpersonal relationships, physical and emotional well-being. Thus, they are based on the Theory of Planned Behavior assumptions, by integrating the reflection on the influence of the social environment, the recognition of the potential and challenges of such behavior and the ability to decide at the planned time⁽⁶⁾.

Still, according to this theory, for the individual to have greater chances of achieving a certain behavior, it is necessary to explore it and keep up to date⁽⁶⁾. Thus, the contents made available constitute a strategy to disseminate information and encourage individuals to deepen their reflection on retirement planning.

Based on the experts’ analysis, it was observed that the web software responds to the proposed objectives, with relevant content, which contributes to the complete retirement planning, presented in an attractive and appropriate environment for the target audience. These results underpin the importance of submitting a new technology developed for assessment by specialists in the area of interest and technological development, to avoid making it directly available to the target audience, before correcting possible flaws that would result in less adherence to the use of the tool⁽¹⁸⁾.

The validation of a new web software allows the inserted resources to be optimized and better applied, to guarantee the suitable function of disseminating relevant information to the users who access it⁽¹⁹⁾. Thus, when reaching high standards of agreement among experts in the field of aging, the ability of this tool to provide diagnostic tools and specific interventions directed to the weaknesses identified by workers at the time of access is observed.

The experts suggested adding to the content other reasons that lead the individual to retire, such as retirement due to illness, disability, or the need to care for a close person. At retirement, people need to use the time once committed to work in different ways, in some cases by a scheduled choice, but in others for unexpected reasons. All the possibilities that can lead to leaving work need to be reflected and understood by the individual, since they can affect mental and physical health in different ways, especially in post-retirement⁽²⁰⁾.

It is also worth mentioning the suggestion to adapt the system programming to save the responses of each user, which would allow both the individual and the researchers to follow the progression from the responses at different times of access to the web software, through the accumulated database. The retirement planning process must be continuous, monitored and adjusted according to the needs that can be changed over time, and participation in preparation program actions is necessary⁽²¹⁾.

It was observed that all items were positively evaluated in terms of functionality, usability, and efficiency. Such dimensions directly influence the quality of the web software, so by assessing them, the performance of this new technology can be improved⁽²²⁾.

The experts in software development suggested to correcting coding flaws, to enable the option of the icon that indicates the advance in the interview screen for the users and to enumerate questions in that same screen. When choosing to incorporate the suggestions, even in the face of high agreement scores, we sought to guarantee quality, proper functioning and make the web software attractive to end-users⁽¹⁾.

The strengths of the study are in the fact that the content experts and technicians carried out the entire validation process in an environment not controlled by the researchers, having as a guide the validation questionnaire, but with the freedom to access the web software wherever and whenever. When using this strategy, we sought to ensure that possible doubts and inconsistencies related to the functioning and contents were scored on the instrument, without interference from researchers.

It is considered that the agreement between the experts and the suggestions incorporated, that the web software Retire with Health contributes as a support tool for the retirement planning of workers from different professional areas, being essential to combine its use with other strategies considering the multifactorial nature of the decision to retire.

Limitations of the study

It stands out as the main limitation of this study the non-validation of web software from the perspective of the target population, since it was first chosen to promote the analysis of experts in technology development and research related to worker health and retirement planning.

Contributions to the field

This study adds to the scientific and technological advancement in the nursing and health field, by ensuring the availability of quality web software, given its validation by experts in the retirement planning and software development area. Moreover, to increasing the visibility of nursing, since both development and validation were coordinated by a team of nurse researchers, which demonstrates the ability of these professionals not only to use but to structure new technologies.

Still, the participation of nurses in the creation of these tools makes them even more appropriate to their objectives, since by outlining them from the initial scope, they can add relevant characteristics that contribute to the users’ adhesion, which might not be known by technical developers.

CONCLUSIONS

The content and usability validation demonstrated that the Retire with Health web software meets the objective of supporting the process of reflection on retirement, showing a high level of agreement among experts.

The improvement of the contents and aspects related to the functioning of the web software, based on the suggestions made by the experts, reinforces the concern in ensuring the availability of materials that contribute to the preparation for retirement and minimize possible failures in the performance of the web system. Thus, the web software becomes suitable to complement institutional actions so that the worker can plan retirement, once retiring results from a complex decision and influenced by several factors, using this strategy as a tool to support multiple approaches necessary for the preparation of the individual.

REFERENCES

• ¹
  Asadi F, Paydar S. Presenting an evaluation model of the trauma registry software. Int J Med Inform. 2018;112:99-103. doi: 10.1016/j.ijmedinf.2018.01.013
  » https://doi.org/10.1016/j.ijmedinf.2018.01.013
• ²
  Stynen D, Jansen NWH, Kant I. The impact of work-related and personal resources on older workers' fatigue, work enjoyment and retirement intentions over time. Ergonomic. 2017;60(12):1692-707. doi: 10.1080/00140139.2017.1334094
  » https://doi.org/10.1080/00140139.2017.1334094
• ³
  Noar ZHM. Knowledge on retirement related financial planning among employees. Revel Science [Internet]. 2018 [cited 2019 Feb 13];8(1):31-7. Available from: http://journals.iium.edu.my/revival/index.php/revival/article/view/219
  » http://journals.iium.edu.my/revival/index.php/revival/article/view/219
• ⁴
  Solem PE, Syse A, Furunes T, Mykletun RJ, Lange A, Schaufeli W, et al. To leave or not to leave: retirement intentions and retirement behaviour. Ageing Soc. 2016;36(2):259-81. doi: 10.1017/S0144686X14001135
  » https://doi.org/10.1017/S0144686X14001135
• ⁵
  Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50(1):179-211. doi: 10.1016/0749-5978(91)90020-T
  » https://doi.org/10.1016/0749-5978(91)90020-T
• ⁶
  Ajzen I. The theory of planned behaviour: reactions and reflections. Psychol Health. 2011;26(9):1113-27. doi: 10.1080/08870446.2011.613995
  » https://doi.org/10.1080/08870446.2011.613995
• ⁷
  Beck L, Ajzen I. Predicting dishonest actions using the Theory of Planned Behavior. J Res Pers. 1991;25(1):285-31. doi: 10.1016/0092-6566(91)90021-H
  » https://doi.org/10.1016/0092-6566(91)90021-H
• ⁸
  Yeung D, Zhou X. Planning for retirement: longitudinal effect on retirement resources and post-retirement well-being. Front Psychol. 2017;8(1):1-14. doi: 10.3389/fpsyg.2017.01300
  » https://doi.org/10.3389/fpsyg.2017.01300
• ⁹
  Miletto EM, Bertagnolli SC. Desenvolvimento de software II: introdução ao desenvolvimento web com HTML, CSS, Javascript e PHP. Porto Alegre: Bookman;2014.
• ¹⁰
  Gvozd R, Guirardello EB, Rossaneis MA, Pissinati PSC, Haddad MCFL. Translation a cultural adaptation of the Retirement Resources Inventory for the Brazilian culture. Rev Saúde Publ. 2019;53:60. doi: 10.11606/s1518-8787.2019053000863
  » https://doi.org/10.11606/s1518-8787.2019053000863
• ¹¹
  Pissinati PSC, Évora YDM, Rossaneis MA, Gvozd R, Santos MS, Haddad MCFL. Development of a web software prototype to support retirement planning. Rev Latino-Am Enfermagem. 2019;27:e3169. doi: 10.1590/1518-8345.3024.3169
  » https://doi.org/10.1590/1518-8345.3024.3169
• ¹²
  Avari M, Feizollahzadeh H, Valizadeh S, Viran F. Development and evaluation of multimedia software of vital drug prescription and calculation principles. Rev Dev Med Educ. 2018;7(1):32-7. doi: 10.15171/rdme.2018.007
  » https://doi.org/10.15171/rdme.2018.007
• ¹³
  Pasquali L. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed;2010.
• ¹⁴
  Sousa VEC. Developing and Testing of a Software Prototype to Support Diagnostic Reasoning of Nursing Students. Int J Nurs Knowl. 2018;29(2):124-32. doi: 10.1111/2047-3095.12145
  » https://doi.org/10.1111/2047-3095.12145
• ¹⁵
  Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciênc Saúde Coletiva. 2011;16(7):3061-8. doi: 10.1590/S1413-81232011000800006
  » https://doi.org/10.1590/S1413-81232011000800006
• ¹⁶
  Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67. doi: 10.1002/nur.20199
  » https://doi.org/10.1002/nur.20199
• ¹⁷
  Liu PC, Zhang HH, Ying J. Retirement planning and work-related variables in Chinese older nurses: A cross-sectional study. J Nurs Manag. 2017;26(2):1-12. doi: 10.1111/jonm.12532
  » https://doi.org/10.1111/jonm.12532
• ¹⁸
  Gadioli B, Fulquini FL, Kusumota L, Gimenes FRE, Carvalho EC. Construção e validação de um objeto virtual de aprendizagem para o ensino da semiologia vascular venosa periférica. Esc Anna Nery. 2018;22(4):e20180043. doi: 10.1590/2177-9465-ean-2018-0043
  » https://doi.org/10.1590/2177-9465-ean-2018-0043
• ¹⁹
  Carvalho LR, Évora YDM, Zem-Mascarenhas SH. Assessment of the usability of a digital learning technology prototype for monitoring intracranial pressure. Rev Latino-Am Enfermagem. 2016;24(1):e2777. doi: 10.1590/1518-8345.1054.2777
  » https://doi.org/10.1590/1518-8345.1054.2777
• ²⁰
  Olds T, Burton NW, Sprod J, Maher C, Ferrar K, Brown WJ, et al. One day you'll wake up and won't have to go to work: the impact of changes in time use on mental health following retirement. Plos One. 2018;13(6):e0199605. doi: 10.1371/journal.pone.0199605
  » https://doi.org/10.1371/journal.pone.0199605
• ²¹
  Cohen-mansfield J, Regev I. Retirement preparation programs na examination of retirement perceptions, self-mastery, and well-being. Res Soc Work Pract. 2016;28(4):428-37. doi: 10.1177/1049731516645194
  » https://doi.org/10.1177/1049731516645194
• ²²
  Lestantri ID, Rosini. Evaluation of software quality to improve application performance using Mc Call Model. J Inform Systems Eng Business Intell. 2018;4(1):18-24. doi: 10.20473/jisebi.4.1.18-24
  » https://doi.org/10.20473/jisebi.4.1.18-24

Edited by

EDITOR IN CHIEF: Antonio José de Almeida Filho

ASSOCIATE EDITOR: Andrea Bernardes

Publication Dates

• Publication in this collection
  24 Mar 2021
• Date of issue
  2021

History

• Received
  27 Apr 2020
• Accepted
  13 Sept 2020