Quality perception in research laboratories from Fiocruz after QMS implementation

La percepción de la calidad en los laboratorios de investigación Fiocruz después de la implementación del SGC

A percepção da qualidade em laboratórios de pesquisa da Fiocruz após a implementação do SGQ

Quality perception in research laboratories from Fiocruz after QMS implementation

A percepção da qualidade em laboratórios de pesquisa da Fiocruz após a implementação do SGQ

La percepción de la calidad en los laboratorios de investigación Fiocruz después de la implementación del SGC

Ivanete Milagres Presot^I; Rodrigo Pedro Pinto Soares^I; Ana Paula Madureira^II; Kelly Alves Bicalho^I,III; Celina Maria Modena^I

^IOswaldo Cruz Foundation (Fiocruz), René Rachou Research Center

^IIFederal University of São João Del Rey

^IIIFederal University of Ouro Preto

ABSTRACT

Quality Management System has been implemented at the René Rachou Research Center since 2003. This study investigated its importance for collaborators (Cs) in laboratories. This was a quantitative and descriptive study performed in a group of 113 collaborators. It was based on the World Health Organization handbook: Quality Practices in Basic Biomedical Research. The questionnaires evaluated the parameters using the Likert scale. Biosafety, training and ethics were considered to be the most important parameters. Supervision and quality assurance, data recording, study plan, SOPs and file storage achieved intermediate evaluation. The lower frequency of responses was obtained for result report, result verification, personnel and publishing practices. Understanding the perception of the collaborators allows the development of improvement actions aiming the construction of a training program directing strategies for disseminating quality.

Keywords: Quality Management Systems; quality perception; biomedical research.

RESUMO

O Sistema de Gestão da Qualidade vem sendo implementado no Centro de Pesquisas René Rachou desde 2003. Este estudo investigou sua importância para os colaboradores dos laboratórios de pesquisa. Tratou-se de um estudo quantitativo, descritivo, realizado com um grupo de 113 colaboradores. Utilizou-se a diretriz Práticas de Qualidade na Pesquisa Biomédica Básica (QPBR), da Organização Mundial de Saúde. Os questionários aplicados avaliaram os parâmetros dessa diretriz utilizando a escala Likert. Biossegurança, treinamento e ética foram os fatores considerados mais importantes. Supervisão e garantia da Qualidade, registro de dados, plano de estudo, POP e arquivo obtiveram uma avaliação intermediária. A menor avaliação foi obtida para relatório, verificação de resultados, pessoal e práticas de publicação. Entender a percepção dos colaboradores permitirá o desenvolvimento de ações de melhoria, contribuindo para a construção de um programa de treinamento e estratégias de disseminação da qualidade.

Palavras-chave: Sistema de Gestão da Qualidade; percepção da qualidade; pesquisa biomédica.

RESUMEN

El Sistema de Gestión de la Calidad ha sido implementado en el Centro de Investigación René Rachou desde el año 2003. En este estudio se investigó la importancia de dicho sistema para los trabajadores de los laboratorios de investigación. Se realizó un estudio de tipo cuantitativo, descriptivo con un grupo de 113 trabajadores. Fueron utilizadas las guías sobre Prácticas de Calidad en Investigación Biomédica Básica (QPBR) de la Organización Mundial de la Salud. El cuestionario evalúa los parámetros de esta guía utilizando la Escala Likert. Bioseguridad, formación y ética fueron los factores considerados como los más importantes dentro del cuestionario. Supervisión y Control de calidad, registro de datos, plan de estudios, POP y el archivo obtuvieron una puntuación intermedia. La puntuación más baja fue obtenida por el informe, la verificación de resultados, el personal y las prácticas de publicación. Conocer las opiniones de los trabajadores permitirá el desarrollo de acciones para la mejora del Sistema de Gestión de la Calidad, contribuyendo al desarrollo de un programa de capacitación y de estrategias para la difusión de la calidad.

Palabras clave: Sistema de Gestión de la Calidad; percepción de calidad; investigación biomédica.

1. Introduction

Quality Management Systems (QMS) was originated in industry and have been expanded to other fields, being an important key of globalization. It is associated with a style of management in which a systemic view and continuing improvement combine with the aim of increasing performance (Vieira et al., 2011). Quality Management Systems worldwide share common requirements, but comprise unique characteristics depending on the context (Presot and Silva, 2006). Among its definitions, quality can be understood as an essential and differential attribute of a product or as a service satisfaction and commitment (Crosby, 1990). According to Juran's trilogy, quality consists of planning, control, and improvement (Juran, 1990), while Deming (1990), defined quality based on the PDCA cycle (Plan, Do, Check, Act).

Total Quality Management, optimized in Japan after World War II, was established in Brazil in the 1980s (Falconi, 1990), followed by the creation of many awards including the National Quality Award (NQA) and the Federal Government Quality Award (FGQA). In 1987, the International Organization for Standardization (ISO) published the 9000 series for QMS to standardize the exchange of products and services. The most recent review of ISO 9001 was in 2008 (ABNT, 2008). Although ISO 9000 is widely adopted in global business its implementation is voluntary.

The trend toward QMS inclusion in business was followed worldwide by academic centers concerned with the verification and reproducibility of scientific research (Silva, 2007). In the 1970s, irregularities in toxicity laboratories were detected by the United States Food and Drug Administration (FDA). This resulted in the development of the Good Laboratory Practices (GLP) manual for non-clinical safety studies. In 1981, the Organization for Economic Co-operation and Development (OECD) published GLP principles, adopted in 1986 by the European Union (OECD, 1998). Those practices are currently widespread on a global scale. Good Laboratory Practices was first imposed on industry by regulatory authorities, followed by Good Manufacturing Practices (GMP) and Good Clinical Practices (GCP) (WHO, 2006). In Brazil, GLP is mandatory in non-clinical laboratories handling pharmaceuticals, agrochemicals, cosmetics, veterinary products, food additives, food, and industrial chemicals (Inmetro, 2009). Other available guidelines include ISO 17025 for testing and calibration. The most recent review of this standard was in 2005 (ABNT, 2005).

Clinical diagnostic laboratories follow ISO 15189, published in 2003 and revised in 2008 (ABNT, 2008a). According to Plebani (2003), the pursuit of accreditation is crucial to the improvement of laboratory services and to move toward an international consensus. In this sense, laboratory medicine can be considered a pioneer in quality promotion (Vieira et al., 2011). Ovretveit (2013) also reinforced the discussion of quality in the public health and in health services pointing an increase in the use of indicators and the importance of quality in the research performed by nurses, physicians and project leaders. Knowledge of research methods, especially the collection and interpretation of data are essential tools for successful results.

Quality is essential in research centers and universities, and, since these institutions are not covered by GLP, it is necessary to implement QMS. Researchers and sponsors have questioned the use of traditional peer review system as the sole means for research evaluation, emphasizing the need for quality principles in these institutions (Robins et al., 2006). Since basic research is not covered by GLP, in 2006 the TDR/WHO published the Quality Practices in Basic Biomedical Research Handbook (QPBR) (WHO, 2006).

In Brazil, the publication of ABNT 16501 (Guidelines for research management systems, development and innovation) by the Brazilian Association of Technical Standards (ABNT) in December 2011 represented an important step for standardization of research, development, and innovation (RD & I). Although it was based on ISO 9001, its focus is not directly relevant to universities and research center laboratories (ABNT, 2011). Therefore, the São Paulo Research Foundation (Fapesp) has published the Guide of Good Scientific Practice, dealing with ethical guidelines in conducting scientific research (Fapesp, 2011).

2. The Oswaldo Cruz Foundation and the René Rachou Research Center

The Oswaldo Cruz Foundation (Fiocruz) is part of the Brazilian Health Ministry, and is one of the most important research institutions in Latin America. Its mission includes promoting health and social development, in addition to generating and disseminating scientific and technological knowledge. For more than a century, Fiocruz has dealt with the major public health problems of Brazil, becoming a center of excellence in biomedical research. Its mission expanded to outpatient hospital services, reference laboratories, and the production of vaccines, drugs, reagents, and diagnostic kits as well as education and training, information and communication, quality control of products and services, and implementation of social programs. Fiocruz was established in Rio de Janeiro in the beginning of the twentieth century and currently has several regional sites, including the René Rachou Research Center (CPqRR/Fiocruz) in Belo Horizonte, Brazil.

The CPqRR/Fiocruz (www.cpqrr.fiocruz.br) is a reference institution for public health research. Its mission is to provide a better life quality based on health care policies, research, technological development, innovation, teaching, and reference services. It consists of 14 laboratories with modern infrastructure allowing the study of cellular and molecular biology, epidemiology, immunology, and the search for new drugs and vaccines for tropical diseases, including Chagas disease, leishmaniases, malaria, schistosomiasis, and intestinal parasites. Besides infectious disease, research is devoted to epidemiological aspects on aging, risk behavior, and occupational diseases. In 2003, a graduate course on health sciences was established at the institution to focus on training students (Master and Ph.D.) for scientific research, teaching, and professional activities in the health field. Additionally, CPqRR provides scientific and technical support for the Brazilian Public Health System (SUS).

In 2003, Fiocruz issued Internal Regulation 430/2002 requiring the implementation of QMS in all reference laboratories. The CPqRR required the implementation of NBR ISO 17025 in its reference laboratories. Some projects of technological development adopted GLP. This was followed by the research laboratories, which lacked specific QMS regulation. In 2006, WHO published the QPBR handbook (WHO, 2006). This guide was developed by a group of specialists, including researchers, and focused on relevant and practical aspects of laboratory routine. Fiocruz has been investing in QMS implementation as exemplified by use of the GLP in technological platforms of the Aggeu Magalhães Research Center (CPqAM/Fiocruz), a regional site of Fiocruz in Recife, Brazil. This reflects a recent interest in the implementation of QMS, not only in Fiocruz, but also for research and development (R & D) institutions. This aims to comply with national and international regulations, an important requirement for certification/accreditation and sometimes necessary for sponsoring agencies (Souza et al., 2012). Another example is a survey that evaluated GLP implementation in a basic research laboratory in the Gonçalo Muniz Institute, demonstrating the viability and positive impacts in the laboratory routine (Fraga et al., 2012).

In the past ten years, QMS has been implemented in the reference and research laboratories of the CPqRR/Fiocruz. During its implementation a group of Quality Coordinators was trained to disseminate QMS culture into laboratories.

After this period, the question of how QMS affected the collaborators (Cs) perception arose as a major concern of the institution's Office of Quality and Biosafety (OQB).

3. Quality perception studies

Perception studies are relevant to understanding unknown aspects of the organization and development of improvement measures (Vaitsman et al., 2003). Perception analyses are widely used in health services. Castellanos (2002) conducted a descriptive study using survey methods and a structured questionnaire in order to assess the customer's and the staff's perceptions in the Hospital of the University of São Paulo. The results demonstrate that the quality perception was different among the two groups. The theoretical framework used by this researcher had a strong contribution of the marketing field, including quality attributes, field methodology and quantitative analysis. More recently, Fadel and Filho (2009) compared clients' and dentists' perceptions of a clinic in the city of Florianopolis, Brazil. This study showed that technical quality was important to the professionals, while interpersonal qualities were more relevant for the patients. The perception evaluation of internal clients offered a reliable tool enabling the detection of improvement opportunities in internal processes. Santos and partners (2008) reported the effectiveness of social responsibility programs from a banking institution. In this survey, the perception of public employees resulted in changes in the current programs indicating new areas of action. There are currently no available reports of perception evaluations about QMS in research laboratories.

4. Objective

This study aimed to analyze the professionals' perception of quality in research laboratories of the René Rachou Research Center (CPqRR/Fiocruz) after ten years of QMS implementation.

5. Methodology

The survey was performed in 12 research laboratories of the CPqRR/Fiocruz regional center of Fiocruz located in Belo Horizonte, Minas Gerais, Brazil. All laboratories involved participated in QMS implementation. The project was approved by the Ethics Committee of CPqRR/Fiocruz (protocol #22/2010). This was an applied, quantitative and descriptive field research. Data collection investigated the quality perception of the laboratory collaborators. The questionnaires were applied to the participants together with an informed consent form. The structured questionnaire was used in order to expose all participants to the same stimuli, open and closed questions, and the same procedures for data collection. The way to collect the data was self-administered while preserving the anonymity of participants. The questionnaires were developed based on the quality parameters defined by the QPBR handbook (figure 1).

The concepts and interpretation of quality parameters presented in figure 1 can be also found in the QPBR handbook, officially translated by CPqRR (WHO, 2010).

The total population of these laboratories is ~350 people. A group of 113 collaborators was interviewed (n=113) including researchers, technicians, lab managers, project managers and students, representing 30% of laboratory collaborators. Names of individual interviewees were not recorded to ensure information confidentiality. The first part of the questionnaire required the evaluation of the quality system importance in a research laboratory with respect to several parameters (figure 1), using the Likert scale with five possible responses: 1) strongly disagree, 2) disagree, 3) neither agree nor disagree, 4) agree, and 5) strongly agree.

The questionnaire also addressed areas that could be improved in the institution. In this question, the collaborators had the opportunity of choosing up to three options among 13 parameters assessed in the first part of the questionnaire. Finally, in the last part the collaborators had the opportunity to include comments about QMS.

In the next step, perception analyses according to professional category were performed. These comprised undergraduate students (15), master's degree students (9), Ph.D. students (17), technicians (34), laboratory/project managers (6), and researchers (32). Data were recorded on electronic spreadsheets (Microsoft Excel®).

6. Results and discussion

Most of the participants were young adults (31 + 8years), with 75% being below 36 years old; female (78.9%); and with a degree in biology (60.5%). The interviewed subjects were students (undergraduate and graduate) (36.0%), researchers (28.1%), technicians (29.8%), and others (6.1%).

The majority of answers were in the "agree" and "strongly agree" columns for all parameters (graphic 1). The higher frequency (88-99%) of responses in the "strongly agree" column was obtained for three parameters: biosafety, training and ethics. The frequency of responses in the "strongly agree" column was between 60 and 70% for data recording, study plan, SOPs, file storage and supervision and quality assurance. The lower frequency (50-56%) of responses in the "strongly agree" column was obtained for result report, result verification, personnel (Job descriptions) and publishing practices. The parameters that needed most investment were: training, biosafety and quality policy/staff responsibility.

The sentences below provide comments made freely by the surveyed collaborators:

The system is extremely important to increase the quality of the research developed in research laboratories. However, it should not interfere with the research, an extremely dynamic and fast process.

An Ethics discussion should be included in the Institution's meetings. Ethics in public service ... how to behave in a public institution.

The awareness of employees about the importance of equipment handling and the workplace.

It is important to encourage the staff to engage in the quality strategies, the quality week is a big event, but it only happens once a year.

Each laboratory should have at least one permanent employee responsible for quality practices.

The current guidelines do not specify the storage period of research records, making difficult to establish the time for file storage.

The next step was to analyze the data according to professional category in the Cs group. The parameters biosafety and training showed the highest agreement among professional categories at 93.3 and 88.2%, respectively. A high level of agreement was also observed for the parameter ethics, except by undergraduate students (66.7%) (figure 2).

Personnel (job description) and recording data exhibited a similar profile, with managers having the highest percent agreement with the statements (figure 4). Finally, SOP showed similar levels of agreement among the professional categories except for master's students with a 100% of agreement (figure 4).

Competition for funding is increasing, and it is essential that scientific research, especially in the health field, be conducted in order to reduce waste of resources and avoid unnecessary duplication of effort. Quality practices are crucial to define, plan, perform, record, monitor, and report research (Robins, 2006; WHO, 2006). According to Grouchau and partners (2010) factors for the successful implementation of QMS in research and development environments may include the acceptance and commitment of all professionals involved (the so-called "quality culture"), the development of a flexible documentation system, and, more importantly, the QMS must add value to the institution. The benefits of QMS implementation have been reported in the research field, especially in institutions with accreditation requirements. It is expected that accreditation may become a future requirement for publication in major scientific journals (Abad, 2005).

Frequency of "strongly agree" responses to the remaining parameters varied among the professional categories. In general, master's students and managers exhibited the highest agreement with the statements, followed by Ph.D. students/technicians and undergraduate students/researchers. This pattern was observed for quality policy, supervision and quality assurance, results verification, result reporting, publishing practices, study plans, and file storage (figures 3 and 4).

In this study, it was evaluated the QMS collaborators perception in laboratories of a public research institution. High agreement was observed in the responses "agree" and "strongly agree", which may reflect the dissemination of quality culture and evidence that the investment in periodic QMS training was effective in the institution. The parameters training, ethics, and biosafety had high agreement scores in both groups (>88%) (graphic 1). Those factors are closely related to professional development and improvement of working conditions, probably understood as benefits for workers. According to Stashevsky and Elizur (2000), the perception by a given collaborator of his/her role in the decision-making process is important for better individual performance. Employees' perception that their work is meaningful has a crucial impact on their performance.

The parameters data recording, study planning, file storage, supervision and quality assurance, SOPs and quality policy/staff responsibility exhibited frequencies below 70% in the "strongly agree" column (graphic 1). Among the parameters with intermediate level (60-70%), the recording data was considered the most important. Although the recording books are well established in the Institution, they still need more investment during the QMS training. This was also reinforced by the Fapesp Scientific Practices, which states data research should be recorded in a concise and complete way. Additionally, the QPBR handbook emphasizes that study plans should be easily distinguished from the data in the register books (WHO, 2006). Concerning file storage, one collaborator identified the need for specifying how long those documents should be kept. Besides the discussion about storage time, it is important to reinforce that, according to QMS regulations, those documents should be located in a specific place, easily identifiable where the documents could be promptly retrieved.

For SOPs, the major difficulty in implementing QMS in scientific environments is related to the standardization and routine control. Standardization may hinder achievement of research objectives. Standardization, SOP and other components are perceived as limiting creativity, but it is important to remember that quality has two sides, technical and humanistic. The implementation of a system depends on the involvement of employees and represents more than just establishing procedures. There are no guidelines about how the QMS should be done, so its strategic implementation should address how to achieve satisfactory results (de Souza, 2012). Supervision and quality assurance besides results verification that is important to ensure reproducibility and publication is also related to the study inspection. In the case of research laboratories, those inspections are not simple tasks, due to the variety of projects and procedures carried out by researchers, undergraduate, and graduate students. According to WHO (2006), to avoid conflict of interest, audits should be conducted by outside professionals, or by someone in the institution who is not directly involved in the study.

QMS implementation and maintenance requires a continuous effort from the Institute Direction providing the infrastructure and human resources necessary for it (Juran, 1990). More importantly, leadership comes from top management and includes individuals and teams aiming commitment, integrity, honesty and truth. This need was clearly demonstrated during the interviews, in which the collaborators identified the need of specific permanent employees for QMS activities.

The QMS parameters rated with the lower scores by the collaborators were: result reporting, result verification, personnel (job description) and publishing practices (values below 56.0%) (graphic 1). Those data may indicate the need for specific training by the office of quality and biosafety in those areas. This is of importance, since result verification, result reporting, and publishing practices are strategic for the research project framework. In addition, the validity of scientific research depends on well-planned schedule, execution, and documentation (WHO, 2006).

The second area of the questionnaire aimed to identify the parameters perceived by Cs that need further investment. Training, biosafety, and quality policy/staff responsibility were the most important parameters. Training and biosafety responses overlapped with those with highest agreement scores in the "strongly agree" column (graphic 1). This emphasizes the need for specific training for personal and professional development to avoid accidents and hazards in research laboratories. In CPqRR/Fiocruz the collaborators are exposed to microbial pathogens, laboratory animals, and chemical substances that are potentially hazardous. With regard to the Quality Policy, the concerns of employees regarding the motivation are implied in the following statements: "... Each laboratory should have at least one permanent employee responsible for quality practices". "... It is important to encourage staff in the quality strategies engagement. The quality week is a big event, but it only happens once a year."

When analyzing the quality parameters per collaborator professional category, again biosafety and training did not exhibit differences among professions (figure 2). However, substantial differences were observed for the remaining parameters (figures 3 and 4). The highest agreement rates were observed for master students and managers for quality policy, supervision and quality assurance, result verification, result report, publishing practices, study planning, and file storage. With the exception of training and biosafety, the lowest levels of agreement were observed for undergraduate students and researchers. These students are part time and less involved with the institution quality policies, thus, specific QMS training in this group should be reinforced, as the students usually enter graduate school and remain at the institution for the subsequent years. The lower ratings observed for researchers may reflect their motivation towards QMS practices. While they recognize the importance of its implementation in the laboratory, they usually elect a QC to be in charge. This may reflect the motivation of researchers, being more involved in scientific matters in the institution. Ortner (2000) reported that leadership behavior plays a decisive role in QMS implementation. Consistent with our results, initiatives to implement QMS in university laboratories and research centers face difficulties depending on the category (Abad, 2005; Grouchau et al., 2010).

Furthermore, the category of master's students showed higher levels of "agree" responses than Ph.D. students, who in turn presented a higher rate than the researchers. As reported by Souza and partners (2012), Fraga and partners (2012) and Camman and Kleibohmer (1998), scientists in some research centers, specifically those with an academic profile, show concern for and resistance to, not only the activities related to the implementation of the system, but also the consequences generated by the modifications and proposals for adjustments. Some argue that the rigidity of a formal QMS and excessive rules can stifle the scientific progress, reduce creativity, and increase bureaucracy. This was clearly demonstrated in the following sentence from a collaborator: "... The system is extremely important to increase the quality of the research developed in research laboratories. However, it should not interfere with the research, an extremely dynamic and fast process."

However, in CPqRR there was a significant improvement in the quality culture. This result may be related to attributes of Ph.D. students and researchers who are overwhelmed with experiments, publishing, mentoring of students, preparation, and submission of grant projects, participation in conferences, and management activities. The excess of activity hinders motivation and participation in training activities of QMS. Therefore, alternative ways of attracting those professional categories should be implemented by OQB. According to Fraga and partners (2012), an important challenge is to convince the researchers of the benefits following the implementation of QMS and suggest training activities as a tool to achieve this objective.

7. Conclusions

This study demonstrated that QMS was successfully implemented in the laboratories of the CPqRR/Fiocruz indicating that a QMS culture was established. Perception of a quality parameter was shown to vary according to professional category. These data reinforce the need for a continuing QMS maintenance program to provide specific training to the groups in which differences were detected. Understanding the perception of collaborators allows the development of improvement actions, directing strategies for educational guidelines and for disseminating quality.

Paper submitted on December 20th, 2012 and accepted for publication on November 12th, 2013.

Ivanete Milagres Presot is an analyst of health management in the René Rachou Research Center/Fundação Oswaldo Cruz (Fiocruz). She has a Ph.D. in Health Sciences. E-mail: [e-mail].

Rodrigo Pedro Pinto Soares is an Associate Researcher II of the laboratory of Biomarkers of the René Rachou Research Center/Fundação Oswaldo Cruz (Fiocruz). He has a post-Ph.D. in Parasitology. E-mail: [e-mail].

Celina Maria Modena is a professor of the post-graduation course of Collective Health of the René Rachou Research Center/Fundação Oswaldo Cruz (Fiocruz). She has a post-Ph.D. in Collective Health. E-mail: [e-mail].

Ana Paula Madureira is an adjunct professor of the Federal University of São João del Rey, in the Biosystems Engineering Department. She has a Ph.D. in Zootechny. E-mail: [e-mail].

Kelly Alves Bicalho is post-Ph.D. student of the Immunology laboratory of the René Rachou Research Center/Fundação Oswaldo Cruz (Fiocruz) and of the Center of Animal Science of the Federal University of Ouro Preto. She has a Ph.D. in Health Sciences. E-mail: [e-mail].

• ABAD, Xavier; BOSCH, Albert; NAVARRO, Carme. Implementation of Good Laboratory Practice in a University Research Unit. Quality Assurance Journal, v. 9, n. 4, p. 304-311, 2005.
• ABNT. Associação Brasileira de Normas Técnicas. NBR ISO 9001: Sistema de gestão da qualidade: requisitos. Rio de Janeiro: ABNT, 2008.
• ABNT. Associação Brasileira de Normas Técnicas. NBR ISO 16501: Diretrizes para sistemas de gestão da pesquisa, do desenvolvimento e da inovação (PD & I). Rio de Janeiro: ABNT, 2011.
• ABNT. Associação Brasileira de Normas Técnicas. NBR ISO/IEC 17025: Requisitos gerais para a competência de laboratórios de ensaios e calibração. Rio de Janeiro: ABNT, 2005.
• ABNT. Associação Brasileira de Normas Técnicas. NBR/NM-ISO 15189: Laboratórios de Análises Clínicas -Requisitos especiais de qualidade e competência. Rio de Janeiro: ABNT, 2008a.
• CAMMANN, Karl; KLEIBOHMER Wolfgang. Need for quality management in research and development. Accreditation and Quality Assurance, v. 3, n. 10, p. 403-405, 1998.
• CASTELLANOS, Pubenza L. Comparação entre a satisfação do usuário com os serviços oferecidos num hospital geral e a percepção gerencial dessa satisfação. Dissertação (Mestrado) Escola de Administração de Empresas de São Paulo, Fundação Getulio Vargas, São Paulo, 2002.
• CROSBY, Philip B. Qualidade, falando sério São Paulo: McGraw-Hill, 1990.
• DEMING, Willian E. Qualidade: a revolução da administração. Rio de Janeiro: Marques Saraiva, 1990.
• FADEL, Marianella A.; REGIS FILHO, Gilsée I. Percepção da qualidade em serviços públicos de saúde: um estudo de caso. Rev. Adm. Pública, v. 43, n. 1, p. 7-22, jan./fev. 2009.
• FALCONI, Vicente C. Gerência da qualidade total, estratégia para aumentar a competitividade da empresa brasileira Belo Horizonte: UFMG, 1990.
• FAPESP. Fundação de Amparo à Pesquisa do Estado de São Paulo. Código de boas práticas científicas São Paulo: Fapesp, 2011.
• FRAGA, Hilda C. et al. Avaliação da implementação de um sistema de qualidade em um laboratório de pesquisa básica: viabilidade e impactos. Einstein, São Paulo, v. 10, n. 4, p. 491-497, out./dez. 2012. Disponível em: <www.scielo.br/scielo.php?pid=S1679-45082012000400016&script=sci_arttext&tlng=pt>. Acesso em: 28 out. 2013.
• GROUCHAU, Inês H. et al. Implementation of a quality management system in university test laboratories: a brief review and new proposals. Accreditation and Quality Assurance, v. 15, n. 12, p. 681-689, 2010.
• INMETRO. Instituto Nacional de Metrologia, Qualidade e Tecnologia Industrial. NIT-DICLA-035: Princípios das Boas Práticas de Laboratório — BPL. Rio de Janeiro: Inmetro; 2009.
• JURAN, Joseph M. Liderança pela qualidade:um guia para executivos. São Paulo: Pioneira, 1990.
• OECD. Organization for Economic Co-operation and Development. Principles on Good Laboratory Practice. Paris: OECD; 1998.(OECD series on principles of Good Laboratory Practice and Compliance Monitoring, n. 1)
• ORTNER, Hugo M. The human factor in quality management. Accreditation and Quality Assurance, v. 5, n. 4, p. 130-141, 2000.
• OVRETVEIT, John. Contemporany quality improvement. Cadernos de Saúde Pública, v. 29, n. 3, p. 424-426, 2013. Disponível em: <www.scielo.br/scielo.php?pid=S0102-311X2013000300002&script=sci_arttext>. Acesso em: 28 out. 2013.
• PLEBANI, Mario. Appropriateness in programs for continuous quality improvement in clinical laboratories. Clinical Chimica Acta, v. 333, n. 2, p. 131-139, 2003.
• PRESOT, Ivanete M.; SILVA, Ana Beatriz M. da. Sistema de Gestão da Qualidade Laboratorial e Biossegurança. In: Biossegurança: informação e conceitos. Textos básicos. Rio de Janeiro: Fiocruz, 2006. p. 227-239.
• ROBINS Margaret M.; SCARLL, Jane S.; KEY, Pauline E. Quality assurance in research laboratories. Accreditation and Quality Assurance, v. 11, n. 5, p. 214-223, 2006.
• SILVA, Ana Beatriz M. da. Proposta para a implantação, implementação e avaliação de um programa de gestão da qualidade nos laboratórios de referência para a vigilância epidemiológica da Fiocruz. Dissertação (mestrado) Escola Nacional de Saúde Pública, Fiocruz, Rio de Janeiro, 2007.
• SOUZA, Renata de et al. Implementation of good laboratory practices (NIT-DICLA-035, Inmetro) in a technological platforms network: the Fiocruz experience. Accreditation and Quality Assurance, v. 17, n. 3, p. 331-339, 2012.
• STASHEVSKY, Shmuel; ELIZUR, Dov. The effect of quality management and participation in decision-making on individual performance. Journal of Quality Management, v. 5, n. 1, p. 53-65, 2000.
• VAITSMAN, Jeni et al. Metodologia de elaboração do Índice de Percepções Organizacionais. Cadernos de Saúde Pública, v. 19, n. 6, p. 1631-1643, 2003.
• VIEIRA, Keila F. et al. A utilidade dos indicadores da qualidade no gerenciamento de laboratórios clínicos. Jornal Brasileiro de Patologia e Medicina Laboratorial, v. 47, n. 3, p. 201-210, 2011.
• WHO. World Health Organization. Handbook: quality practices in basic biomedical research. Geneva: WHO, 2006.
• WHO. World Health Organization. Manual: práticas de qualidade na pesquisa biomédica básica Tradução de Editione Editoração e Consultoria Científica S/C Ltda. Belo Horizonte: Centro de Pesquisas René Rachou, 2010.

Publication Dates

• Publication in this collection
  28 Feb 2014
• Date of issue
  Feb 2014

History

• Accepted
  12 Nov 2013
• Received
  20 Dec 2012