EVALUATION INSTRUMENTS AND QUALITY PARAMETERS FOR EARLY CHILDHOOD EDUCATION

Guerres-Zucco, Dirce; Zanella, Andreia; Coutinho, Angela Scalabrin

Acessibilidade / Reportar erro

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

PUBLIC POLICIES, EVALUATION AND MANAGEMENT • Cad. Pesqui. 52 • 2022 • https://doi.org/10.1590/198053147958 copy

EVALUATION INSTRUMENTS AND QUALITY PARAMETERS FOR EARLY CHILDHOOD EDUCATION

Authorship SCIMAGO INSTITUTIONS RANKINGS

Abstract

The article, with a qualitative approach, deals with evaluation instruments and quality parameters for early childhood education. It is proposed to identify, describe and compare different models for assessing the quality of early childhood education, analyzing their suitability for application in the public education system of Florianópolis, Santa Catarina, Brazil. Among the models analyzed, two showed greater theoretical and methodological coherence to the context, because they are based on access, inputs and processes and provide for the participation of the educational community. The implementation and management of an evaluation system requires an inseparability between evaluation policy and evaluation instruments through intersectionality, fostered by public policies for children to meet the quality parameters of early childhood education.

EDUCATION ASSESSMENT; EDUCATION QUALITY; EARLY CHILDHOOD EDUCATION

ITERS-R and ECERS-R - United States
Type	External evaluation
Methodology	An external evaluator observes the spaces and environments, identifies, and assigns quality to indicators.
Indicators	ITERS-R Evaluates the space and furniture; personal care routines; speak and understand; activities; interaction; structure of the program, parents, and staff, with 39 items detailed in 455 indicators.	ECERS-R Evaluates the space and furniture; personal care, language, and reasoning routines; activities; interaction, program structure; parents and staff, with 43 items and 470 indicators.
Time	One day, 3.5-hour observation period.
Participation	Do not feature participatory characteristic, only queries parents and team when the external evaluator cannot identify the indicators in the environment.
Results	Data and information are collected and measured, and a value is attributed by the evaluator who does not discuss the results of the evaluation.
Post-results	The external evaluator provides indications on points for improvement and action strategies.
Viability	Viable for ease of access and little time in the preparation of external evaluators.
Validity and reliability	Valid and reliable for being comprehensive with common parameters of early childhood education, facilitating quality mapping in diverse social and educational realities.
Maturity	Degree of maturity established by having been applied in different historical times and in different contexts of the United States and later in countries of the five continents.
Source: Harms et al. (2006Harms, T., Cryer, D., & Clifford, R. M. (2006). Infant/toddler environment rating scale. Teachers College Press.), Harms (2013) and Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].).
ISQUEN and AVSI - Italy
Type	Meta-evaluation and self-assessment
Methodology	Comprising an external evaluator, internal evaluators (among them, an articulator) and other segments of the institution (families and professionals). Present pre-defined indicators and promote debate among participants in a procedural and formative manner.
Indicators	ISQUEN Subjects, contexts and practices, the knowledge of doing, guarantees.	AVSI Educational experience, professional activities, adults, and their relations, guarantees, structure.
Time	Varies between institutions. It does not prescribe the application time or frequency.
Participation	Of a negotiated and dialog-based nature, provide for the participation of all segments.
Results	Items in disagreement are socialized in order to reflect and find consensus from conception, values of that reality and the principles of early childhood education.
Post-results	Drafting a collective action plan, involving everyone in improvements.
Viability	Due to being procedural and given the interaction of the external evaluator, as articulator of the instrument, require higher qualification and more time.
Validity and reliability	Valid and reliable. Feature internal consistency, the evaluation was constructed from the adaptation of ITERS and ECERS, foresee adaptation to contextual reality.
Maturity	Degree of maturity established, also applied in other countries. However, due to evaluating the context, require adaptation to the reality observed.
Source: Becchi et al. (2014Becchi, E., Bondioli, A., & Ferrari, M. (2014). ISQUEN: Indicadores e escala de avaliação da qualidade educativa da creche. In L. Cipollone (Org.), Instrumentos e indicadores para avaliar a creche: Um percurso de análise da qualidade (pp. 149-197). UFPR.), Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].) and Souza et al. (2017Souza, G. de, Moro, C., França, F. F., & Rodrigues, A. J. L. (2017). A pesquisa em rede na educação infantil: Avaliação de contexto, modos de proceder e possibilidades de reflexão. RELAdEI: Revista Latinoamericana de Educación Infantil - Evaluación de Contextos en Educación Infantil, 6(1-2), 23-32.).
ECCP - Mexico
Type	External evaluation
Methodology	Evaluation of programs carried out by external evaluators. The instrument combines observation of spaces and interviews with principals and faculty with respect to the institution and classrooms.
Indicators	Establishment, resources, educational process, educational management, family and community relations, classroom, resources.
Time	External evaluators: visits at the beginning and end of the school year. Two days with interviews, families, and children. Three visits: one to discuss the results, verifying their validity and two to develop improvement actions with teachers and principals. And, finally, three visits to check the deployment of said actions.
Participation	The instrument combines observation and interviews, but the quality opinion is issued by external evaluators.
Results	Improvements were noted in the quality of infrastructure, materials, management, health, institutional identity, daycare-family relationship.
Post-results	Evaluation feedback is the basis for the continuity of curricula and reorienting teacher training and supervision.
Viability	Instrument constructed by the Mexican educational system between 2003 and 2007, sought to adapt to the guidelines (general principles of early childhood education), viable for that context.
Validity and Reliability	Not described. Instrument in test period for a standard evaluation for a diverse and multicultural context, which coincided with the expansion of the offer of openings.
Maturity	Maturity level not defined. Has been reformulated five times, including new dimensions: health, climate, children with disabilities.
Source: Myers (2011Myers, R. (2011). Em busca da qualidade educacional na pré-escola: Uma experiência mexicana. Cadernos de Pesquisa, 41(142), 100-115.) and Martínez Preciado (2010Martínez Preciado, J. F. (2010). La construcción de indicadores y evaluación de la calidad en centros educativos: Seis experiencias en México. REICE: Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 8(5), 133-153.).
NQS - Australia
Type	External evaluation
Methodology	External evaluator evaluates the quality of services. Visits of authorized agent to observe the spaces, question, analyze documents, check the application of the plan.
Indicators	Educational and practical program; children’s health and safety; physical environment; personnel arrangements; relationships with children; collaborative partnerships with families and communities; leadership and management of services.
Time	Application time in not described.
Participation	Participation limited to professionals and families. External evaluators have major influence on the decision on the outcome.
Results	Legislation provides for the dissemination of results (regulatory assessment). Quality levels must be achieved for accreditation of institutions.
Post-results	Penalties are provided by law for institutions not meeting quality levels established in the improvement plan. Subject to accountability of professionals, institutions, and suspensions of services.
Viability	Viable. Because it is established, articulated, and guaranteed by law, resources for execution are provided for.
Validity and reliability	Recognizes the existence of multicultural contexts, presents coherence with national guidelines, making the instrument reliable and valid, but sets precedents for ranking of institutions.
Maturity	It has consistent maturity level. It is systemic, comprehensive, nationally standardized.
Source: Tayler (2014Tayler, C. (2014). Avaliação da qualidade da educação infantil na Austrália. Estudos em Avaliação Educacional, 25(58), 126-151.) and Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].).
IDEA - Spain
Type	Self-assessment
Methodology	Voluntary participation by institution. Comprised of steps: a) information collection: interviews, review of documents, observation, application of questionnaires; b) valuing: drafting of reports and highlighting strengths and weaknesses; c) decision-making: proposals for improvement.
Indicators	Faculty qualification, number of children per classroom, physical structure in relation to health and safety (in buildings, facilities, indoor and outdoor equipment), the curriculum and learning and educational practice.
Time	Instrument deployment time is not described.
Participation	Participation conditional on the evaluation questionnaire applied to families and faculty. External evaluators define the level of quality.
Results	Highlights institution’s weaknesses and strengths.
Post-results	Improvements are proposed, but no description on how to do it.
Viability	Does not describe difficulty or success in the application.
Validity and reliability	Since it is a self-assessment by volunteer institution adoption, the information is insufficient regarding efficiency, validity and reliability.
Maturity	It does not describe the level of maturity.
Source: Motiejunaite et al. (2014Motiejunaite, A., Delhaxhe, A., Balcon, M.-P., & Borodankova, O. (2014). La educación y atención a la primera infancia. Agencia Ejecutiva en el Ámbito Educativo, Audiovisual y Cultural.) and Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].).
QUALITÀ LÚDICA - Italy
Type	Self-evaluation
Methodology	Systematic observation of the spaces and data collection by evaluators on the expansion of the repertoire of games and the playful behaviors and competencies of the children.
Indicators	Educational project; space (in the room, in the internal and external common areas); materials (in the room, in the internal and external common areas); time; group composition; adult and game.
Time	Application time is not described.
Participation	Promotes participation from the perspective of context assessment.
Results	The data collected are discussed with the external evaluator and the segments of the educational community to reach a consensus.
Post-results	Not described.
Viability	Not described, requires knowledge about child development and game-based activities, requiring greater qualification and mediation of the external evaluator.
Validity and reliability	Subjective evaluation depends on the interpretation of the evaluators. The instrument does not present sufficient information on the degree of reliability.
Maturity	Level of maturity is not described. Still a pilot/research project.
Source: Moro and Souza (2016Moro, C., & Souza, G. de. (2016). Para uma análise pedagógica dos contextos educativos. (Entrevista com Anna Bondioli, Monica Ferrari, Donatella Savio). Universidade de Pávia/Itália.).
ERVIS
Type	Self-evaluation - inclusive education
Methodology	Comprised of five areas evaluated, subdivided into 41 items, observation of spaces performed by professionals preferably trained in inclusive education, but improvements target all children.
Indicators	Structure; qualification; professional activities; relationship among adults; integration process.
Time	Application time is not described.
Participation	Participatory, dialog- based with the professionals of the institution.
Results	Data targets identifying the presence or absence of significant factors for the education of children with disabilities.
Post-results	With the results, a specific project is drafted, with participation by faculty in proposing, implementing, evaluating, and modifying personalized paths designed to teach children with “disabilities”.
Viability	Contextual, requires specific training in special education.
Validity and reliability	Insufficient information is provided on the degree of reliability, since it depends on the interpretation of the evaluators, as well as the mastery of inclusive processes.
Maturity	The level of maturity could not be identified.
Source: Bondioli (2009Bondioli, A. (2009). ERVIS: Elementi per Rilevare e Valutare l’integrazione Scolastica. Editora Junior.) and Pimenta (2017Pimenta, C. O. (2017). Avaliações municipais da educação infantil: Contribuições para a garantia do direito à educação das crianças brasileiras? [Tese de doutorado, Universidade de São Paulo - Faculdade de Educação].).
SPRING - Italy
Type	Self-evaluation and hetero-evaluation
Methodology	External evaluator makes the initial contact, meetings, interviews and observations of spaces and practices, preparation of the final report on the data collected, debated, and analyzed for the return to the institution. With dimensions that relate to each other, they serve both for self-assessment and hetero-assessment. Quantifiable data is not used, being totally descriptive, requires time for subjectivity, including to consult the results.
Indicators	Organization of the educational context; functioning of the working group; the institution’s relations with families and the community; evaluation processes.
Time	On average six months. Reapplication depends on the progress achieved in the improvement plan.
Participation	Provides for the participation of the different segments of the institution, formative, dialog-based, debate and confrontation, including pedagogical concepts for the quality negotiated.
Results	Drafting of improvement plan.
Post-results	To put improvement plan into practice with the changes and continuities indicated.
Viability	Viable, in a systemic perspective, with qualified human structure, of specific knowledge about the instrument and pedagogical conception to support the institution.
Validity and reliability	Offers internal consistency, by presenting analysis of the context from the standpoint of different segments, providing validity and reliability.
Maturity	Not described. Recently created instrument.
Source: Marcuccio and Zanelli (2013Marcuccio, M., & Zanelli, P. (2013). Sguardi sul nido... Strumento per lo Sviluppo di Processi Riflessivi e Indagini valutative nei Nidi da parte dei Gruppi di lavoro educativi (SPRING). Edizioni Junior-Spaggiari Edizioni.) and Moro (2018Moro, C. (2018). Diferentes olhares para a creche: A avaliação de contexto com o instrumento SPRING em um município da Emilia Romagna. Revista Linhas, 19(40), 138-160.).
EARLY CHILDHOOD EDUCATION QUALITY INDICATORS - Brazil
Type	Self-evaluation
Methodology	Participants are divided into seven teams, one for each dimension, with coordinator and reporter. The quality level is based on the colors of the semaphore. At the conclusion of this stage, the subgroups return to plenary and each dimension is summarized, in order to negotiate and reach a consensus on the outcome and prepare an improvement plan.
Indicators	Institutional planning; multiplicity of experiences and languages; interactions; health promotion; spaces, materials, and furniture; faculty and other professionals qualifications and working conditions; cooperation and exchange with families and participation in social protection networks.
Time	One day.
Participation	Participation and positioning of the different segments of the institution is essential.
Results	To identify the institution’s strengths and weaknesses for the purpose of drafting an action plan consisting of: dimension, indicator, problems, actions, responsible and deadlines.
Post-results	Monitoring of the action plan by all segments is not proposed, nor does it propose attributions to the Secretaria Municipal de Educação [Municipal Department of Education] and/or public policies.
Viability	Accessible and easy to apply.
Validity and reliability	Not being formative interferes with validity and reliability. Depends on the design and recognition of the educational community to identify the quality of services.
Maturity	Maturity level is not described but can be applied at different times and contexts.
Source: MEC (2009).

	ITERS-R/ ECERS-R - USA	ISQUEN/AVSI - Italy	ECCP - Mexico	NQS - Australia	IDEA - Spain	Qualità Lúdica - Italy	ERVIS - Italy	SPRING - Italy	Indicadores da qualidade - Brazil
Criteria	ITERS-R/ ECERS-R - USA	ISQUEN/AVSI - Italy	ECCP - Mexico	NQS - Australia	IDEA - Spain	Qualità Lúdica - Italy	ERVIS - Italy	SPRING - Italy	Indicadores da qualidade - Brazil
1. In line with the Brazilian curricular guidelines of early childhood education (ECE)	@	@	@	@	@	@	@	@	S
2. Indicates the quality standard, according to the principles and parameters of the Brazilian ECE	@	@	@	@	@	!	!	@	S
3. Considers values and purposes of the context as quality parameters	*	S	@	@	!	S	S	S	S
4. Promotes the participation of all segments of the educational community	*	S	*	N	*	S	S	S	S
5. The methodology provides for external evaluator and internal evaluator	N	S	N	N	N	S	S	S	N
6. Promotes evaluation as a formative process	N	S	N	N	N	S	S	S	*
7. Because of being dialog-based, there is consensus on the result for the validation of the construct	N	S	N	N	N	S	S	S	S
8. Provides for the collective drafting of the improvement plan and action management	N	S	N	N	N	S	S	S	S
9. Enables monitoring of the improvement plan by the educational community	N	S	@	@	@	!	!	S	S
10. Provides for inter-sectorality through an external support network for the institution	N	*	N	N	!	!	N	N	*
11. Presents validity, reliability, and consolidated maturity levels	S	S	*	S	!	!	!	!	*

Fundação Carlos Chagas Av. Prof. Francisco Morato, 1565, 05513-900 São Paulo SP Brasil, Tel.: +55 11 3723-3000 - São Paulo - SP - Brazil
E-mail: cadpesq@fcc.org.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] TRANSLATED BY

José Alvaro de Botelho Néia^IV