Reading comprehension skills: your teaching and your assessment through Provinha Brasil

ABSTRACT

Through document analysis, classroom observation and analysis of children's performance in reading, we investigated tests from Provinha Brasil, as well as the relationships between children's outcomes and practices of their teachers. It was shown that there was a relationship between the results of Provinha Brasil and teaching practices, but the teachers didn't dedicate enough time to teaching reading texts. It was concluded that the impacts upon Provinha Brasil stemmed from teaching the alphabet basis and that the reading level required by Provinha's questions did not demand more elaborated text comprehension skills.

KEYWORDS:
reading; evaluation; teaching; Provinha Brasil

RESUMO

Por meio de análise documental, observação de aulas e análise do desempenho de crianças em leitura, investigamos testes da Provinha Brasil, assim como as relações entre os resultados das crianças e as práticas de suas professoras. Foi evidenciado que houve relação entre os resultados da Provinha Brasil e as práticas docentes, porém as professoras dedicavam-se pouco ao ensino de leitura de textos. Concluiu-se que os impactos nos resultados da Provinha Brasil decorreram do ensino da base alfabética e que o nível de leitura exigido nas questões da prova não demandava habilidades de compreensão de textos mais elaboradas.

PALAVRAS-CHAVE:
leitura; avaliação; ensino; provinha brasil

RESUMEN

A través de análisis de documentos, observación de clases y análisis del rendimiento de niños en lectura, investigamos exámenes de Provinha Brasil, bien como las relaciones entre los resultados de los niños y las prácticas de sus maestras. Se demostró que hubo relación entre los resultados en Provinha Brasil y las prácticas docentes, pero los maestros poco se dedicaban a la enseñanza de la lectura de textos. Se concluye que los impactos en los resultados de Provinha Brasil han resultado de la enseñanza de la base alfabética y que el nivel de lectura solicitado en las cuestiones de Provinha no requería habilidades de comprensión de texto más elaboradas.

PALABRAS CLAVE:
lectura; evaluación; enseñanza; provinha brasil

INTRODUCTION

Since its creation, the "Provinha Brasil" [The Brazil Exam] has been the object of considerable debate on the Brazilian educational scene. Among other controversies, questions about the need to evaluate children at the beginning of elementary school has been a recurrent theme. At the heart of this issue we can identify tensions concerning the role of school in the first few years of elementary education. According to Saviani, "the school is an institution whose role consists in the socialization of systematized knowledge" (2008, p. 14). Therefore, "the school exists to provide the acquisition of the instruments that enable access to complex knowledge (science), as well as the access to the rudiments of that knowledge" (idem, p, 15). Saviani affirms:

systematized knowledge, erudite culture, is a literate culture. Therefore, the first requirement to have access to this type of knowledge is to learn to read and write. Moreover, it is also necessary to know the language of numbers, the language of nature and the language of society. (idem, ibidem)

Therefore, according to Saviani (2008), it is a duty of the school to ensure access to reading and writing. This discussion is also closely related to the discussion of the role of early childhood education. Brandão and Rosa (2010), among other authors indicate that it is possible and desirable to promote exposure and reflection on writing with very young children, as long as these situations are ludic and suitable to children's aspirations.

From this perspective, it is important to question the knowledge and skills that children are expected to have by the end of early childhood education and at the end of the first grade of basic education, and the need to begin work focused on learning the alphabetic system of writing at the beginning of the schooling process.

In connection with this initial question, we can also inquire about what is the role of the school and what are the expectations for the acquisition of basic reading and writing skills in the subsequent schools years. Franco, Brooke and Alves (2008), in presenting some of the data collected in the GERES Project [The Longitudinal Study of School Generation], conducted from 2005 to 2008, which evaluated 20,000 students in Belo Horizonte, Rio de Janeiro, Campinas, Campo Grande and Salvador, revealed that, in the first grade of elementary school, children in the public and private school systems showed progress in reading abilities; however, in subsequent grades (the second and third grades of basic education), the pace of learning decreased. Brooke's study provokes reflections on the possible difficulties teachers may have to really know what is the focus of their work after students' acquire basic reading and writing skills.

Other authors, such as Oliveira (2010) and Cruz (2012), also reported low rates of learning progress in the classes they accompanied during the school year. Oliveira (2010) investigated nine classes, and found poor progress in reading, text production and activities related to the alphabetic system among the different grades in the schools studied.

Cruz (2012, p. 304), however, noted that "with regard to the axis of textual production and reading", the data indicated that, although the majority of the children in the second and third grades had advanced over the year, the consolidation of these core areas presented a deficit concerning the genres studied".

Considering this situation, there is a clear need to deepen the debate about children's school progress and expectations about what they should learn in the first grades of basic education.

By adopting an assessment in the second grade of basic education, the Brazil Exam assumes the position that the learning of certain reading and writing skills should be introduced by the end of the first grade and that, by the end of second grade, the learning of certain skills should be consolidated. But we might ask: what are the conditions for children to consolidate the learning of these skills? What needs to be ensured in teaching practices so these appropriations can take place?

In parallel to these discussions, we can also mention the debates about the evaluation process, that is, considering the complexity of the reading assessment process, we can question if the Brazil Exam can be considered a suitable instrument for judging children's command of reading, considering that reading is a process of interaction between the reader and the text that aims to achieve a certain goal. With the purpose of evaluating reading, the Brazil Exam proposes activities that range from the recognition of letters and the reading of words to the evaluation of reading comprehension. According to Kleiman (2014, p. 61), reading comprehension is "the process by which one puts into operation the skills and cognitive strategies needed for comprehension, which allow the reader to simultaneously extract and construct meanings from the text to make sense of written language."

Teachers should help their students to become self-regulated readers (who have clear reading objectives), who are active and possess several comprehension strategies (idem).

Reading strategies are procedures that must be taught. These strategies can help students to conduct autonomous reading (Solé, 1998). Therefore, the Brazil Exam proposes descriptors that seek to identify children's autonomy concerning reading in order to understand what reading skills they master, such as making inferences, locating information, identifying the theme, the purpose of the text and others.

In this article, we focus on an analysis of the Brazil Exam and its potential as a reading evaluation tool, and on the relations between the appropriations conducted by children in the second grade of basic education and the practices of their teachers concerning the teaching of reading. We start, therefore, from the point of view that it is possible and desirable for children to finish the second grade of basic education being able to read texts. Our reflections will be conducted in two directions: (1) What reading skills does the Brazil Exam assess? Is the Brazil Exam a good instrument for evaluating those skills?; (2) What are some of the possible relationships between results on the Brazil Exam and teachers' practices?

METHODOLOGICAL PROCEDURES

The educational systems of three municipalities in the metropolitan area of Recife participated in the study. For each system, we had the participation of four second grade teachers, for a total of 12 teachers. In this study, however, we only analyze the results of 11 classes - the results of the Brazil Exam for one of the classes were disregarded because in that case the test was based on the students' collective answers.

To meet our goals, we observed the administration of the Brazil Exam to selected second-grade classes and, later, nine classes given by each teacher before the administration of the test in the second semester; we conducted semi-structured interviews with the teachers, school administrators and the people who issued the test; and an analysis of both assessment instruments administered to the classes (i.e., at the beginning and the end of 2010) to understand the instrument's complexity and the possibility to evaluate reading skills.

The interviews and observations were recorded in audio and then transcribed, and data were categorized through content analysis.

To begin our discussion, in the next section we will present an analysis of the test used in the Brazil Exam in the first and second semesters of 2010 to investigate how many questions required children to have an autonomous reading capacity and the suitability of this tool to judge the children's reading capacities. We will then discuss the results achieved by 214 children in eleven second grade elementary school classes, and conclude with reflections about the teaching practices of the teachers of four classes, relating those practices to the results achieved.

THE BRAZIL EXAM IN 2010: AN ANALYSIS OF THE INSTRUMENTS

In 2010, the test used in the Brazil Exam comprised 24 questions. In the initial assessment in 2010, four questions (1, 5, 6 and 13) evaluated knowledge related to letters of the alphabet; eleven questions (2, 3, 7, 8, 9, 10, 11, 12, 14, 16, 17) assessed phonological awareness with and without graphophonic correspondence and skills related to the reading of syllables, words and sentences; and one question (4) assessed the ability to identify the initial letter of words. Of the 24 questions, eight sought to evaluate skills related to the reading of texts.

An analysis of the test's general features shows that the assessment focuses on the mastery of the Alphabetic Writing System. This approach is therefore based on the acquisition of reading and writing skills, which recognizes this specificity in the curriculum of the initial grades of basic education. Therefore, we can see an approximation to presuppositions maintained by Soares (2004, p. 15-16) according to which, in pedagogical practices for the initial grades of basic education, it is necessary to consider the multiple facets of reading and writing skills acquisition: "phonological and phonemic awareness, identification of phoneme-grapheme relationships, written language coding and decoding skills, knowledge and recognition of the processes of translation of the voiced form of speech into the graphic form of writing".

Although we recognize that, indeed, this dimension of the pedagogical process needs to be a priority at the beginning of schooling, we emphasize that other dimensions are also indispensable, such as those related to the development of reading and text production. Authors who advocate an approach to reading and writing skills acquisition in the perspective of literacy,¹ among which we include ourselves, share this view. These authors include: Albuquerque, Morais and Ferreira (2008); Brandão and Rosa (2005, 2010); Cruz (2008); Frade (2005); Leal and Albuquerque (2004); Leal, Albuquerque and Morais (2010); Maciel, Baptista and Monteiro (2009); Maciel and Lúcio (2008); Morais (2012); Picolli and Camini (2012); Santos and Albuquerque (2005); Silva (2007, 2008); Soares (2003, 2004); Souza and Cardoso (2012).

In the exam administered at the end of the school year, which was also carried out by means of a 24-item test, five questions (1, 2, 3, 6 and 11) assessed knowledge related to letters of the alphabet; eleven questions (4, 5, 8, 9, 10, 12, 13, 14, 15, 16, 18) assessed phonologic awareness with and without graphophonic correspondence and skills related to the reading of syllables, words and sentences; and one question (7) assessed the capacity to identify the initial letter of words. Of the 24 questions, seven sought to assess skills related to the reading of texts.

In both tests, four skills related to text reading were included in items focusing on: recognizing the goal of the text; finding information; identifying the theme; and making inferences. The decision to choose these reading activities in the assessment process is representative of what current curricular documents propose for the teaching of Portuguese in the first grades of basic education.

Leal and Brandão (2012), in their investigation of 26 curricular documents from Brazilian state capital cities and state education departments, showed that 73.07% indicate strategies that should be developed in reading activities. With regard to this aspect, Solé (1998, p.70) says that "it is necessary to teach strategies for understanding texts. These strategies do not mature, or develop, emerge, or appear. They are taught - or not - and learned - or not". Serra and Oller (2003, p. 35), also in relation to the core area of teaching reading, affirm that

The strategies that are activated in the course of reading can work as procedures that regulate reading itself. As such, these requisites require the presence of objectives to be achieved, an assessment of results, and the subsequent change, if necessary, of the learning activity. The integration of reading strategies in the student's field of competence will allow him to advance towards the self-regulation of his own reading activity.

With regard to the ability to recognize the purpose of a text, two questions (15 and 23) were formulated in the initial evaluation. In question 15, we can see that the child has support from an image of an educational campaign poster to answer the question. In addition, the test administrator reads the optional responses.

Figure 1:
Brazil Exam 2010, initial test, question 15 (Brasil, 2010a, questão 15)

Therefore, in question 15, if children read the word "vacinar" [vaccinate] in the poster, they will be able to answer the question correctly, since the optional answers are read by the test administrator.

In question 23, the text format and reading the text already indicate the correct answer, particularly because the word that appears in the title is present in the correct option.

Figure 2:
Brazil Exam 2010, initial test, question 23 (Brasil, 2010a, questão 23)

Therefore, we can see that, with question 15, the image and the reading of alternatives by the test administrator will favor the possibility of answering the question without actually reading the text. If the child tries to read the text and is able to read the word vacinar [vaccinate], it can reach the correct answer, which is read by the teacher: Informar sobre a vacinação [Provide information about vaccinations]. With question 23, reading the title could be a clue for the child to mark the option Fazer uma brincadeira de adivinhação [Play a guessing game].

In the final assessment, that same ability - recognizing the goal of the text - was assessed in items 17 and 21. Question 17 suggests the reading of a non-verbal text.

Figure 3:
Brazil Exam 2010, final test, question 17 (Brasil, 2010b, questão 17)

As with question 15 of the initial assessment, in the final test, the examiner reads the optional responses to question 21. Thus, if the child is able to interpret the non-verbal text, it can answer correctly. There is no need to be able to read verbal texts. In contrast, question 21 requires reading the text and the optional responses to reach the correct answer.

Figure 4:
Brazil Exam 2010, final test, question 21 (Brasil, 2010b, questão 21)

Comparing the four questions above, which aim to assess a student's ability to recognize the purpose of texts, we can say that only one of them requires the autonomous reading of a verbal text to recognize its purpose. The others require the reading of non-verbal texts or can be answered based on the reading of the title or isolated words in the text.

Therefore, only question 21 of the final test actually required children to recognize the goal of a verbal text by conducting an autonomous reading of it. Thus, we can see that, with regard to that ability, the final exam was more difficult than the initial one.

With regard to the ability to find information, the initial exam included two such items (18 and 20). Item 18 had a very short text (a short comic strip). The answer appears in one of the two balloons, that is the one indicating what the girl says, since the other indicates the mother's question.

Figure 5:
Brazil Exam 2010, initial test, question 18 (Brasil, 2010a, questão 18)

The fact that the question refers to what the character Magali asked for gives the child a clue that it can look for the answer in the girl's speech balloon. To respond to the item, the child only has to read the second balloon, which is very short: livro de receitas [cookbook]. In the correct option, the expression "cookbook" appears explicitly.

Item 20 requires reading the text presented to find the correct answer.

Figure 6:
Brazil Exam 2010, initial test, question 20 (Brasil, 2010a, questão 20)

In the final exam, questions 23 and 24 were dedicated to assessing the ability to locate information.

Figure 7:
Brazil Exam 2010, final test, question 23 (Brasil, 2010b, questão 23)

With regard to question 23, besides the fact that many children probably know the song's lyrics by heart, the question Where is the cururu frog? can be answered based on the image or by reading the first sentence of the text, unlike question 24 below.

Figure 8:
Brazil Exam 2010, final test, question 24 (Brasil, 2010b, questão 24)

As we can see, question 24 of the final exam and question 20 on the initial exam required the child to actually read the text. By comparing the four questions dedicated to assessing the ability to locate information in the text, we can see that two of them (one in the initial test and one in the final test) required children to conduct autonomous reading of a verbal text, and two of them could be answered with the support of images or by reading a very short passage of the text.

The ability to identify the subject of the text was assessed, in the initial test, by questions 21 and 24. Question 21 could be answered based on a comparison between the underlined sentence and the sentences that appear in the optional responses, since they are very similar.

Figure 9:
Brazil Exam 2010, initial test, question 21 (Brasil, 2010a, questão 21)

Question 24, however, requires more from the children, since it asks them to read the text Por que dormimos? [Why do we sleep?].

Figure 10:
Brazil Exam 2010, initial test, question 24 (Brasil, 2010a, questão 24)

In the final assessment, questions 19 and 22 were used to evaluate the ability to identify the theme of the text.

Figure 11:
Brazil Exam 2010, final test, question 19 (Brasil, 2010b, questão 19)

Figure 12:
Brazil Exam 2010, final test, question 22 (Brasil, 2010b, questão 22)

Question 19 is more complex. To fully understand it, the child must read the text and grasp its general meaning to recognize the theme. Question 22 can be answered by simply reading the first sentence and the first choice of answers. However, the presence of words such as combata, guerra and soldado [combat, war and soldier] could induce marking option C. Therefore, both questions required inferences, although question 19 required a greater ability to autonomously read the text.

Therefore, we can see that, of the four questions related to the ability to identify the subject of the text, two of them (one in the initial exam and one on the final exam) actually required greater reading autonomy, and another question in the initial exam required knowledge of vocabulary and drawing inferences, since children could make a more literal interpretation that the text was in fact about war.

Finally, with regard to the ability to make inferences, two questions were used in the initial test (19 and 22) and one in the final assessment (20). This oscillation in the number of items that evaluate each ability seems problematic, since it reduces the comparability between the two editions of the Brazil Exam.

Question 19 of the initial exam used a song that is widely known to children (O Sapo não Lava o Pé [The Frog Doesn't Wash its Feet]).

Figure 13:
Brazil Exam 2010, initial test, question 19 (Brasil, 2010a, questão 19)

By recognizing the song, children could answer the question relying not only on autonomous reading of the text, but on their knowledge of the lyrics (from memory). Of course, we could say that the question still required inferential elaboration, not from the text autonomously read, but from the text that is known by heart. Autonomous reading was thus required only for reading the optional responses. However, a closer analysis also shows that the question could be answered by simply retrieving previous knowledge. In other words, the answer to the question Por que o sapo tem chulé? [Why does the frog have smelly feet?] could come from previous experiences in children's lives, dispensing with the actual reading of the text. Therefore, this question was very easy for children.

The other question (question 22), however, required the ability to relate information in the text to previous knowledge, as we can be seen below.

Figure 14:
Brazil Exam 2010, initial test, question 22 (Brasil, 2010a, questão 22)

In sum, in the initial assessment, one of the questions was very easy, since it did not necessarily require inferential elaboration (question 19); however, the other required the reading of a short text (with only two sentences) and inferential elaboration. Since the data are computed together, question 19 raised the amount of correct answers for this descriptor in the initial edition of the test.

In the final exam, inferential elaboration was assessed by only one question. Question 20 required inference in the reading of a short comic strip, which is a genre that usually requires considerable inferential elaboration to establish meanings. However, because the text was predominantly non-verbal, it did not require mastering the autonomous reading of verbal texts. Moreover, in the question in the initial assessment, reading the alternatives required autonomous reading, as we can see below.

Figure 15:
Brazil Exam 2010, final test, question 20 (Brasil, 2010b, questão 20)

Analyzing the questions of the Brazil Exam that were answered by the children allowed us to see that, of the three inference-based questions, question 19 of the initial assessment was the easiest one, offsetting the results of question 22, which was a little more difficult than question 20 of the final exam. Therefore, we concluded that there is no actual equivalence between the initial and the final exams, although we can point out that the differences in terms of complexity are not significant, according to the account of Mariana (a seven-year-old child who can read and write):

The one with the frog is impossible to get wrong. Everybody knows that song and knows that someone who doesn't wash their feet, has smelly feet. The one with the dog is easy too, because you just have to understand the drawings. The one with the umbrella is more difficult, but it's also easy. If you pay attention, you won't get it wrong.

The reflections above, which refer to a judgment of the items based on a pedagogical analysis of the questions, reveal that few questions actually required an autonomous reading of the texts to answer the questions correctly. The oscillation in the complexity level of the items used in different editions of the Brazil Exam has also been pointed out by Thaís Silva (2013).

It is important to emphasize that reading, as construction of meaning, is a very unique phenomenon because it is only made concrete when there is a significant reconstruction of a statement, that takes place in a social interaction. From this perspective, the act of reading involves much more than the ability to turn codes into sounds or the simple attribution of meaning to words. As Marcuschi (2008, p. 228), Kock and Elias (2013, p. 11) and Solé (1998, p. 22) emphasize, to read is to construct meanings.

By analyzing the amount of correct answers to the questions presented above, we can confirm the hypotheses raised earlier about a questions' level of difficulty. As can be seen in Table 1, the questions requiring autonomous reading for each descriptor were correctly answered by a smaller number of children.

Table 1
Number of correct answers on the Brazil Exam’s initial and final assessments

Descriptor	First Test		Final Test
	Question	Percentage of children who answered correctly	Question	Percentage of children who answered correctly
Purpose of the Text	15	59	17	85
	23	55	21	43
Locating Information	18	60	23	70
	20	53	24	48
Identifying the Subject of the Text	21	69	19	61
	24	45	22	69
Developing an Inference	19	53	20	58
	22	43	---	--

• Source: Bank survey data.
  Prepared by the authors.

This table indicates that, by the end of the school year, even considering that most of the questions did not actually require the autonomous reading of texts, many children were unable to answer the questions proposed. On the other hand, the data also reveal a possible "worsening" in children's performance in some skills. This result, as mentioned earlier, may be due to the fact that the tests did not have the same level of complexity. Other perspectives, such as those of Porto (2011) and Silva (2013, T.T.) also reveal this problem in the Brazil Exam administered in the fifth grade of basic education.

Returning to the questions presented above, we can present a few partial reflections. An aspect worth considering is that, although the Brazil Exam assesses reading skills that are important for educating readers, the questions in the editions we analyzed required limited command of autonomous reading of verbal texts. Therefore, the tests we examined were not actually instruments that allowed a diagnosis of children's ability to read texts. Another fact to be emphasized is that, since the levels of complexity of the texts issued at the beginning and the end of the school year were not the same, one cannot say for sure what the progress was in each class.

Therefore, in 2010, the Brazil Exam does not seem to have significantly helped teachers understand what skills the children had already acquired by the time of the initial assessment, thus failing to support planning of the teaching of reading texts. Moreover, it does not seem to have significantly helped teachers assess, by the end of the year, the progress made. Thus, it does not seem to have succeeded in what Horta Neto (2010) recognizes as the goal of external assessments, which would be "to learn more about the educational process in order to seek improvement".

As mentioned earlier, one of the reasons this assessment failed to make a contribution is that it fails to provide reliable indications about the progress of classes due to the lack of equivalence between the instruments. As discussed by Silva (2013, F. S.), various factors determine the complexity of a reading assessment instrument, such as text complexity, size, vocabulary, the location where information can be found in the text, the kind of previous knowledge necessary to solve questions, the alternatives provided for children to choose from and others. In the tests analyzed, these aspects do not seem to have been used as references for composing the tests.

In the same direction, Colomer & Camps (2002) assert that reading assessment instruments often fail to consider important aspects in the assessment process.

Many reading tests are based on a very general concept of reading comprehension, in which reading comprehension is inseparable from factors that are common to different mental processes. When a test is not based on a more current concept of reading comprehension, as the articulation of a complex set of skills, one cannot know exactly what is being measured and, therefore, what the result depends on. This concept of reading precludes considering the need to create measuring instruments that distinguish between different types of text and the capacity to adapt reading to the readers intent. On the other hand, the lack of clear knowledge about the aspects involved encompasses various problems, both in the content of tests and in the interpretation of results. (idem, p. 174)

Based on these considerations, we understand that for the tests to be used for a diagnostic purpose, it is more important to conduct an analysis by items than by scales, which, as indicated by Morais, Leal and Albuquerque (2009) have no theoretical support, since they presuppose a linear gradation among skills, which is not supported by research on reading.

Considering this finding, we reflect on the second question raised at the beginning of this article: What are some of the possible relations between the results on the Brazil Exam and teachers' practices?

RESULTS OF THE EVALUATION OF CHILDREN ON THE BRAZIL EXAM

In the survey that was the basis for the reflections presented in this article, we analyzed the results of 12 classes in the Brazil Exam in 2010, encompassing schools in three municipalities in the metropolitan region of Recife. However, the data for class 4 in municipality 3 were not used, because the teacher helped the children to take the Brazil Exam at the end of the year. Therefore, 11 classes were actually investigated.

Of the 11 classes, we selected for discussion in this article the two classes that had the greatest improvement in performance and two that showed no change. The table below shows the percentage of children who got more than 20 correct answers on the test in each class.

We decided to consider children who got 20 or more correct answers because, as discussed previously, most of the items assessed knowledge and skills that focused on mastering the alphabetic writing system. Therefore, a child who could read words or sentences could get 18 or more answers right, since, as mentioned earlier, some text comprehension questions could be answered based on the reading of a sentence (very short texts or when the focus of the question only required the reading of the title) or the reading of non-verbal texts.

As shown in Table 2, all classes had children who, by the end of the year, could read texts. This result is important to refute the idea that two years are not enough to teach children to read and write. Nevertheless, we found that for two classes, the percentage of correct answers was greater at the beginning of the year than at the end, as mentioned earlier (classes 1 and 11). A few hypotheses can be raised: (1) correct answers reached by chance at the beginning of the year caused results to be slightly better; (2) the test at the beginning of the year was easier than the one used at the end of the year; (3) new students entered the classes in the middle of the year, or some were absent on the day of the assessment. As shown earlier, the analyses of the tests revealed that the levels of complexity of the instruments were not the same.

Table 2
Analysis of the overall performance of 11 classes in the initial and final tests of the brazil exam: percentage of students who got more than 20 correct answers on the Brazil Exam

Municipality	Class	First Test (%)	Final Test (%)	Difference between the percentages (final – initial)
Municipality A	01	26	15	-11
	02	0	36	36
	03	31	54	23
	04	10	37	27
Municipality B	05	10	30	20
	06	25	37	12
	07	10	20	10
	08	16	36	20
Municipality C	09	40	75	35
	10	76	80	04
	11	35	25	– 10

• Source: Bank survey data.
  Prepared by the authors.

Although a "regression" occurred in two classes, we can say that, in general, the children improved significantly during the school year, because in eight classes the results at the end of the year were better than in the beginning. It is also noteworthy that in two classes, the number of children with more than 20 correct answers was higher (classes 2 and 9).

To deepen our discussion, we decided to focus our analyses on the classes that had the greatest progress and on two classes in which there was no progress at all according to the results in the Brazil Exam.

As we can see in the comparison of classes shown in Table 2, class 2 had the best progress, because it had the biggest difference between the beginning and the end of the year in the number of children who got more than 20 correct answers. The question, therefore, is: What is the practice of that teacher concerning the core didactic area of reading?

As mentioned earlier, in the last section of the investigation, we sought to reflect on the work with reading in the classes examined.

THE CLASSES GIVEN BY THE TEACHERS

Continuing our analyses, we investigated whether text reading situations were favored during the classes observed in the four selected classes (i.e., the ones with the best and worst results). The table below summarizes these data, indicating the total days in which the teachers conducted these activities on the nine days each class was observed over the school year.

Table 3
Frequency of types of activities conducted in class by teacher

Types of Activities	Teachers / classes
	1	2	9	10
Text Reading by the Teacher	7	2	4	6
Collective Text Reading	2	2	-	2
Text Reading by the student	1	2	3	5
Silent Reading	2	-	2	1
Total	12	06	09	14

• Source: Bank survey data.
  Prepared by the authors.

A first reflection to be made based on the data is that the frequency of text reading activities conducted by teachers (whether collectively or individually) was low for all the classes. Considering that the children were in a stage of acquiring reading and writing skills, we could expect the reading of texts to occur daily. This low frequency of activities related to the teaching of reading skills could signal the understanding that mastering the basic knowledge related to "decoding" text would be enough for students to develop reading comprehension skills. This counters the proposals of various authors such as Bofarull (2003), Serra and Oller (2003), Colomer and Camps (2002), Solé (1998) and others.

By analyzing the table summarizing reading activities, we can see that classes 1 and 10 performed more reading activities. The class with the greatest progress in terms of the number of children who got more than 20 correct answers in the test examined (class 2) had little promotion of text reading situations (only six classes).

One hypothesis for this result can be that the Brazil Exam gives more emphasis to knowledge of the writing system than to actual text comprehension skills. Indeed, our class observations revealed that teacher 2 conducted a range of systematized activities of appropriation of the alphabetic writing system (using crosswords, word searches, sentence and word writing exercises and others). Therefore, we can say that the complexity levels of text reading items on the Brazil Exam might be compatible with the levels achieved by the children who managed to master the writing system, even though they may not have not reached more sophisticated levels of text comprehension. This data corroborates our initial analysis that the Brazil Exam, at least in the tests analyzed, required little in terms of autonomous reading of verbal texts.

The comparison in relation to situations of reading conducted by the teachers shows that classes 2 and 9, which had the greatest progress in Brazil Exam results, had fewer situations in which teachers read to the children. Two occurrences were found for class 2; four for class 9; six for class 10; seven for class 1.

This indicates that increased text reading by teachers does not seem to be enough to expand students' reading skills. In fact, it is more important to know what types of activities are conducted with the texts that are read. The example below, extracted from a class report from teacher 1, illustrates a way of working with texts that does not help to build reading skills.

Teacher 1 “At 1:55 pm, the teacher reads a poem (…). After finishing the reading, at 2:05 pm, she hands out an activity focused on the concepts of male and female. Then she corrects it on the board. Male Female O menino [The boy] O gato [The male cat] O sapo [The male frog] A menina [The girl] A gata [The female cat] A sapa [The female frog] At the end of the explanation, the teacher asks if pinto [chick] (a word from the text) is male or female. The children answer that it is female. She explains again that “o menino” is male and “a menina” is female. Then, the children correct themselves, saying that pinto is male because it is a boy. Then, everybody laughed and were amused at the fact they had initially called the pinto a girl. At 2:45 pm, the teacher asked the gender of the other words in the text. (Is limão [lemon] male or female? And sol [sun]? Andverão [summer]?). After this conversation, the teacher explains the second question of the activity. She> says that everybody will have to find the female words.”

Teacher 1
“At 1:55 pm, the teacher reads a poem (…). After finishing the reading, at 2:05 pm, she hands out an activity focused on the concepts of male and female. Then she corrects it on the board.
Male	Female
O menino [The boy] O gato [The male cat] O sapo [The male frog]	A menina [The girl] A gata [The female cat] A sapa [The female frog]
At the end of the explanation, the teacher asks if pinto [chick] (a word from the text) is male or female. The children answer that it is female. She explains again that “o menino” is male and “a menina” is female. Then, the children correct themselves, saying that pinto is male because it is a boy. Then, everybody laughed and were amused at the fact they had initially called the pinto a girl. At 2:45 pm, the teacher asked the gender of the other words in the text. (Is limão [lemon] male or female? And sol [sun]? And verão [summer]?). After this conversation, the teacher explains the second question of the activity. She > says that everybody will have to find the female words.”

As we can see, although the teacher read to the children, the focus of the activity was not on the constitution of textual meanings. Similarly to what occurred on that day, we found that other reading activities conducted by the teachers usually did not focus on building reading skills. They were commonly a pretext for approaching other contents.

In combination with this question, it is worth highlighting that in few situations did the children themselves read texts.

Therefore, one hypothesis to explain why the greatest improvement was achieved by a class whose teacher promoted few text reading activities is that because children in this class (group 2) were in a very early stage in learning the writing system, and their results in the initial assessment (at the beginning of the year) were very poor, the didactic intervention focusing on the learning of the writing system had a significant effect on the final results. However, this does not mean that the same type of practice is sufficient for building reading strategies, which are so essential in life and participation in various social situations. Only 36% of the children in this class got more than 20 correct answers by the end of the year.

This reflection is also supported by the analyses of classes 9 and 10, which already had quite advanced levels at the beginning of the year, and displayed little progress. In other words, the children needed to expand their capacity to work with texts, but they did not receive intervention inducive to that progress. There was a high frequency of activities that merely involved copying text in these classes. On the other hand, class 1, which had many children in early writing stages and did not have a didactic intervention that promoted comprehension of the basic alphabet, was the one with the lowest rate of progress.

FINAL CONSIDERATIONS

A disturbing finding in this study was that even the teachers in charge of the classes that had the best improvement in scores on the Brazil Exam had little dedication to reading activities, particularly activities that challenged children to try to read texts autonomously.

Our data show, therefore, that the impacts on the Brazil Exam results are likely to have stemmed from didactic intervention focused on learning the alphabetic system. Thus, we can say that teaching alphabetic writing promotes improvements in the reading of texts. However, we must be cautious, since we have also stressed that the level of text comprehension demanded by the Brazil Exam may not have required the ability to comprehend more elaborate texts, which is held by more skilled readers. Thus, children who master the alphabetic writing system but show difficulties in understanding texts could perform well on items of the Brazil Exam that could lead us to classify them at the highest levels in the test.

Therefore, we suggest that items requiring a higher level of reading autonomy be tested in the Brazil Exam to truly apprehend whether verbal text reading skills have been successfully acquired by children.

Moreover, we emphasize the need for further analyses of the design of tests and an effort to attain an equivalence between them, to ensure greater comparability between the beginning and the end of the year, since, as we have shown, with regard to inferential skills, the two 2010 tests did not seem to be equivalent.

Finally, we suggest that the method for providing teachers with feedback to the Brazil Exam results be changed to emphasize an analysis of the skills and items used, rather than presenting result scales that do little to help teachers understand what the children are capable of doing.

REFERÊNCIAS

• Albuquerque, E. B. C.; Morais, A. G.; Ferreira, A. T. B. As práticas cotidianas de alfabetização: o que fazem as professoras? Revista Brasileira de Educação, Rio de Janeiro: ANPEd; Campinas: Autores Associados, v. 13, n. 38, p. 252-264, maio/ago. 2008.
• Bofarull, M. T. Avaliação da compreensão da leitura. Propostas de um roteiro de observação. In: Teberosky, A. et al. Compreensão da leitura: a língua como procedimento. Porto Alegre: ArtMed, 2003. p. 127-136.
• Brandão, A. C. P.; Rosa, E. C. S. (Orgs.). Leitura e produção de textos na alfabetização. Belo Horizonte: Autêntica, 2005.
• ______.; ______. (Orgs.). Ler e escrever na educação infantil: discutindo práticas pedagógicas. Belo Horizonte: Autêntica, 2010.
• Brasil. Instituto Nacional de Estudos e Pesquisas Educacionais "Anísio Teixeira". Provinha Brasil - avaliando a alfabetização. Caderno do aluno, Teste 1, 1º semestre 2010. Brasília, DF: MEC/INEP, 2010a.
• ______. Instituto Nacional de Estudos e Pesquisas Educacionais "Anísio Teixeira". Provinha Brasil - avaliando a alfabetização. Caderno do aluno, Teste 2, 2º semestre 2010. Brasília, DF: MEC/INEP, 2010b.
• Colomer, T.; Camps, A. Ensinar a ler, ensinar a compreender Porto Alegre: ArtMed, 2002.
• Cruz, M. C. S. Alfabetizar letrando: alguns desafios do 1º ciclo no ensino fundamental. Recife: Editora Universitária da UFPE, 2008.
• ______. Tecendo a alfabetização no chão das escolas seriada e ciclada: a fabricação das práticas de alfabetização e a aprendizagem da escrita e da leitura pelas crianças. 2012. 341f. Tese (Doutorado em Educação) - Universidade Federal de Pernambuco, Recife, 2012.
• Frade, I. C. A. S. Formas de organização do trabalho de alfabetização e letramento. In: Brasil. Ministério da Educação. Secretaria de Educação Básica. Boletim 09: alfabetização e letramento na infância. Brasília, DF: MEC, 2005. p. 28-41.
• Franco, C.; Brooke, N.; Alves, F. Estudo longitudinal sobre qualidade e equidade no ensino fundamental brasileiro: GERES 2005. Ensaio: Avaliação e Políticas Públicas em Educação, Rio de Janeiro: CESGRANRIO, v. 16, n. 61, p. 625-638, out./dez. 2008.
• Horta Neto, J. L. Avaliação externa de escolas e sistemas: questões presentes no debate sobre o tema. Revista Brasileira de Estudos Pedagógicos, Brasília: INEP, v. 91, n. 227, p. 84-104, jan./abr. 2010.
• Kleiman, Â. B. Compreensão leitora. In: Frade, I. C. A. S.; Costa Val, M. G.; Bregunci, M. G. C. (Orgs.). Glossário CEALE: termos de alfabetização, leitura e escrita para educadores. Belo Horizonte: UFMG/Faculdade de Educação, 2014. p. 61-62.
• Koch, I. V.; Elias, V. M. Ler e compreender: os sentidos do texto. 3. ed. São Paulo: Contexto, 2013.
• Leal, T. F.; Albuquerque, E. B. C. (Orgs.). Alfabetização de jovens e adultos em uma perspectiva de letramento. Belo Horizonte: Autêntica, 2004.
• ______.; ______.; Morais, A. G. (Orgs.). Alfabetizar letrando na EJA: fundamentos teóricos e propostas didáticas. Belo Horizonte: Autêntica, 2010.
• ______.; Brandão, A. C. P. A. Alfabetização e ensino de língua portuguesa: investigando o currículo no Brasil. Brasília, DF: CNPq, 2012. (Relatório de Pesquisa)
• Maciel, F. I. P.; Baptista, M. C.; Monteiro, S. M. A criança de 6 anos, a linguagem escrita e o ensino fundamental de 9 anos: orientações para o trabalho com a linguagem escrita em turmas de seis anos de idade. Belo Horizonte: UFMG/FAE/CEALE, 2009.
• ______.; Lúcio, I. S. Os conceitos de alfabetização e letramento e os desafios da articulação entre teoria e prática. In: Castanheira, M. L.; Maciel, F.; Martins, R. (Orgs.). Alfabetização e letramento na sala de aula Belo Horizonte: Autêntica Editora: CEALE, 2008. p. 13-33.
• Marcuschi, L. A. Produção textual, análise de gênero e compreensão São Paulo: Parábola Editorial, 2008.
• Morais, A. G. Sistema de escrita alfabética. São Paulo: Melhoramentos, 2012.
• ______.; Leal, T. F.; Albuquerque, E. B. C. Provinha Brasil: monitoramento de aprendizagens e formulação de políticas educacionais. Revista Brasileira de Política e Administração da Educação, Porto Alegre: UFRGS, v. 25, n. 2, p. 301-320, maio/ago. 2009.
• Oliveira, S. A. Progressão das atividades de língua portuguesa e o tratamento dado à heterogeneidade das aprendizagens: um estudo da prática docente no contexto de ciclos. Recife, 2010. 450f. Tese (Doutorado em Educação) - Universidade Federal de Pernambuco, Recife, 2010.
• Picolli, L.; Camini, P. Práticas pedagógicas em alfabetização: espaço, tempo e corporeidade. Porto Alegre: Edelbra, 2012.
• Porto, C. C. R. Práticas de ensino de compreensão de leitura e conhecimentos de alunos do último ano do ensino fundamental I. 2011. 221f. Dissertação (Mestrado em Educação) -Universidade Federal de Pernambuco, Recife, 2011.
• Santos, C.; Albuquerque, E. Alfabetizar letrando. In: Santos, C. F.; Mendonça, M. (Orgs.). Alfabetização e letramento: conceitos e relações. Belo Horizonte: Autêntica, 2005. p. 95-109.
• Saviani, D. Pedagogia histórico-crítica: primeiras aproximações. 10. ed. Campinas: Autores Associados, 2008.
• Serra, J.; Oller, C. Estratégias de leitura e compreensão do texto no ensino fundamental e médio. In: Teberosky, A. et al. Compreensão da leitura: a língua como procedimento. Porto Alegre: ArtMed, 2003. p. 35-43.
• Silva, C. S. R. O planejamento das práticas escolares de alfabetização e letramento. In: Castanheira, M. L.; Maciel, F.; Martins, R. (Orgs.). Alfabetização e letramento na sala de aula Belo Horizonte: Autêntica; CEALE, 2008. p. 35-58.
• ______. O processo de alfabetização no contexto do ensino fundamental de nove anos. In: Rangel, E. O.; Rojo, R. H. R. Língua portuguesa: ensino fundamental. Brasília, DF: MEC/SEB, 2007. p. 37-64.
• Silva, F. S. Leitura e Prova Brasil: ensino e avaliação. 2013. 263f. Tese (Doutorado em Educação) -Universidade Federal de Pernambuco, Recife, 2013.
• Silva, T. T. Avaliação da alfabetização: um exame de diferentes edições da Provinha Brasil. 2013. 256f. Dissertação (Mestrado em Educação) -Universidade Federal de Pernambuco, Recife, 2013.
• Soares, M. Alfabetização e letramento São Paulo: Contexto, 2003.
• ______. Letramento e alfabetização: as muitas facetas. Revista Brasileira de Educação, Rio de Janeiro: ANPEd; Campinas: Autores Associados, n. 25, p. 5-17, 2004.
• Solé, I. Estratégias de leitura. Tradução de Cláudia Schilling. 6. ed. Porto Alegre: ArtMed, 1998.
• Souza, I. P. M.; Cardoso, C. J. Práticas de alfabetização e letramento: o fazer pedagógico de uma alfabetizadora bem-sucedida. In: Reunião Nacional da Associação Nacional de Pós-Graduação e Pesquisa em Educação, 35., 2012, Caxambu. Anais... Caxambu: ANPEd, 2012. p. 1-16.

Publication Dates

• Publication in this collection
  Jan-Mar 2017

History

• Received
  27 Nov 2014
• Accepted
  16 Feb 2016