Application of Project-Based Learning in the first year of an Industrial Engineering Program: lessons learned and challenges

Abstract

The Industrial Engineering Program at the School of Engineering of Lorena at the University of São Paulo has been using the Project-Based Learning (PBL) methodology in the first term of the academic year since 2013. This article describes the experience of the first four years of the use, the lessons learned, and the remaining challenges. The use of PBL has been widely recognized by the students as one of the Industrial Engineering program’s differentiating features. Lessons have been learned continuously and have been subsequently incorporated to the application of the project. Much has been refined during this time frame but there is much more to be done. Some challenges persist, such as: (i) improving interaction with the course of Calculus I; (ii) enhancing peer evaluation; (iii) expanding the project coordination team, and (iv) refining tools for data collection and analysis

Keywords
Active learning; Engineering education; Project-Based Learning; Freshmen; Industrial Engineering

1. Introduction

Cultural, social, and especially technological changes are occurring at an incredible pace. Similarly, labor market requirements are also changing, and at a similar speed. These changes directly affect the formation of engineers, which nonetheless remains focused, at most schools, on traditional models of teaching. That is, the teacher is the active agent of knowledge transmission, and the student acts as the passive agent. Yet when knowledge is more widely disseminated by means of various information technology platforms, students increasingly feel less attracted to this traditional model. Thus, the degree of learning is lower because most of the time students limited their study time to “pass a test” rather than more deeply assimilating knowledge.

We need to rethink education as a whole, and this includes engineering education. For Felder (2006), it is critical that future engineers know how to work in a group and face the inner reality of a company that is increasing its dynamism.

United Nations Educational, Scientific and Cultural Organization (2010, p. 6) published a study on engineering challenges for the next years, which included “[…] to transform engineering education, curricula, and teaching methods to emphasize relevance and problem-solving approach to engineering”. Another highlight of this document is the importance of having problem and project based educational activities, to prepare students to face future challenges as engineers. In response to the report, leading universities have turned to various active learning methods which propose the learning to be focused on the student solving real world problems in an academic setting.

The inclusion of PBL in engineering curricula is considered one of the most efficient ways to enhance student learning, and it has been recognized as an effective way to prepare students for professional careers (Helle et al., 2006; Jollands et al., 2012; Litzinger et al., 2011; Prince & Felder, 2006). According to Lu (2007), students who participate in activities focused on PBL are more participative and proactive, communicate better, know how to work in groups, and stand out as top professionals, since they are able to better apply the acquired knowledge to solve a professional problem.

Multiple experiences of PBL use to first-year engineering students can be found in the literature (Kilgore et al., 2007; Lima et al., 2012; Montero & Gonzalez, 2009; Palmer & Hall, 2011). This article describes four years of PBL application during the first semester of Industrial Engineering and points out the results and learning, year after year, as well as remaining challenges.

2. Project-Based Learning (PBL)

PBL is an active learning method which aims to engage students in acquiring knowledge and skills through real-world experiences and well planned activities.

One early definition for PBL was given by Adderley et al. (1975). For them, PBL involves five distinct aspects: (i) resolution of a problem that can be proposed by the students themselves, though it is not required to be; (ii) initiative to solve the problem comes from the students and requires integration of a range of educational activities; (iii) delivery of a final product, coherent with the initial problem; (iv) the solution for the problem will usually be handled as a project and, most of the time, it is time consuming and (v) changing the role of the instructor from an authoritarian position to a consultant position.

More recently, Thomas (2000) sought to answer the question: “What must a project have in order to be considered an example of PBL?” presents five core criteria for a successful approach: (i) centralization – fundamental strategy to the development of activities; (ii) existence of a “driving” question; (iii) necessity of a constructive investigation – by the students; (iv) incentive to autonomy and (v) realism – focusing on problems related to students’ reality.

This methodology involves students working in teams to solve concrete problems using the theory in practice. Furthermore, they must also learn to relate what they are learning to their future profession. Moreover, for them, PBL should place the student as the main actor of the teaching-learning process and relate content from various courses on a project. The students also need to delegate tasks for each group member and integrate the different parts into the final project (Powell & Weenk, 2003).

Helle et al. (2006) sought to define and distinguish PBL pedagogical or psychological reasons in this kind of pedagogical approach. For them, the most important feature of PBL is having a defined problem to be solved, which serves as guide for learning. Furthermore, they have proposed several additional reasons to justify the use of PBL. Among them are: how the proposition of the project forces the students to do a series of learning activities while developing the project, how students can control the development of the project and of the learning path, how the problem is inserted in a real-world context, and how they have multiple paths and forms of representation to solve the problem, especially given that each student has a different understanding of the world and can contribute in different ways.

PBL should lead students in search of open problem solving, as well as the acquisition of skills, such as problem-solving ability, oral communication, written communication and teamwork, among others. In such learning environment, the students are able to create their own knowledge, at their own pace. They also develop self-directed learning skills and critical thinking. While developing their projects, the students are also exposed to many aspects of a given process. This brings them closer to real life expectations. Studies show that the acquired and developed skills are aligned with those possessed by more experienced engineers (Duch et al., 2001; Frank et al., 2003; Prince & Felder, 2006).

To Costa et al. (2007) and Mills & Treagust (2003), the labor market is demanding extraordinary professional skills and just knowledge is not enough. Thus, teaching through PBL provides many benefits for students, and improves their academic development (Costa et al., 2007). These last authors emphasize, among others, the following benefits for students: they do not only gain knowledge, but they learn to do a project; they practice their skills and acquire others; they know how to behave in a group; they gain as practical activity, as it approaches those of their profession. In addition, the authors propose that: projects, whenever possible, should involve the university and the communities in surroundings; should evaluate students based on the reality that they will find in the labor market; and should increase communication and unity within the classrooms.

3. Research methodology

The methodology used in this research has a qualitative nature, based on evidences collected each year from instructors and students in a first-semester course using PBL methodology between 2013 and 2016 in the Industrial Engineering program, at School of Engineering of Lorena.

Action research is the methodology used every year to plan, execute and analyze the results and then to propose improvements to be implemented in the next year’s class.

The methodology of action research is qualitative and empirical, where the researcher studies phenomena through observation and interaction with the research subject. It also has a participative character since it involves the research object in the action (Coughlan & Coghlan, 2002; Westbrook, 1995). Action research is also characterized by its constant cycles of intervention and reflection (Checkland, 1991; Lewin, 1946). Each one of the cycles should be conducted through the following steps, as proposed by Coughlan & Coghlan (2002) and Mello et al. (2012): Data Gathering, Data Collection, Data Analysis and Action Planning, Implementation and Evaluation.

3.1. Data gathering

The first year the project-based class was implemented was 2013. For this class, the step of data gathering is distinct from the other years, because it is the project genesis. In 2012, some instructors from course coordination visited the Massachusetts Institute of Technology (MIT) and Harvard University in the United States, the University of Minho in Portugal, and the University of Brasilia in Brazil. These visits had as an objective to learn about the experience of engineers’ undergraduate program in each one of those universities. At the two American universities, the learning acquired was regarding the importance of preparing engineers for problems resolution through active learning methodologies, such as the CDIO approach at MIT (Crawley et al., 2008; Svensson & Gunnarsson, 2012) or peer instruction at Harvard University (Crouch & Mazur, 2001; Miller et al., 2015). At University of Minho, the learning acquired was about the implementation of PBL for first-year Industrial Management and Engineering students in a class entirely devoted to a project involving interdisciplinary activities with other classes from the first term of the academic year (Fernandes et al., 2012; Lima et al., 2007). Lastly, at University of Brasilia, the learning acquired was about the application of PBL at the Industrial Engineering Program in specific project courses from the fourth to the final (tenth) semester before graduation (Lima et al., 2012). At the end of 2012, the coordination team decided to initiate PBL application with the freshmen students beginning in 2013 and using a format very similar to the one implemented at University of Minho.

From 2014, with the second class of freshmen, the data gathering step occurred after the end of each semester, when, from experience and data collected from all parts involved in the last application of the project, improvements were proposed. So, every year a new cycle starts with this planning step, considering all lessons learned during the past cycle.

3.2. Data collection

Data collection instruments have been frequently utilized in PBL applications within engineering programs in order to evaluate and improve its utilization. For example, Kilgore et al. (2007) studied students’ behavior in two different design projects. They analyzed how strongly teams considered contextual factors in their decisions using both qualitative and quantitative tools. Montero & Gonzalez (2009) applied PBL to Electronic Engineering freshmen. Their objective was to create a heat sink prototype. The tools used in data collection were: (i) reports and prototype evaluation, (ii) analysis of students’ behavior during classes and (iii) questionnaires answered by the students. In another study, Palmer & Hall (2011) investigated PBL methodology itself and which of its aspects could be improved by surveying the students with an open-ended questionnaire. They applied PBL in a design class for freshmen engineering, inviting them to evaluate the methodology through the questionnaire, pointing out outstanding aspects and key aspects to be improved.

In the studied case, data collection has been a fundamental part of the improvement of each new cycle of research action. The main tools of data collection being used are participant observation, questionnaires, and interviews. The participant observation has been made by the project’s coordination team which interacts directly with all others involved in the project, every year. The questionnaires have been answered by instructors and students. And the interviews have been conducted with tutors.

3.3. Data analysis and action planning

The data analysis from multiple sources of evidences and from different perspectives allows the use of triangulation, a methodology used to observe the same phenomenon trough different perspectives in order to increase precision of results and strength of conclusions. Several specialists highlight the value of using a combination of data from different sources in light of weaknesses in qualitative research when a single source of data is used (Denzin & Lincoln, 2011; Flick, 2005; Webb, 1966).

Triangulation, as well as content analysis, has been used since the first research cycle (2013). From the third cycle (2015), an action plan was completely elaborated to be implemented in the following research cycle (2016), and it aimed to answer questions raised by Coughlan & Coghlan (2002), such as: What needs to change? In what parts of the project? What types of change are required? And whose support is required?

Content analysis (Bardin, 2001) has been used in the analysis of interviews and other narratives collected during each research cycle. It consists in a technique that explores the communication content through organization of data into different categories, according to the main topics related to the research questions.

3.4. Implementation

From 2013 to 2015, improvements have been implemented each year, especially based on participant observation and the questionnaire results, and an action plan was elaborated to be partially implemented in the following year. Starting in the 2016 cycle, the action plan was elaborated with greater methodological rigor when compared with the past three years and, in function thereof, about 90% of foreseen improvements were implemented.

3.5. Evaluation

Each year from 2013 to 2016, after the end of each research action cycle, significant learning has been achieved by the coordination team, either from data gathering and analysis of questionnaires and interviews, or from experience acquired by the instructors involved through participant observation.

Many lessons learned were during these four years, and in this present article the most significant ones were portrayed year after year. The fact is that every application has been revisited and improved, and its results are analyzed, enhancements were planned and, in the following year, implemented. The results are analyzed again, repeating the typical cycle of an Action Research.

Many relevant analyzes were made since the first class and this produced several papers presented at national (COBENGE: Brazilian Conference of Engineering Education) and international events (PAEE: Symposium on Project Approaches in Engineering Education; SEFI – Conference of Europe Society of Engineering Education; EUROMA: Conference of European Operations Management Association) as presented at Table 1.

Table 1
Papers presented at conferences from 2013 to 2016.

Objective	Event, Year
Presentation of first experience of PBL at School of Engineering of Lorena	COBENGE, 2013
PBL as a catalyst to development of soft skills	PAEE, 2014
Analysis of leader change in the middle of the semester in a PBL project	PAEE, 2014
Description of the two-year experience with PBL application	SEFI, 2014
Presentation of positive results of PBL application in a freshmen class	EUROMA, 2015
Analysis of the development of project management competence	PAEE, 2015
Advances in the development of teamwork competence over a two-year experience with PBL	PAEE, 2015
Analysis of the development of Leadership	COBENGE, 2016
Analysis of the development of teamwork competence	COBENGE, 2016
Lessons and Learnings after three years of PBL implementation	PAEE, 2016

4. Context study

The research action portrayed in this study has been conducted with a class of first-year students of Industrial Engineering at the School of Engineering of Lorena, University of São Paulo. This program was implemented in 2012 with 40 students. Since then, every year, the program receives 40 new students. The first class, 2012, did not apply PBL in their first year of course. PBL was introduced in the program in the second class, in 2013. During the years of 2013 and 2014, PBL was applied in the “Introduction to Industrial Engineering” class, where part of the class was designated to general topics about Industrial Engineering and the other part was dedicated to the project. From a successful experience of both years, a specific course for the project’s development was created, starting in 2015, which is offered during the first semester of the program: “Integrated Project in Industrial Engineering” (IPIE). The “Introduction to Industrial Engineering” class was held in the grid and returned to its main goal, which is to give a general view of Industrial Engineering for students entering the program.

4.1. The Project Guide

A Project Guide (Pereira, 2016), based on the model used by the University of Minho (Fernandes et al., 2012; Lima et al., 2007), is presented to all students enrolled in the design course in the first class of the semester. The students receive a printed of the Project Guide. The guide is a tool that introduces students to the concept of PBL and explains the main objectives to be pursued, throughout the semester, from the use of this methodology. It presents the courses integrated with the project (section 4.2), establishes the criteria to form the teams (section 4.3), and presents the coordination team (section 4.4) and the project theme (section 4.5). It also defines the responsibilities of tutors (section 4.6), and establishes the schedule of said project (section 4.7) as well as how the course will be graded (section 4.8).

The Project Guide explains that the technical skills to be acquired by students during the execution of the interdisciplinary project are the specific skills to be acquired in direct support of the project courses. In addition to technical skills, the Project Guide provides soft skills that are expected to develop in students throughout the semester: project management, teamwork, personal development and communication, as shown at Table 2. This soft skills model has been the same for the past four years.

Table 2
Soft Skills Model.

Project Management	Team Work	Personal Development	Communication
Research Ability Decision-making Ability Organizational Ability Time Management	Autonomy Initiative Responsibility Leadership Problem Resolution Interpersonal Relationship Conflict Management	Creativity/Originality Critical Thinking Auto Evaluation Autoregulation	Oral Communication Written Communication

4.2. Courses integrated with the project

The freshmen students in the Industrial Engineering program are enrolled in seven courses during the first semester and, since 2015, five of them are integrated with the project: Calculus I, General Chemistry, Experimental Chemistry, Reading and Production of Academics Texts, and Introduction to Industrial Engineering. The first four courses are common to all engineering programs of the School and, every year, instructors do a rotation regarding which group of students they are going to teach.

At the beginning of the semester, meetings take place with instructors of these five courses to present the project. After that, throughout the semester, they act as project evaluators in two moments: pre-project (tenth week) and closure (fifteenth week). Figure 1 presents the courses integrated with the project.

Figure 1
Integrated courses.

4.3. Class composition

As follows, Table 3 presents the total number of students enrolled in the course during these four years, divided into two categories: (i) freshmen, the ones entering an undergraduate program for the first time and (ii) non-freshmen, those who had already enrolled in another undergraduate program before, in another context.

Table 3
Number of students enrolled each year.

Type of Student	2013	2014	2015	2016
Freshmen	40	37	35	33
Non-freshmen	6	6	6	4
Total enrolled	46	43	41	37

Table 4 presents the number of teams formed each year. In the first class (2013), the enrolled students were divided into six different teams, without any distinction between freshmen and non-freshmen students. This approach revealed itself inappropriate since, naturally, non-freshmen students assumed leadership positions over the freshmen students. As a result, the freshmen tended to resign and the possibility of leadership development was lower. From 2014 onward, non-freshmen were put into different teams.

Table 4
Number of teams formed each year.

Teams	2013	2014	2015	2016
Number of teams	6	8	6	7
Freshmen and Non-freshmen together	6	-	-	-
Only freshmen	-	7	5	6
Only non-freshmen	-	1	1	1

4.4. Composition of the coordination team

Following, Table 5 exhibits the coordination team year after year. The course coordinator instructor has been the same since 2013. A psychology instructor, hired in 2014, entered as a definitive coordination team member in 2015. And every year different students – sometimes undergraduate, sometimes graduate students – have acted as teaching assistants of the team.

Table 5
Coordination team.

Team members	2013	2014	2015	2016
Coordinator instructor	1	1	1	1
Psychology instructor			1	1
Undergraduate students	2	2	1	1
Graduate students			1	1

4.5. Projects theme

The choice of the project theme is made by the coordination team and by the tutors before the beginning of the semester. The theme should be innovative, aligned with an actual society need, that requires use of a scientific component and, also, challenging to first-year engineering students. In 2013, the theme proposed for the project was “Sustainability of a university campus”. In 2014, the theme was: “Biofuel Production”. In 2015, the theme was “Safe drinking Water Production from unconventional sources”. And in 2016, the theme was “Global Warming”.

The project theme is presented to the students during the first week of class. In 2016, the theme was described as follows in the “Project Guide”:

Global Warning is the process of increase in the mean temperature in oceans and air of Earth’s surface caused by greenhouse gases’ emission. This increase in the temperature has been happening since 19^th century and it is expected to continue onto the 21^st. Main gases emitted by humans are carbon dioxide and methane, especially in function of burning fossil fuels, use of fertilizers and deforestation. And these gases block heat dissipation from Earth to space. Each team should present a concrete solution to contribute to the decrease of global warming (Pereira, 2016, p. 5).

Proposed solutions by the end of the semester are quite distinct. For example, the solutions proposed in 2016 were the following: (i) reduction of excess carbon dioxide from the atmosphere through growth of Chlorella sp microalga; (ii) construction of a compost for paper recycling by cultivating plants fertilized with organic material; (iii) production of biodiesel from residual vegetable oil used in food preparation; (iv) implantation of green roofs; (v) reduction of waste that blocks the entrance of sun light in water bodies; (vi) utilization of biochar to agricultural application, and (vii) construction of a vermicomposter.

4.6. Tutors

The tutor had some technical knowledge of the problem and had the responsibility to guide the group. It was not up to tutor to interfere with the actions taken by the group but to question whether these would be the best for each situation. During the first three years, the tutor was a School of Engineering of Lorena’s instructor invited by the coordination team and not necessarily responsible for one of the integrated courses. Between 2013 and 2015, about 2/3 of the tutors changed each year, since they acted as volunteers and some of them declined to participate again due to their other obligations.

In the 2016 class, one significant change occurred in the role of tutor, who rather than an instructor was now an Industrial Engineering student that had prior experience with the project. This change occurred because, during 2014 and 2015 projects, each team had a godfather/godmother, that is, an Industrial Engineering student who had prior contact with this type of project. In a natural and spontaneous way, the teams leaned more towards these students to look for help than towards the tutor, perhaps because the godparents were more reachable or because this relationship was more empathic. The coordination team, then, produced a Tutors Manual with general guidelines on what was expected from a tutor. It was defined that the tutor’s role is to guide students because they possess technical knowledge to help the team in their project. Tutors don’t have solutions, but motivate and support students while they discover their own knowledge. Their role is to support the teams when questioned about practical matters that surface on a daily basis while conducting a project, since they are more familiar with the university community.

At the end of 2015, a selection process occurred to choose the student tutors. Eighteen students signed up and seven were selected. An important aspect of this change was that during the semester there were five meetings between the Coordination team and the tutors, something that never happened in any of the previous years due to the difficulty of synchronizing schedules among the instructors. From 2013 to 2015, only two meetings took place, each year, one at the beginning of the semester and one at the end, and never with all tutors involved. In 2016, the first meeting with the tutor students happened in the first week of classes. During this occasion, every student received the Tutors Manual and detailed instructions about their roles and responsibilities. Four other meetings occurred throughout the semester. This change will remain in 2017.

4.7. Project phases and milestones

Milestones are established to adequately follow the project development, similar to the experience of a practicing engineer, where goals and deadlines are part of a professional routine. These milestones fix, in an objective way, all deliveries expected from each team throughout the semester and their deadlines. Table 6 shows all milestones for the 2016 class. Every cycle, this calendar has been improved. During the first two years, the students did three oral presentations (conceptual project, preliminary report, and final report). For the last two years, 2015 and 2016, the students had to do a fourth oral presentation about the research project, between the conceptual project presentation and the preliminary report one. The first two presentations were made to the course coordination team. The last two were made to instructors of the integrated courses.

Table 6
2016’s milestones.

Milestone	Date	Week	Activity
1	Feb, 22, 2016	1	Project Presentation; Teams Formation
2	Feb, 29, 2016	2	Oral presentation of conceptual project
3	Mar, 28, 2016	5	Delivery of research Project (maximum of 5 pages)
4	Apr, 04, 2016	6	Oral presentation of research project
5	May, 09, 2016	9	Delivery of preliminary Project (maximum of 15 pages)
	May, 16, 2016	10	Oral presentation of “pre-product”: Preliminary Report
7	June, 13, 2016	14	Delivery of Final Report (maximum of 40 pages)
8	June, 20, 2016 (morning)	15	Prototype Demonstration
9	June, 20, 2016 (afternoon)	15	Oral presentation of Project: Final Report

The conceptual project consists in the general idea the team intends to work on during the semester. The research project is the first written document where they formally write about their proposal. The preliminary report (considered as “pre-product”) is a document where each team describes in a clear way the solution they are developing. During these three oral presentations, each team has 5 minutes to present and another 5 minutes for discussion.

The final report is the closing document of the project. Each team has 20 minutes to deliver their oral presentation followed by 10 minutes of discussion.

In 2013, the final product was delivered only as a written report. From 2014, teams started to present a prototype on a bench scale, in addition to the final written report. The prototype is evaluated by two instructors. The final presentation is made to an examining board composed of two instructors who had prior access to the written report.

4.8. Project assessment

The grading for the IPIE course has evolved every cycle. In the first year (2013), the evaluation was made during the last class from the oral presentation of the final project. Since the second year, two other factors composed the grade: (i) the development of each team regarding soft skills, which was discussed in a meeting by the coordination team and the tutors, and (ii) the prototype, which was evaluated by two invited instructors, both with experience related to the project theme. Table 7, extracted from 2016 Project Guide (Pereira, 2016), details the evaluation criteria.

Table 7
Evaluation criteria.

Evaluation Criteria	Weight	How it is evaluated	Evaluators
PROJECT Research Project	20%	Written portion (10%) + Oral Presentation (10%)	Coordination team and instructor of Reading and Production of Academic Texts
PRE-PRODUCT Prelimany Report	20%	Report + Oral Presentation	Instructors of Integrated Courses
PROCESS Project Development	20%	Meeting with tutors	Tutors
PRODUCT Prototype	20%	Prototype Presentation	Instructors with experience in the field
PRODUCT Final Report	20%	Report + Oral Presentation	Instructors of Integrated Courses

Since 2015, every team receives these five grades, each one of them with a 20% weight in the composition of the final grade. However, a corrective factor is used to individualize team grades. This corrective factor is calculated based on the degree of participation and contribution to the results of each student. All students evaluate their peers and themselves.

4.9. Tools for data collection

The Table 8 presents tools for data collection used at each research cycle.

Table 8
Data collection tools.

Tool	Answered by	2013	2014	2015	2016
Questionnaire about the development of the project	Team	V	V	V	V
Questionnaire about PBL evaluation	Students	V	V	V	V
Questionnaire about development of soft skills	Tutors		V	V	V
Semi-structured interview	Tutors		V	V	V
Questionnaire about tutors’ evaluation	Students				V

The “Questionnaire about development of the Project” is an open-ended questionnaire about the development of the project – answered by the teams in the middle of the semester. The objective of this questionnaire is to evaluate the progress of the project- while in progress. Eight open questions are formulated to be answered by the team and are related to the following subjects: team meetings and their productivity; communication protocols used; definition of roles within the team; division of tasks; perceived difficulties; and suggestions of improvements. The answers are sent in writing to be analyzed by the coordination team. If necessary, adjustments are made to the second part of the semester based on the results. In the 2016 class, the analysis of those answers was done in a meeting with the coordination team and tutors.

The “Questionnaire about PBL evaluation” is a closed questionnaire about PBL evaluation answered individually by students once in the middle and again at the end of the semester. This questionnaire, in its first version in 2013, had 33 questions. It was then refined to 27 questions to be answered in the Likert scale from 1 to 5, where 1 means “completely disagree” and 5 means “completely agree”. The objective of this tool is to evaluate eight subjects: (i) PBL, (ii) team-work competence, (iii) personal development competence, (iv) communication skills, (v) project management competence, (vi) students’ perception regarding the interaction of other courses with the project, (vii) evaluation of tutors, and (viii) evaluation of leader of the student team.

The “Questionnaire about development of soft skills” was introduced in 2014. It has been answered by the tutors in the week prior to the last class of the semester. The objective is to investigate tutors’ perception regarding the development of soft skills. Moreover, the questionnaire guides tutors while they analyze the development of the teams throughout the semester. Once again, the questionnaire is answered in Likert scale, the same as used the “Questionnaire about PBL evaluation”. There was also an extra option to every question: “Can’t evaluate”.

The “Semi-structured interview” was also introduced in 2014. The interview has been conducted in the week prior to the last class of the semester. The objective is to evaluate the process of tutoring through the tutors’ eyes. It is a structured interview, conducted and recorded by one of the teaching assistants. The four base questions are related to what motivated them to become tutors, if there were gains in the process, which were the difficulties faced, and suggestions for next tutors.

The “Questionnaire about tutors’ evaluation” was introduced only in 2016. It is answered by the students during the last class of the semester. It has seven questions to be answered using a Likert scale, as presented above. The questionnaire also has one open question, where students can make suggestions for “tutors’ performance.”

Finally, a unique tool for data collection was introduced in 2015, namely monitoring the teams’ main communication channel, a WhatsApp group with all team members. The teaching assistants were members of those groups but without ever intervening. Only after the end of the semester was the entire content of the messages transferred and saved to a digital file to be further analyzed.

4.10. Lessons learned

Lessons have been learned from each research cycle. The learning leads to actions with the objective of further improving this PBL application.

In the first research cycle (2013), several positive points that stood out this first application were: (i) the fact that the students did four oral presentations during the semester, (ii) a lecture on teamwork delivered in the third week of class, (iii) the use of structured data collection tools, and (iv) enhanced motivation of students in the last class of the semester to deliver the oral presentation for the Final Project. These four positive points have been held every year since then. Only the data collection tools have been enhanced year after year. But, the main lesson drawn from this first class was that it lacked a prototype as a final product of the project, because the students just delivered a conceptual design proposal for improving campus sustainability. Another relevant difficulty detected was to the integration of General Chemistry and Calculus I courses with the project.

In the second research cycle (2014), positive points that stood out this year beyond the four already highlighted in the previous year were: (i) the construction of the prototype during the semester increased contact between students, and (ii) the presentation of a prototype in the last class. The difficulties encountered in the integration of General Chemistry class in the previous year were solved. This is due to the design to be linked with chemistry and the construction of the prototype to have been run in a chemistry laboratory. However, the difficulties in integration with Calculus I continued. Several learning opportunities occurred for the coordination team and the main change for 2015 was the creation of a course totally dedicated to the development of the project.

In the third research cycle (2015), positive points that stood out beyond those already highlighted were: (i) the psychology instructor’s entry into the coordination team, and (ii) the monitoring of WhatsApp throughout the semester by both monitors. However, difficulties with integration of Calculus I continued. Another fact that drew much attention in 2015 was that the teams were turning to their godfathers (students) more than for their tutors (instructors) to seek guidance on the project. Thus, in the 2016 class, tutors became students who had relevant experience with projects in the past.

In the fourth research cycle (2016), the most positive aspect was the student tutors’ prior experience with projects. In fact, it was observed that the tutors were closer to the teams than the instructors were. Student tutors made significant contributions to the project during the meetings, including: (i) implementation of a lecture on how to elaborate meeting minutes, and (ii) elaboration of a report template. The report template created a fixed structure with the following topics: (i) Introduction (contextualization, justification and objectives), (ii) Theoretical Validity, (iii) Materials and Methods, (iv) Results, (v) Discussion (about the prototype, technical viability, economic and financial viability and environmental viability), (vi) Conclusions, (vii) References and (viii) Appendix: Relation between the project and each integrated course. Another positive aspect was the implementation of a lecture about economic and financial viability, taught by an Economics instructor. The difficulties found in the integration of Calculus I persist, but were less significant. The instructor teaching Calculus for the freshmen students and two other colleagues from the Mathematics department acted as evaluators in the presentation of pre-project, in the middle of the semester, when they analyzed teams’ partial reports and presentations. Additionally, one of the required appendices requested that students formally addressed how concepts of Calculus I were applied in the project.

5. Challenges

Much was already done in these four years, but there is much more to do. Therefore, challenges remain to enhance PBL utilization and, as consequence, students’ learning remains.

5.1. Integration between Calculus I and the project

As noted, one significant challenge is the integration between the project and the content of the Calculus I course, a problem also faced by University of Minho, Portugal (Alves et al., 2016). As Table 9 shows, student perceptions from 2013 to 2015 reveal that they disagreed more than agreed about the existence of integration between the project and Calculus I. In the 2016 class, the results got better, but still far from suggesting a good integration. Some actions taken during the 2016 class, such as participation of Calculus I instructor as a project evaluator and the mandatory delivery of an appendix about this integration in the written report that required a real application of Calculus I knowledge in the project, helped the results seen in 2016, with students answering they agreed more than disagreed such relationship existed, but still in a very limited way.

Table 9
Integration between Calculus I and the project.

Question	2013	2014	2015	2016
I can notice a direct relationship between Calculus I and what I have been working on the project.	2.66	2.35	1.97	3.16

Calculus I is the class with higher rate of failure in the first semester of all engineering programs of School of Engineering of Lorena, which also occur in most Brazilian engineering schools. There are several possible causes for these results: deficiency of Mathematics teaching during high school, the large volume of content of the course, traditional teaching and evaluation practices used, among others. The important point is that, independent of the cause, efforts should continue to search for greater integration between Calculus I and the project, even as a motivator factor that could drive the students to study and learn more about Calculus I.

5.2. Coordination team

The project coordination team has varied over the years of implementation. Only during the third and fourth research cycle (2015 and 2016) have two instructors joined the team permanently: the coordinator instructor and the psychology instructor. The psychology instructor brings a unique contribution to the team, reflecting a diversified understanding regarding the development of soft skills. A challenge that can bring greater outcomes for the course will be the contribution of a few more instructors, who can add value by sharing their experience in other areas, as it was with the Economics instructor, who helped the students regarding the economic and financial viability analysis, a contribution that is permanent from now on. Additionally, during the four years of application, the class always counted with the collaboration of teaching assistant students, which change every year.

5.3. Peer evaluation

In the four years of the project, in the last class of the semester, a peer evaluation has been applied. Students have participated with maturity and the results have revealed that in some teams, there are some students with low participation in the final result of the project. The only concrete result of the peer evaluation has been an impact on the final grade of the student in the class.

A first solid step towards improvement of peer evaluation took place in 2016, when there was a preliminary assessment conducted by the student tutor in the middle of the semester. The student tutors were previously instructed by the psychology instructor about how they should conduct this assessment.

This meeting was positive, but the fact is the peer evaluation applied in the last class needs to be improved, in order to contribute in a given way such that every student can receive feedback about their effective participation, which has not been happening since this evaluation is done using a closed questionnaire.

5.4. Improvement of Soft Skills Model

The soft skills model (Table 2) used during these four years was based on the model adopted by University of Minho (van Hattum-Janssen & Mesquita, 2011).

After four years, we decided it was time to re-evaluate this model, which has served as starting point and has showed itself as adequate throughout these years. The objective is to improve it, developing a better alignment with the educational project of the program.

5.5. Improvement of data collection and analysis

Since its beginning, five tools were developed over the course of four years. A summary of them can be seen at Table 8.

Data collection has been working adequately over these years, but some of those tools need to be revisited to verify their effectiveness, as well as to improve them, if needed. The analysis of the numerical results from the questionnaires has been done using basic statistical calculations, such as simple and weighted means. However, it is necessary to evolve the statistical treatment of the numerical data collected, including testing some hypothesis after these first successful years of implementation of PBL.

6. Conclusion

Since 2013 PBL has been used in a project class for first-year students in an Industrial Engineering program. The study of this application has been done, year after year, through an action research, where each group of freshmen corresponds to a research cycle. Lessons are learned every year and improvements are implemented in the next cycle.

The main lessons learned and implemented, over these years, are: (i) the students present four oral presentations during the semester, (ii) the use of structured data collection tools; (iii) the construction and the presentation of the prototype; (iv) the psychology instructor's entry into the coordination; (v) the monitoring of WhatsApp throughout the semester; (vi) implementation of a lecture on how to elaborate meeting minutes; (vii) elaboration of a report template for the teams; and (viii) involving student tutors with experience in working with projects instead of having instructors as tutors.

Challenges remain to enhance PBL utilization and, as consequence, students’ learning process. Five stand out: (i) increase interaction with the course of Calculus I; (ii) the expansion of the project coordination team; (iii) a more effective peer assessment; (iv) evolving the model of soft skills, and (iv) improving data collection and analysis tools.

In summary, the acceptance of the use of PBL was great in these four years such that it became visible to the whole school community. Because of this, from 2015, three other programs have adopted PBL in some subjects of their classes. It is noteworthy that the program of Chemical Engineering has created a design class, along the lines of what has been implemented in Industrial Engineering, from the second semester of 2015.

• How to cite this article: Pereira, M. A. C., Barreto, M. A. M., & Pazeti, M. (2017). Application of Project-Based Learning in the first year of an Industrial Engineering Program: lessons learned and challenges. Production, 27(spe), e20162238. DOI http://dx.doi.org/10.1590/0103-6513.223816

References

• Adderley, K., Ashwin, C., Bradbury, P., Freeman, J., Goodlad, J., & Greene, D., Jenkins, J. R. & Uren, O. (1975). Project methods in Higher Education. Guilford: Society for Research into Higher Education.
• Alves, A. C., Sousa, R. M., Fernandes, S., Cardoso, E., Carvalho, M. A., Figueiredo, J., & Pereira, R. M. S. (2016). Teacher’s experiences in PBL: implications for practice. European Journal of Engineering Education, 41(2), 123-141. http://dx.doi.org/10.1080/03043797.2015.1023782
  » http://dx.doi.org/10.1080/03043797.2015.1023782
• Bardin, L. (2001). L’analyse de contenu (10th ed.). Paris: Presses Universitaires de France.
• Checkland, P. (1991). From framework through experience to learning: the essential nature of action research. In H. E. Nissen, H. K. Klein & R. Hirschheim (Eds.), Information systems research: contemporary approaches and emergent traditions. Amsterdam: Elsevier North-Holland, Inc.
• Costa, L. R. J., Honkala, M., & Lehtovuori, A. (2007). Applying the problem-based learning approach to teach elementary circuit analysis. IEEE Transactions on Education, 50(1), 41-48. http://dx.doi.org/10.1109/TE.2006.886455
  » http://dx.doi.org/10.1109/TE.2006.886455
• Coughlan, P., & Coghlan, D. (2002). Action research for operations management. International Journal of Operations & Production Management, 22(2), 220-240. http://dx.doi.org/10.1108/01443570210417515
  » http://dx.doi.org/10.1108/01443570210417515
• Crawley, E. F., Brodeur, D. R., & Soderholm, D. H. (2008). The education of future aeronautical engineers: Conceiving, designing, implementing and operating. Journal of Science Education and Technology, 17(2), 138-151. http://dx.doi.org/10.1007/s10956-008-9088-4
  » http://dx.doi.org/10.1007/s10956-008-9088-4
• Crouch, C. H., & Mazur, E. (2001). Peer Instruction: ten years of experience and results. American Journal of Physics, 69(9), 970-977. http://dx.doi.org/10.1119/1.1374249
  » http://dx.doi.org/10.1119/1.1374249
• Denzin, N. K., & Lincoln, Y. S. (Eds.) (2011). The SAGE Handbook of qualitative research (4th ed.). Thousand Oaks: SAGE.
• Duch, G., Duch, B. J., & Allen, D. E. (2001). The power of problem-based learning: a practical “how to” for teaching undergraduate courses in any discipline. Sterling: Stylus Publishing.
• Felder, R. M. (2006). Teaching Engineering in the 21st Century with a 12th-century Teaching Model: How Bright is That? Chemical Engineering Education, 40(2), 110-113.
• Fernandes, S., Flores, M. A., & Lima, R. M. (2012). Students’ views of assessment in project-led engineering education: findings from a case study in Portugal. Assessment & Evaluation in Higher Education, 37(2), 163-178. http://dx.doi.org/10.1080/02602938.2010.515015
  » http://dx.doi.org/10.1080/02602938.2010.515015
• Flick, U. (2005). Qualitative methods in scientific research (2nd ed.). Lisboa: Monitor.
• Frank, M., Lavy, I., & Elata, D. (2003). Implementing the project-based learning approach in an academic engineering course. International Journal of Technology and Design Education, 13(3), 273-288. http://dx.doi.org/10.1023/A:1026192113732
  » http://dx.doi.org/10.1023/A:1026192113732
• Helle, L., Tynjälä, P., & Olkinuora, E. (2006). Project-based learning in post-secondary education: theory, practice and rubber sling shots. Higher Education, 51(2), 287-314. http://dx.doi.org/10.1007/s10734-004-6386-5
  » http://dx.doi.org/10.1007/s10734-004-6386-5
• Jollands, M., Jolly, L., & Molyneaux, T. (2012). Project-based learning as a contributing factor to graduates’ work readiness. European Journal of Engineering Education, 37(2), 143-154. http://dx.doi.org/10.1080/03043797.2012.665848
  » http://dx.doi.org/10.1080/03043797.2012.665848
• Kilgore, D., Atman, C. J., Yasuhara, K., Barker, T. J., & Morozov, A. (2007). Considering context : a study of first-year. The Journal of Engineering Education, 96(4), 321-334. http://dx.doi.org/10.1002/j.2168-9830.2007.tb00942.x
  » http://dx.doi.org/10.1002/j.2168-9830.2007.tb00942.x
• Lewin, K. (1946). Action research and minority problems. The Journal of Social Issues, 2(4), 34-46. http://dx.doi.org/10.1111/j.1540-4560.1946.tb02295.x
  » http://dx.doi.org/10.1111/j.1540-4560.1946.tb02295.x
• Lima, R. M., Carvalho, D., Flores, M. A., & Van Hattum-Janssen, N. (2007). A case study on project led education in engineering: students’ and teachers’ perceptions. European Journal of Engineering Education, 32(3), 337-347. http://dx.doi.org/10.1080/03043790701278599
  » http://dx.doi.org/10.1080/03043790701278599
• Lima, R. M., Silva, J. M., Van Hattum-Janssen, N., Monteiro, S. B. S., & Souza, J. C. F. (2012). Project-based learning course design: a service design approach. International Journal of Services and Operations Management, 11(3), 293-313. http://dx.doi.org/10.1504/IJSOM.2012.045660
  » http://dx.doi.org/10.1504/IJSOM.2012.045660
• Litzinger, T. A., Lattuca, L. R., Hadgraft, R., & Newstetter, W. (2011). Engineering education and the development of expertise. The Journal of Engineering Education, 100(1), 123-150. http://dx.doi.org/10.1002/j.2168-9830.2011.tb00006.x
  » http://dx.doi.org/10.1002/j.2168-9830.2011.tb00006.x
• Lu, S.-Y. (2007). A scientific foundation on collaborative engineering. International Academy for Production Engineering, 56(2), 605-634.
• Mello, C. H. P., Turrioni, J. B., Xavier, A. F., & Campos, D. F. (2012). Pesquisa-ação na engenharia de produção: proposta de estruturação para sua condução. Produção, 22(1), 1-13. http://dx.doi.org/10.1590/S0103-65132011005000056
  » http://dx.doi.org/10.1590/S0103-65132011005000056
• Miller, K., Schell, J., Ho, A., Lukoff, B., & Mazur, E. (2015). Response switching and self-efficacy in Peer Instruction classrooms. Physical Review Special Topics - Physics Education Research, 11(1), 1-8. http://dx.doi.org/10.1103/PhysRevSTPER.11.010104
  » http://dx.doi.org/10.1103/PhysRevSTPER.11.010104
• Mills, J. E., & Treagust, D. F. (2003). Engineering Education: is Problem-Based or Project-Based Learning the answer? Australasian Journal of Engineering Education, 3(2), 2-16. Retrieved in 03 October 2016, from https://www.researchgate.net/publication/246069451_Engineering_Education_Is_Problem-Based_or_Project-Based_Learning_the_Answer
  » https://www.researchgate.net/publication/246069451_Engineering_Education_Is_Problem-Based_or_Project-Based_Learning_the_Answer
• Montero, E., & Gonzalez, M. J. (2009). Student engagement in a structured problem-based approach to learning: a first-year electronic engineering study module on heat transfer. IEEE Transactions on Education, 52(2), 214-221. http://dx.doi.org/10.1109/TE.2008.924219
  » http://dx.doi.org/10.1109/TE.2008.924219
• Palmer, S., & Hall, W. (2011). An evaluation of a Project-Based Learning initiative in Engineering Education. European Journal of Engineering Education, 36(4), 357-365. http://dx.doi.org/10.1080/03043797.2011.593095
  » http://dx.doi.org/10.1080/03043797.2011.593095
• Pereira, M. A. C. (2016). Projeto Integrado de Engenharia de Produção I (LOQ4236): guia do projeto – turma 2016. Lorena: Universidade de São Paulo. Retrieved in 03 October 2016, from http://www.marco.eng.br/piep-I/Guia-Projeto-2016.pdf
  » http://www.marco.eng.br/piep-I/Guia-Projeto-2016.pdf
• Powell, P., & Weenk, W. (2003). Project-Led Engineering Education. Ultrecht: Lemma Publisher.
• Prince, M. J., & Felder, R. M. (2006). Inductive teaching and learning methods: definitions, comparisons, and research bases. The Journal of Engineering Education, 95(2), 123-138. http://dx.doi.org/10.1002/j.2168-9830.2006.tb00884.x
  » http://dx.doi.org/10.1002/j.2168-9830.2006.tb00884.x
• Svensson, T., & Gunnarsson, S. (2012). A design-build-test course in electronics based on the CDIO framework for engineering education. International Journal of Electrical Engineering Education, 49(4), 349-364. http://dx.doi.org/10.7227/IJEEE.49.4.1
  » http://dx.doi.org/10.7227/IJEEE.49.4.1
• Thomas, J. (2000). A review of research on project-based learning. San Rafael: Autodesk Foundation. Retrieved in 03 October 2016, from http://www.ydae.purdue.edu/LCT/HBCU/documents/AReviewofResearchofProject-BasedLearning.pdf
  » http://www.ydae.purdue.edu/LCT/HBCU/documents/AReviewofResearchofProject-BasedLearning.pdf
• United Nations Educational, Scientific and Cultural Organization. (2010). Engineering: issues challenges and opportunities for development. London: UNESCO. Retrieved in 03 October 2016, from http://unesdoc.unesco.org/images/0018/001897/189753e.pdf
  » http://unesdoc.unesco.org/images/0018/001897/189753e.pdf
• van Hattum-Janssen, N., & Mesquita, D. (2011). Teacher perception of professional skills in a project-led enginerring semester. European Journal of Engineering Education, 36(5), 461-472. http://dx.doi.org/10.1080/03043797.2011.606501
  » http://dx.doi.org/10.1080/03043797.2011.606501
• Webb, E. J. (1966). Unobtrusive measures: nonreactive research in the Social Sciences. Madison: Rand McNally.
• Westbrook, R. (1995). Action research: a new paradigm for research in production and operations management. International Journal of Operations & Production Management, 15(2), 6-21. http://dx.doi.org/10.1108/01443579510104466
  » http://dx.doi.org/10.1108/01443579510104466

Publication Dates

• Publication in this collection
  2017

History

• Received
  03 Oct 2016
• Accepted
  10 May 2017