The impact of Duolingo in developing students’ linguistic competence: an aspect of communicative language competences

O impacto do Duolingo no desenvolvimento da competência linguística dos alunos: um aspecto das competências linguísticas comunicativas

Abstract

This paper reports the impact of a type of m-learning on developing students’ linguistic and communicative competence in English as a second language. This study is a Mexican-based case using Duolingo to improve virtual courses in higher education students whose first language is Spanish. This study is a quasi-experimental research that analyzed pre-test and post-test results on a sample of 40 participants (control group of 20 participants and experimental group of 20 participants). The measuring instruments were designed for this specific study and focused on testing the six components of linguistic competence according to CEFR at level A2 (2018). A pilottest was previously conducted, and pre-test (0.81) and post-test (0.98) coefficients were considered acceptable according to Cronbach’s Alpha. This study addressed the following questions: a) How does Duolingo contribute to the development of linguistic competence in higher education students and support English learning in virtual courses via the m-learning methodology? and b) Which components were improved in student´s linguistic competence after using Duolingo? Results revealed a profound improvement in the experimental group participants. The post-test result of the experimental group is superior ( M=43.75) compared with the control group (M=30.35), mainly in the phonological control (+5.20) and the orthographic control (+2.65) compared with the control group, whose score slightly increased primarily attributed to the English virtual classes.

Keywords
Duolingo; M-learning; Higher education; Linguistic competence; Second language acquisition; Communicative competences

Resumo

Este documento relata o impacto de um tipo de m-learning no desenvolvimento da competência linguística e comunicativa dos estudantes em inglês como segunda língua. Este estudo é um caso mexicano que usa Duolingo para melhorar cursos virtuais voltados para estudantes do ensino superior cuja língua materna é o espanhol. Trata-se de uma pesquisa quase-experimental que analisou resultados pré e pós-testes em uma amostra de 40 participantes (grupo controle de 20 participantes e grupo experimental de 20 participantes). Os instrumentos de medição foram projetados para este estudo específico e se concentraram em testar os seis componentes da competência linguística de acordo com o CEFR no nível A2 (2018). Um teste piloto foi conduzido anteriormente e os coeficientes de pré-teste (0,81) e pós-teste (0,98) foram considerados aceitáveis de acordo com o Alfa de Cronbach. Este estudo abordou as seguintes questões: a) Como o Duolingo contribui para o desenvolvimento da competência linguística de estudantes do ensino superior e apoia a aprendizagem do inglês em cursos virtuais através da metodologia m-learning?; e b) Quais componentes foram melhorados na competência linguística do estudante após o uso do Duolingo? Os resultados revelaram uma profunda melhoria nos participantes do grupo experimental. O resultado pós-teste do grupo experimental foi superior (M=43,75) em comparação ao grupo controle (M=30,35), principalmente no controle fonológico (+5,20) e no controle ortográfico (+2,65), cuja pontuação aumentou ligeiramente, o que se atribui principalmente às suas aulas virtuais de inglês.

Palavras-chave
Duolingo ; M-learning ; Ensino superior ; Competência linguística ; Aquisição de segunda língua ; Competências comunicativas

Introduction

English has become the most useful foreign language for communication worldwide ( HOLFESTER, 2019). One of students’ main goals is to develop communicative language competences and participate effectively and efficiently in various contexts by using this international language ( HSUAN, 2019).

However, second language acquisition is a complex process since users need to internalize a new language system ( HALL, 2011). Learning a second language, such as English, is sometimes more difficult than acquiring their first language since many variables can affect the learning process. The target language and their first language have several differences, “younger learners and adults have neurological, cognitive, and psychological differences that come into play in second language acquisition. Children are usually considered to be better learners than adults” ( BENATI, 2014, p. 183). English can be taught with face-to-face classes, virtual classes, or mixed learning ( TOMLINSON; WHITTAKER, 2013). Technology can help students’ learning process ( PLATT; RAILE; YU, 2014). To develop communicative competences and enhance communication in class, teachers must implement diverse principles and methodologies ( AZAR; NASIRI, 2014; CHO et al., 2018 ) due to students’ limited exposure and artificial situations ( KNAPP; SEIDLHOFER, 2009).

The teacher’s role is essential to promote students’ learning process and build a communicative environment in classes ( GLOMO, 2013; PLATT et al., 2014 ; TOMLINSON; WHITTAKER, 2013). Virtual courses are as complex and authentic as face-to-face classes ( KENT; SIMPSON, 2010). Students can have the same face-to-face communication and interaction by using technology ( AL SAMARRAIE, 2019; SHYAM, 2012). Technological innovations such as MALL (mobile-assisted language learning) can adjust to students’ needs and personalize students’ learning with their mobile learning styles ( KUKULSKA, 2009; PARK; YANG; LEE, 2011).

Literature review

For our research, we provided an overview of linguistic competence, one of the four main aspects of communicative language competences (CEFR, 2018); an approximation on m-learning as an innovative support to second language acquisition; a synthesis of current studies in the field of m-learning and mobile assisted-language learning applications to promote English language learning; and a description of Duolingo.

Communicative and linguistic competences

The concept of communicative competences started with the linguistic theory of Noam Chomsky (1965, p. 3), who introduced “an ideal speaker-listener, in a completely homogeneous speech community, who knows its language perfectly and is unaffected”. He also contrasted “competence (the speaker-hearer’s knowledge of his language) and performance (the actual use of language in concrete situations)” ( CHOMSKY,1965, p. 4). Hymes (1972) integrated a sociolinguistic vision to Chomsky’s grammatical competence concept. To Hymes (1971, p. 16), communicative competence is related “to speaking as a whole” and comprises grammatical, pragmatic, and sociolinguistic competence.

Communicative competence is essential for language learning due to users needing to participate in sociolinguistic discussions, specific sociocultural interactions, and develop a social, psychological, and physical life ( FRIEDRICH, 2012). Tahir (2018) recognized that linguistic competence focuses mainly on the idea of a speaker and a listener applying language knowledge in actual performance, which is a relevant aspect in language learning and teaching due to being one of the main goals in ELT (English language teaching).

The Common European Framework of Reference for Languages: learning, teaching, assessment (CEFR) grouped language proficiency into three broad categories: basic users (A1 & A2), independent users (B1 & B2), and proficient users (C1 & C2). The CEFR also exposes that communicative language competences involve linguistics, psychology, and sociopolitical approaches. They are organized into four main aspects: a) strategic competence, b) linguistic competence, pragmatic competence, and d) sociocultural competence. Linguistic competence considers six main components: a) general linguistic range, b) vocabulary range, c) grammatical accuracy, d) vocabulary control, e) phonological control, and f) orthographic control, which are not separate ‘components’ and cannot be isolated from each other in a real context ( COE, 2018).

M-learning as an innovative tool

M-learning (mobile learning) is a learning methodology that provides extensive possibilities for learning and practicing the target language by using electronic devices (ZOU; LI, 2015) since it allows a learner to move anytime, anywhere ( CHO et al., 2018 ). M-learning is an alternative approach to access to learning resources on mobile devices which changed how languages are taught and learned in formal education, virtual, or face-to-face classes since mobile devices promote portability, connectivity, mobility in users’ language learning process and a flexible learning mode ( KUMAR; WOTTO; BÉLANGER, 2018; MUNDAY, 2016; PEDRO; BARBOSA; SANTOS, 2018). Its three main requirements are mobility of technology, time and space, and learning, which promote learning in different contexts (LAI; ZHENG, 2018; MUYINDA, 2007). Thereby, m-learning can complement e-learning, mixed learning, autonomous learning, or face-to-face classes ( KUMAR; WOTTO; BÉLANGER, 2018; RASSKAZOVA et al., 2017 ).

The possibility of frequently using devices has promoted m-learning and language mobile learning. A lot of research has focused on this field, which offers numerous opportunities and promotes an alternative approach to language learning ( GOMES; LOPES; ARAÚJO, 2016). M-learning is immersed in existing learning theories such as behaviorist, constructivist, situated, collaborative, informal, and lifelong learning theories ( MUYINDA, 2007).

Some advantages and disadvantages of using mobile apps can modify learners’ experiences, with advantages such as: a) mobility ( CHO et al., 2018 ), b) ease of using apps ( GONZÁLEZ; MEDINA, 2018), c) immediate feedback and self-testing ( GAFNI; ACHITUV; RACHMANI, 2017), d) continuous access to information ( DUKIC; CHIU; LO, 2015) and e) high exposure to the language ( YANG; ZHOU; JU, 2013). On the other hand, m-learning main limitations are a) need of internet access to look up information, b) the limited downloads to use some apps ( GAFNI; ACHITUV; RACHMANI, 2017) and c) the devices’ size ( KUMAR; WOTTO; BÉLANGER, 2018).

Studies in learning English with M-learning

The use of mobile technologies has recently received significant attention in second language acquisition. Many studies have shown that m-learning provides a potential possibility of learning a foreign language to students ( BÁRCENA et al., 2015 ). A current meta-analysis study explored how useful mobile devices are in language learning by a random effects model. Results showed a moderate positive overall effect of mobile apps on language acquisition, such as improving language learning skills ( CHO et al., 2018 ).

Gafni, Achituv, and Rachmani (2017) analyzed learners’attitudes toward using the Duolingo app and simultaneously took their foreign language course. Findings showed how the MALL Duolingo app enhanced students’ learning process and how MALL apps gamification can encourage autonomous and ubiquitous learning.

González and Medina (2018) examined students’ experiences in three Mexican universities with a qualitative study. Students could use the e-learning platform on their mobile devices everywhere, and their perceptions of m-learning were positive since it helped them achieve their learning objectives. Findings showed that the portability and accessibility of mobile devices promote students’ learning, and they valued these characteristics of m-learning.

Mospan (2018) examined mobile devices effectiveness in teaching and learning English from university students’ perspectives. Findings confirm that students’ motivation and understanding increased when they use digital technology in classes, and most believe that MALL and CALL (computer-assisted language learning) promote positive learning environments.

Another study focused specially on using Duolingo as a complement to traditional foreign language classes at a college. Results concluded that Duolingo is an easy-to-use app that can improve students’ language learning. Students found Duolingo activities enjoyable due to their gamification aspects ( MUNDAY, 2016).

No quantitative study shows the impact of using these language learning apps. Studies mainly focused on students’ perceptions on using mobile assisted-language learning apps to register how useful, practical, and beneficial they were for participants. Therefore, a potential study could assess students’ language acquisition and their improvement in linguistic competences and analyze this communicative language competence aspect according to the CEFR ( COE, 2018).

Duolingo as an innovative application for language acquisition

Duolingo, one of the most popular and downloaded apps in the market, is promoted as a MALL app that helps users acquire the language by using communicative activities ( CASTRO; HORA MACEDO; PINTO BASTOS, 2016). Users can improve their listening, reading, speaking, and writing skills by playing gamification activities. Activities are structured into units, and the user achieves a higher level upon completing each section. Each unit is semantically or grammatically themed (DUOLINGO, 2020). Table 1 shows the six main components of linguistic competence covered by Duolingo activities.

Table 1 –
Duolingo activities linked to the six main components of linguistic competence

Duolingo activity	Component of linguistic competence promoted in the activity
a) Activities to write vocabulary after seeing a picture of it.	Vocabulary control: the user practices a repertoire dealing with concrete everyday needs. Vocabulary range: the user checks his vocabulary to express basic communicative needs.
b) Activities to translate a phrase or a sentence into the user’s native language.	General linguistic range: the user checks brief everyday expressions to satisfy simple needs of a concrete type, e.g., personal details, daily routines, wants and needs, requests for information.
c) Activities to translate a phrase or a sentence into the target language.	General linguistic range: the user applies of short, memorized phrases which cover predictable survival situations. Grammatical accuracy: the user must use simple structures correctly. He can check word meaning while translating the sentence or phrase. Vocabulary range: the user employs vocabulary for simple survival needs.
d) Dictation activities for listening to sentences and phrases and then writing them in the target language.	Orthographic control: the user must write a phrase or a sentence with reasonable phonetic accuracy. He must recognize short words that are in his oral vocabulary.
e) Pronunciation activities for reading sentences and checking correct pronunciation using voice recognition software.	Phonological control: the user can check his pronunciation since the voice recognition software provides immediate feedback on aspects such as stress, rhythm, intonation, and intelligibility.
f) air-word activities to match vocabulary from their first language with the target language.	Vocabulary range: the user practices vocabulary to explore routine, everyday transactions involving familiar situations and topics. Vocabulary range: the user checks vocabulary to express basic communicative needs.
g) Activities requiring the unscrambling of words into meaningful sentences or phrases.	Vocabulary range: the user checks a vocabulary to conduct routine, everyday transactions involving familiar situations and topics.
h) Activities for practicing the correct ordering of sentences from three ones given in random order in the target language.	General linguistic range: the user employs phrases and sentences to deal with predictable, everyday situations. General linguistic range: the user utilizes phrases and sentences which enable him/her to deal with everyday situations with predictable content.

Source: Own elaboration based on Duolingo activities (DUOLINGO, 2020) and linguistic competence components and descriptors at level A2 (CEFR, 2018).

Also, teachers can use Duolingo School free to track their students’ progress and their interaction within the app. This app can give each student a personalized learning practice with continuous and immediate feedback. The Duolingo school promotes learning in and out of the classroom and allows teachers’ monitoring with a virtual classroom. Thus, Duolingo allows teachers to have their own virtual space with different classrooms and track each student´s progress to support their classes, which can be useful in classes (DUOLINGO, 2020). By using apps like these in formal education contexts, teachers gain the potential to support their students´ learning outside the classroom. Thereby, analyzing the effect of using m-learning in higher education students and the language skills developed with the help of apps such as Duolingo becomes relevant to promote pedagogical strategies.

Methodology

This section describes the stages of this study that implemented Duolingo as a support to virtual classes in formal education contexts. The use of m-learning methodologies allowed students to practice the target language using electronic devices, allowing progress to be monitored and analyzed. This study was based on a quantitative approach with a scientific base with a delimited and specific sample (BISQUERRA; ALZINA, 2004; VALENZUELA; FLORES, 2013).

Research questions

The literature discussed in the previous section shows a limited understanding of the development of linguistic competence in higher education. This study intends to build on the current literature by providing an understanding of m-learning impact the development of linguistic competence in the English language via the use of Duolingo. This study attempted to answer the following research questions:

1. How does Duolingo contribute to the development of linguistic competence in higher education students and support English learning in virtual courses via the m-learning methodology?

2. Which components were improved in student´ linguistic competence: a) general range, b) vocabulary range, c) grammatical accuracy, d) vocabulary control, e) phonological control, and f) orthographic control according to CEFR at level A2 after using Duolingo?

Participants and research background

The study was conducted at a university in Mexico involving 40 students divided into a control group (20 participants) and an experimental group (20 participants). Both groups were at the same level at the language center of the university and around level A2 (CEFR, 2018). The study relies on a non-probabilistic sample with two specific groups with a similar elementary level in English. They studied in different academic fields of higher education, such as a) Humanities and Social Science, b) Natural and Formal Sciences, and c) Professions and Applied Sciences. The predominant academic field (90%) was professions and applied sciences (Business, Management, Manufacturing, Construction Engineering, and Technology). Among 40 participants, 25 were females (65%) and 15 were males (45%), with 85% of them having digital competences and 95% of them reporting that their preferred device is a smartphone used 2-5 hours a day. Both groups took the pre-test at the beginning of the study and they attended their virtual classes during the pandemic on a Saturday course at the language center. The experimental groups additionally used Duolingo apps for seven weeks, and at the end of the language learning course, 40 participants took the post-test. The institute heads towards a constructivist approach promoting active learners who develop their general competences and a communicative approach aiming to develop communicative language competences.

Due to health restrictions taken against the COVID-19 pandemic in Mexico, face-to-face classes were suspended, and the challenge of acquiring a second language via virtual classes was the best possible solution by the school. However, this situation has limited English use to students who lack the same interaction and exposure to the language. Elementary students can interact in their synchronic classes without an English Conversation Club after this situation. Thereby, the most accessible and suitable device for this situation is a mobile phone, and apps designed for language learning can support language exposure. One of the potential apps to promote language learning is Duolingo since it includes grammatical and vocabulary activities from A1 to B2, and its free version does not limit students’ possibilities.

Instruments

This study utilized a questionnaire to collect participants’ demographic information, including age, gender, academic field of study, digital competences, and their interaction with the mobile app. Indeed, pre-and post-tests were used as measuring instruments designed for this specific study to collect quantitative data and evaluate the six components of linguistic competence. Each test was balanced, being divided into six dimensions according to the descriptors of the six linguistic competence components: a) general range, b) vocabulary range, c) grammatical accuracy, d) vocabulary control, e) phonological control, and f) orthographic control at level A2 of the CEFR. Both tests had a total of 48 items and each dimension, a total of 8 items, which were created considering the A2 level, and the 48 items were validated by an index of item-objective congruence.

The instruments were also validated with the technique of expert judgment by the categories of coherence and relevance by six experts in English language teaching who verified each item of the pre-and post-tests as 1) not relevant, 2) the item needs some revision, and 3) very relevant. After experts’ judgment of each item, the formula for the content validity ratio (CRV) was driven to select the acceptable items according to the Lawshe method, modified by Tristán (2008). The proportion of agreement among the experts was also calculated, and standard items were selected considering an agreement equal or greater than 0.58 for an acceptable item. On the other hand, complete tests were validated using the content validity index (CVI). The resulting CVI of the pre-test (0.97) and the post-test (0.98) showed them to be acceptable tests. After the content validity and face validity process, both instruments were built with 48 items into six dimensions for the pilot test. Tests were validated by Kuder-Richardson (KR-20) for internal consistency reliability with dichotomous items, a special formula for Cronbach’s alpha by a pilot-test conducted with 16 elementary students. The pre-test (0.81) and the post-test coefficients (0.98) are considered acceptable according to Cronbach’s alpha ( FRÍAS, 2019; LENKE, 1977).

Research procedure and method

This quasi-experimental study collected participants’ background via a survey. First, participants for the control and experimental groups answered a Google form survey. Secondly, both groups took the pre-test at the beginning of the study and started their English language course with Saturday virtual classes. The experimental group used the Duolingo app simultaneously to a seven-week course. At the end of the language learning course, both groups took the post- test, and results were examined with statistical analysis of data by the SPSS software. Results were also compared via statistical analysis and inferential statistics. Participants’ progress in the experimental group was followed by Duolingo school to track their advance and the app’s gains. The method used in this study is quasi-experiment with non-equivalent pre-test-post-test control group design ( KLOCKARS, 1992; VALENZUELA; FLORES, 2013):

Chart 1

Group	Pre-test	Treatment	Post-test
Experimental group	Y 1	X	Y 2
Control group	Y1	—	Y 2

Description:

Y ₁

= Pre-test (dependent variable or conditions before treatment).

X

= Using Duolingo (independent variable).

Y ₂

= Post-test (dependent variable or conditions after treatment).

Data analysis

The first stage of data analysis was the descriptive statistics of each group profile after collecting participants’ information: a) their background (gender, age, and academic field of study at the university), b) their digital competences, and c) their use of mobile devices to identify their profile and contact with technology. The second stage was a statistical analysis of pre-and post-tests-results by the SPSS software to obtain the centrality and dispersion of both groups in each test dimension: a) general range, b) vocabulary range, c) grammatical accuracy, d) vocabulary control, e) phonological control, and f) orthographic control. The third stage was inferential statistics by an independent two-sample t-test to assume that the dependent variable was normally distributed at the beginning of the study, and a paired t-test to verify whether the post-test score is significantly different in both groups due to the independent variable in the experimental group, carried out with the SPSS software. A virtual classroom was created by Duolingo school to track students’ progress and validate the interaction between participants and Duolingo.

Findings

This section analyzes and presents the pre- and post-tests results of the control and experimental groups. We validate the hypothesis of the study to evaluate the effect of Duolingo as a language teaching-learning support to complement language courses and promote the development of students’ linguistic competence in virtual classes.

Tests results

Descriptive statistics exhibit that most participants of both groups were female (65% in the control group and 60% in the experimental group) aged from 18 to 24 years in both groups. They belonged to three academic fields of study: a) Humanities and Social Sciences, b) Natural and Formal Sciences, and c) Professions and Applied Sciences. In both groups, the predominant academic field was Professions and Applied Sciences (85% in the control group and 95% in the experimental group) (see Table 2).

Table 2 –
Descriptive statistics of the control and experimental groups

Participants’ background		Control group N	Percentage	Experimental group N	Percentage
Gender	Male	7	35%	8	40%
	Female	13	65%	12	60%
Age	18	1	5%	5	25%
	19	3	15%	5	25%
	20	3	15%	2	10%
	21	3	15%	1	5%
	22	3	15%	1	5%
	23	3	15%	4	20%
	24	4	20%	2	10%
Academic fields of study	Humanities and social sciences	1	5%	1	5%
	Natural and Formal sciences	2	10%	0	0%
	Professions and applied sciences	17	85%	19	95%

Source: Research data.

The statistical analysis of pre-and post-tests results gave the measures of centrality and dispersion. All statistical tests were run by the SPSS software, version 26.0. Results show no significant difference between the control group ( M=26.40) and the experimental group ( M=24.50) on the pre-test. However, the experimental group results ( M=43.75) on the post-test are higher than those of the control group ( M=30.35). The experimental group had a significant increase in scores from the pre-test ( M=24.50) to the post-test ( M=43.75) with the independent variable (the use of Duolingo). Nevertheless, the control group, which only interacted with the language in their virtual language course, increased its score from the pre-test ( M=26.40) to the post-test ( M=30.35). Therefore, we observed a positive learning effect of the virtual language courses on students. Virtual classes are a feasible option to continue and promote student´s language learning ( AL SAMARRAIE, 2019) since they allow visual, auditory, and verbal interaction in real-time ( MARTINOVIC; PUGH; MAGLIARIO, 2010) and are similar to face-to-face classes ( KENT; SIMPSON, 2010). Dispersion of test results was lower in the post-test of the experimental group ( V=6.303, SD=2.511). Experimental group participants showed a similar level on the post-test, which was considerably different from the control group (see Table 3).

Table 3 –
Measures of centrality and dispersion of the experimental and control groups on the pre-test and post-test

Group	Test	Mean	Median	Variance	SD
Control group	Pre-test	26.40	30	92.253	9.605
	Post-test	30.35	28	28.766	5.363
Experimental group	Pre-test	24.50	25	40.684	6.378
	Post-test	43.75	44	6.303	2.511

Source: Research data.

In the second stage, we analyzed each dimension of the tests, and each group’s score results since they show some significant differences between the pre-and post-tests. First, the pre-test score is the same in both groups ( M=5.00). However, the post-test results are quite different since the experimental group score ( M=7.75) is higher than that of the control’s ( M=6.80). We found the highest dispersion in the pre-test of the control group ( V=3.789, SD=1.947) and the lowest in the post-test of the experimental group ( V=0.408, SD=0.639). The pre-test ( M=6.35) and post-test ( M=7.85) on the experimental group show a constant difference regarding the vocabulary range dimension. However, the control group got a related score on the pre-test ( M=6.05) and post-test ( M=6.95). The least consistent score was the pre-test on the control group ( V=2.682, SD=1.638), and the most homogeneous score was the post-test results on the experimental group ( V=0.134, SD=0.366). Concerning grammatical accuracy items, the pre-test of the control group ( M=5.25) was higher than the experimental group ( M=4.95), which was the same for the post-test of the control group ( M=4.95), with the post-test of the experimental group ( M=7.45) being considerably higher. Thereby, the experimental group’s performance after using the app was the highest and their score was the most consistent for all participants ( V=0.471, SD=0.686). In the vocabulary control dimension, the pre-test of both groups was the lowest on the control group ( M=2.00) and the experimental group ( M=1.40). The experimental group’s score ( M=6.05) was better on the post-test, increasing considerably more than for the control group, and participants’ results were quite similar ( V=0.892, SD=0.945). The pre-test ( M=3.15) and post-test ( M=3.45) of the control group show an insignificant difference regarding the phonological control dimension. However, the experimental group score differs considerably from the pre-test ( M=2.45) to the post-test ( M=7.65). Participants’ pre-test scores were heterogeneous both in the control ( V=4.661, SD=2.159) and experimental groups ( V=4.471, SD=2.114). Nonetheless, most experimental group participants got a more homogenous score ( V=0.239, SD=0.489). The last dimension of tests was orthographic control, with control ( M=4.95) and experimental ( M=4.35) groups showing similar scores on the pre-test. Participants of the experimental group answered quite similarly ( V=2.000, SD=1.414) and got a better score after using Duolingo on the post-test ( M=7.00) than the control group ( M=4.10), which got a lower score than on the pre-test, as shown in Table 4.

Table 4 –
Measures of Centrality and Dispersion of the experimental and control groups on the pre-test and post-test six dimensions

Group	Test	Dimension	Mean	Median	Variance	SD
Control group	Pre-test	Linguistic general range	5.00	5.50	3.789	1.947
		Vocabulary range	6.05	6.50	2.682	1.638
		Grammatical accuracy	5.25	6.00	3.039	1.743
		Vocabulary control	2.00	1.00	4.316	2.007
		Phonological control	3.15	3.00	4.661	2.159
		Orthographic control	4.95	6.00	8.050	2.837
	Post-test	Linguistic general range	6.80	7.00	1.642	1.281
		Vocabulary range	6.95	7.00	1.208	1.099
		Grammatical accuracy	4.95	5.00	2.155	1.468
		Vocabulary control	4.10	4.50	4.411	2.100
		Phonological control	3.45	3.00	3.945	1.986
		Orthographic control	4.10	2.00	8.411	2.900
Experimental group	Pre-test	Linguistic general range	5.00	5.00	2.632	1.622
		Vocabulary range	6.35	6.00	1.397	1.182
		Grammatical accuracy	4.95	5.00	2.682	1.638
		Vocabulary control	1.40	1.50	2.147	1.465
		Phonological control	2.45	2.00	4.471	2.114
		Orthographic control	4.35	4.00	8.555	2.925
	Post-test	Linguistic general range	7.75	8.00	0.408	0.639
		Vocabulary range	7.85	8.00	0.134	0.366
		Grammatical accuracy	7.45	8.00	0.471	0.686
		Vocabulary control	6.05	6.00	0.892	0.945
		Phonological control	7.65	8.00	0.239	0.489
		Orthographic control	7.00	8.00	2.000	1.414

Source: Research data.

To show any significant differences between the two groups and determine whether Duolingo activities impacted participants’ improvement in their linguistic competence, we conducted independent and paired sample t-tests. The independent samples t-test was conducted to investigate whether the means of the experimental and control groups’ scores in pre-and post-tests showed any significant difference. Table 3 shows that the control group mean ( M=26.40) was higher than the experimental group ( M=24.50). Table 5 shows that the significance level (sig=0.466) is not statistically different between both groups at the beginning of the study. Thereby, both groups started in equal conditions. The mean on the post-test of the experimental group ( M=43.75) is higher that of the control group ( M=30.35). The level of significance was calculated to be 0.00, indicating that the score of both control and experimental groups differs significantly, as shown in Table 5. Results could confirm the hypothesis of this study about the use of Duolingo by the experimental group:

Original Hypothesis: The use of Duolingo to support the language learning process in higher education students with an elementary level (A2 to CEFR) positively impacts their English linguistic competence.

Table 5 –
T-test for the equality of means in independent samples of the pre-test and the post-test

	T-test for Equality of Means
	t	df	Sig.(2-tailed)	Mean difference	Std. Error Difference	95% Confidence Interval of the Difference
						Lower	Upper
A2prem-learning Pre-test	0.737	38	0.466	1.900	2.578	−3.319	7.119
A2postm-learning Post-test	−10.120	26.944	0.000	−13.400	1.324	−16.117	−10.683

Note: Significant at <0.05. Source: Research data.

Table 6 shows the paired samples t-test used to compare the pre-and post-tests of each group. The difference between the control group pre-and post-tests is not significant since Sig. (0.133) is greater than 0.05. On the other hand, the difference between the pre-and the post-test of the experimental group is significant since Sig. (0.000) is less than 0.05. Students who were using Duolingo while taking their virtual language courses in formal education contexts got a better score than those who only took their language courses. Thus, we can say that using Duolingo to support virtual language classes had a significant effect on students’ linguistic competence, a communicative language competence aspect.

Table 6 –
Paired samples t-test comparing the pre-test and the post-test scores of the control and experimental groups

Paired differences
Groups	Tests	t	df	Sig. (2-tailed)
Control group	Pre-test – Post-test	−1.569	19	0.133
Experimental group	Pre-test – Post-test	−11.844	19	0.000

Note: Significant at <0.05. Source: Research data.

Use of Duolingo

Test results analysis indicated that the experimental group outperformed the control group. Furthermore, it proved that the students of the experimental group achieved significant progress on the post-test, compared with those of the control group, who only took their virtual language learning classes. Focusing on the experiment group’s interaction with Duolingo, the phonological control dimension showed the most significant improvement, with its average increasing by 4.20 from the pre-test ( M=3.45) to the post-test ( M=7.65). Considering the overall phonological control descriptor of A2 users about clear enough pronunciation ( COE, 2018, p. 136) Duolingo activities promoted better pronunciation for phrases in the target language. The app checked for correct, clear pronunciation via the use of voice recognition software (see Table 1), which provided users with immediate feedback, which is a key advantage of m-learning ( GAFNI; ACHITUV; RACHMANI, 2017).

Orthographic control also significantly increased from the pre-test ( X=4.1) to the post-test ( X=7.0), which could be attributed to the increasing difficulty level in some of the app activities as they progressed from the first to last section, e.g., dictation activities requiring listening to phrases followed by writing them in the target language (see Table 1). The user progresses from easy to difficult levels for spelling and phonetic accuracy as the orthographic control descriptors mentioned at level A2 ( COE, 2018). Note that the average score from the pre-test ( M=5.0) to the post-test ( M=7.75) in the general linguistic range dimension increased ( M=2.75) after using Duolingo. This component was promoted with activities to translate a phrase or a sentence into the user’s native language, activities to translate a phrase or a sentence into the target language, and activities to choose the correct sentences in the target language from a set of three sentences (see Table 1). Additionally, the experimental group participants could repetitively check and review those continuously, which is a m-learning advantage that is impossible in face-to-face or virtual classes ( YANG; ZHOU; JU, 2013) (see Figure 1).

Figure 1 –
Polar diagram of the results of pre-test and post-test of the experimental group

To verify participant interaction with the app, we used the Duolingo school. The maximum exposure of a participant was 40 days, and the minimum exposure was 20 days as shown in the virtual classroom. Each participant’s score ranging from 46/48 to 41/48 in the experimental group is more related to the achieved gains and completed units than their daily exposure. The daily use of app is not related to the student’s progress and gains since each user has personalized learning and they can move forward in the unit quickly or slowly as allowed by m-learning ( KUKULSKA, 2009; MUYINDA, 2007). This teaching section removes monitoring limits, allows tracking of student progress, and promotes personalized learning. Therefore, the Duolingo school promotes an inherent m-learning quality ( PEDRO; BARBOSA; SANTOS, 2018). However, considering students’ background and the possibility of accessing smartphones and the internet is essential before implementing free apps such as Duolingo so that they promote better language learning instead of leading to students’ frustration. Thereby, teachers must consider students’ devices and digital competencies to promote these mobile-assisted language learning apps out of class.

Conclusions

To answer the first research question, “How does Duolingo contribute to the development of linguistic competence in high education students and support English learning in virtual courses via the m-learning methodology?”, this study shows a significant difference between the pre-test and post-test for the experimental and control groups, attributed to the experimental group´s use of Duolingo and validated by independent samples and paired samples t-test. Based on the results reported above, participants who used Duolingo to study outperformed the control group, which was only exposed to the language in a virtual language course.

This study focused on analyzing the development of linguistic competence and the six components of this aspect of communicative language competences based on CFER (2018) descriptors. In this part, the second research question “Which components were improved in student´s linguistic competence: a) general range, b) vocabulary range, c) grammatical accuracy, d) vocabulary control, e) phonological control, and f) orthographic control according to CEFR at level A2 after using Duolingo?” is answered with the analysis of the experimental group registered above. As mentioned in the previous section, the experimental group showed considerable increases between pre-test and post-test scores in the phonological and orthographic control groups. Pronunciation and dictation activities promoted progress. The portability of mobile technologies allows students to access learning resources and mobile-assisted language learning apps that can support their language learning in language courses.

This study shows the advantage of using Duolingo as a learning resource. Comparing the experimental and control group results clearly illustrated the benefits of using this strategy. The experimental group score increased in the six linguistic competence components on the post-test, considerably more in the phonological control and the orthographic control, which can be attributed to Duolingo activities as shown in Table 1. According to their level, this app can support developing students’ linguistic competence without frustration and loss of motivation due to interacting with English at their elementary level. To teachers, this is a potential tool to promote and monitor students’ language learning. They can select specific units as activities for students to complete while using the unit content as a basis for planning their classes as they were selected in this study by using Duolingo school. In addition to this, Duolingo school allows teachers to track learners’ progress out of class by creating as many virtual classrooms as they need for free.

To conclude, this app is an innovative support that allows students to continue their language learning, promoting situated learning with an authentic context and culture with phrases and words used in specific scenarios. Furthermore, this mobile-assisted language learning app promotes students’ autonomous learning that a teacher monitors initially, and then the student can move forward in its gamification activities. Mobile-assisted language learning apps have several advantages, such as helping students become autonomous learners ( MUYINDA, 2007). Teachers, significant social agents who promote students’ language learning, have an essential role in using m-learning to remove the barriers of learning traditionally and implement it to support their language teaching for successful language learning ( YANG; ZHOU; JU, 2013).

Limitations

This study has several limitations. First, it used a sample by convenience due to limited time and the low number of participants. Participants’ English level was elementary (A2 according to the CEFR) and only their linguistic competence was analyzed. In future studies, the model should be applied to students with higher English levels, such as intermediate or advanced. It should also be applied to larger groups and for longer than seven weeks. Second, participants were all in higher education, and their ages ranged from 18 to 24 years. More diverse background variables are needed in future research, such as participants in middle school and high school or adult learners, to measure their language learning process and the development of linguistic competence using Duolingo during their language courses. Third, the study was limited to using Duolingo, out of the numerous mobile-assisted language learning apps. Investigating other apps that promote students’ linguistic competence is also suggested. Finally, this study mainly focused on linguistic competence and its six components. Longitudinal studies on the development of communicative language competences would be necessary to gain more insights into the m-learning field and mobile-assisted language learning apps.

References

• AL SAMARRAIE, Hosam. A scoping review of videoconferencing systems in higher education: learning paradigms, opportunities, and challenges. International Review of Research in Open and Distributed Learning, Athabasca, v. 20, n. 3, p. 121-140, 2019.
• AZAR, Ali Sorayyaei; NASIRI, Hassan. Learners’ attitudes toward the effectiveness of mobile assisted language learning (MALL) in L2 listening comprehension. Procedia Social and Behavioral Sciences, Oxford, v. 98, p. 11836-11843, 2014.
• BÁRCENA, Elena et al. State of the art of language learning design using mobile technology: sample apps and some critical reflection. In: EUROCALL CONFERENCE, 22., 2015, Padova. Proceedings […]. Voillans: Research-publishing.net, 2015. p. 36-43.
• BENATI, Alessandro. Second language acquisition. In: FÄCKE, Christiane (ed.). Manual of language acquisition. Berlin: De Gruyter, 2014. p. 179-197.
• BISQUERRA, Rafael. Metodología de la investigación educativa. v. 1. Madrid: La Muralla, 2004.
• CASTRO, Ana Paula de; HORA MACEDO, Suzana da; PINTO BASTOS, Hélvia Pereira. Duolingo: an experience in English teaching. Journal of Educational and Instructional Studies, Turkey, v. 6, n. 4, p. 59-64, 2016.
• CHO, Kyunghwa et al. The effects of using mobile devices on student achievement in language learning: A meta-analysis. Education Sciences, Basel, v. 8, n. 3, p. 13-15, 2018.
• CHOMSKY, Noam. Aspects of the theory of syntax. Cambridge: MIT Press, 1965. (Special technical report; v. 11).
• COE. Council of Europe. Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Companion Volume with new descriptors. Strasburg: Council of Europe, 2018.
• DUKIC, Zvjezdana; CHIU, Dickson KW; LO, Patrick. How useful are smartphones for learning? Perceptions and practices of Library and Information Science students from Hong Kong and Japan. Library Hi Tech, Bingley, v. 33, n. 4, p. 545-561, 2015.
• DUOLINGO INC. Duolingo. Teaching English as a Second or Foreign Language, Berkeley, v. 19, n. 1, p. 1-9, 2015.Available at:http://www.tesl-ej.org/pdf/ej73/m1.pdf Access in: 17 jun. 2022.
  » http://www.tesl-ej.org/pdf/ej73/m1.pdf
• FRÍAS NAVARRO, Dolores. Apuntes de consistencia interna de las puntuaciones de un instrumento de medida. Valencia: University of Valencia, 2019. Available at: https://www.uv.es/~friasnav/AlfaCronbach.pdf Access in: 29 jun. 2022
  » https://www.uv.es/~friasnav/AlfaCronbach.pdf
• FRIEDRICH, Patricia. Communicative competence. New York: Routledge, 2012.
• GAFNI, Ruti; ACHITUV, Dafni Biran; RACHMANI, Gila. Learning foreign languages using mobile applications. Journal of Information Technology Education, Santa Rosa, v. 16, n. 1, p. 301-317, 2017.
• GLOMO NARZOLES, Danebeth Tristeza. Classroom communication climate and communicative linguistic competence of EFL learners. Theory and Practice in Language Studies, London, v. 3, n. 3, p. 404-410, 2013.
• GOMES, Nelson; LOPES, Sérgio; ARAÚJO, Sílvia. Mobile learning: a powerful tool for ubiquitous language learning. In: PAREJA-LORA, Antonio; CALLE-MARTÍNEZ, Cristina; RODRÍGUEZ-ARAGON, Pilar (ed.). New perspectives on teaching and working with languages in the digital era. Dublin: Research-publishing, 2016. p. 189-200.
• GONZÁLEZ, Rosa María Isasi; DEL CARMEN MEDINA, Gladys. Uso de dispositivos móviles como herramientas para aprender TT. Píxel-Bit, Sevilla, n. 52, p. 217-227, 2018.Available at:https://recyt.fecyt.es/index.php/pixel/article/view/62533. Access in: 17 jun. 2022.
  » https://recyt.fecyt.es/index.php/pixel/article/view/62533.
• HALL, Graham. Exploring English language teaching. Oxon: Routledge, 2011.
• HOLFESTER, Chris. Teaching English as a second language: English as a foreign language. New York: Salem Press Encyclopedia, 2019.
• HSUAN YAU, Lai. University student perceptions of learning English as an international language. The International Journal of Learning in Higher Education, [S. l.], v. 26, n. 2, p. 57-72, 2019.
• HYMES, Dell. Competence and performance in linguistic theory. In: HUXLEY, Renira; INGRAM, Elisabeth (ed.). Language acquisition: models and methods. New York: Academic Press, 1971. p. 3-28.
• HYMES, Dell. On communicative competence. In: PRIDE, John B.; HOLMES, Janet (ed.). Sociolinguistics: selected readings. London: Penguin Books, 1972. p. 269-293.
• KENT, Andrea M.; SIMPSON, Jennifer L. Interactive videoconferencing: connecting theory to practice for preservice teachers. Journal of Digital Learning in Teacher Education, Waynesville, v. 27, n. 1, p. 12-21, 2010.
• KLOCKARS, Alan J. Evaluating pretest-posttest control group experiments. Perceptual and Motor Skills, Thousand Oaks, v. 75, n. 1, p. 96-98, 1992.
• KNAPP, Karlfried; SEIDLHOFER, Barbara; WIDDOWSON, Henry George (ed.). Handbook of foreign language communication and learning. Berlin: De Gruyter, 2009.
• KUKULSKA-HULME, Agnes. Will mobile learning change language learning? ReCall, Cambridge, v. 21, n. 2, p. 157-165, 2009.
• KUMAR BASAK, Sujit; WOTTO, Marguerite; BELANGER, Paul. E-learning, M-learning and D-learning: conceptual definition and comparative analysis. E-Learning and Digital Media, Thousand Oaks, v. 5, n. 4, p. 191-216, 2018.
• LAI, Chun; YEUNG, Yuk; HU, Jingjing. University student and teacher perceptions of teacher roles in promoting autonomous language learning with technology outside the classroom. Computer Assisted Language Learning, Cambridge, v. 29, n. 4, p. 703-723, 2016.
• LENKE, Joanne M. et al. Differences between kuder-richardson formula 20 and formula 21 reliability coefficients for short tests with different item variabilities. In: ANNUAL MEETING OF THE AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, 61., 1977, New York. Anais… New York: ERIC, 1977.Available at: https://files.eric.ed.gov/fulltext/ED141411.pdf. Access in: 29 jun. 2022.
  » https://files.eric.ed.gov/fulltext/ED141411.pdf
• MARTINOVIC, Dragana; MAGLIARO, Jelena; PUGH, Timothy. Pedagogy for mobile ICT learning using videoconferencing technology. Interdisciplinary Journal of Information, Knowledge, and Management, [S. l.], v. 5, p. 375-394, 2010. Available at: https://www.questia.com/library/journal/1G1-247880129/pedagogy-for-mobile-ict-learning-using-video-conferencing.Access in: 17 jun. 2022.
  » https://www.questia.com/library/journal/1G1-247880129/pedagogy-for-mobile-ict-learning-using-video-conferencing.
• MOSPAN, Natalia. Mobile teaching and learning English: a multinational perspective. Teaching English with Technology, [S. l.], v. 18, n. 4, p. 105-125, 2018.Available at: http://cejsh.icm.edu.pl/cejsh/element/bwmeta1.element.desklight-263522eb-dbc5-4ee6-b561-519e55b8ca57.Access in: 17 jun. 2022.
  » http://cejsh.icm.edu.pl/cejsh/element/bwmeta1.element.desklight-263522eb-dbc5-4ee6-b561-519e55b8ca57
• MUNDAY, Pilar. The case for using Duolingo as part of the language classroom experience. Ried, Madrid, v. 19, n. 1, p. 83-101, 2016.
• MUYINDA, Paul B. MLearning: pedagogical, technical and organizational hypes and realities. Campus-Wide Information Systems, Bingley, v. 24, n. 2, p. 97-104, 2007.
• PARK, Seongwon; KIM, Kwangeak; LEE, Bong Gyou. Future information technology. In: INTERNATIONAL CONFERENCE ON FUTURE INFORMATION TECHNOLOGY, 6., 2011, Crete. Proceedings... Crete: FutureTech2011, 2011. p. 62-71.
• PEDRO, Luís Francisco Mendes Gabriel; BARBOSA, Cláudia Marina Mónica de Oliveira; SANTOS, Carlos Manuel das Neves. A critical review of mobile learning integration in formal educational contexts. International Journal of Educational Technology in Higher Education, [S. l.], v. 15, n. 1, p. 10, 2018.
• PLATT, Carrie Anne; AMBER, N. W. Raile; YU, Nan. Virtually the same?: Student perceptions of the equivalence of online classes to face-to-face classes. Merlot Journal of Online Learning and Teaching, New York, v. 10, n. 3, p. 489-503, 2014.
• RASSKAZOVA, Tatiana et al. Challenges of teaching a blended English course to university students. In: THE INTERNATIONAL SCIENTIFIC CONFERENCE ELEARNING AND SOFTWARE FOR EDUCATION, 13., 2017, Bucharest. Proceedings… Bucharest: Else, 2017. p. 303-308.
• RICKHEIT, Gert; STROHNER, Hans (ed.). Handbook of communication competence. Berlin: De Gruyter, 2008.
• SHYAM, Narayanan. Virtual classrooms as an alternative platform. Singapore: Centre for Instructional Technology National University of Singapore, 2012.
• TAHIR, Ismail. Teachers’ beliefs in balancing linguistic competence and teaching performance in EFL classrooms. Brain, Iasi, v. 9, n. 1, p. 50-58, 2018.
• TOMLINSON, Brian; WHITTAKER, Claire. Blended learning in English language teaching: course design and implementation. London: British Council, 2013.
• TRISTÁN-LÓPEZ, Agustín. Modificación al modelo de Lawshe para el dictamen cuantitativo de la validez de contenido de un instrumento objetivo. Avances en Medición, Bogotá, v. 6, n. 1, p. 37-48, 2008. Available at:https://www.humanas.unal.edu.co/lab_psicometria/application/files/9716/0463/3548/VOL_6._Articulo4_Indice_de_validez_de_contenido_37-48.pdf.Access in: 29 jun. 2022.
  » https://www.humanas.unal.edu.co/lab_psicometria/application/files/9716/0463/3548/VOL_6._Articulo4_Indice_de_validez_de_contenido_37-48.pdf
• VALENZUELA GONZÁLEZ, Jaime Ricardo; FLORES FAHARA, Manuel. Fundamentos de investigación educativa. Mexico, DC: Digital Tecnológico de Monterrey, 2013.
• YANG, Bowen; ZHOU, Shijun; JU, Weijie. Learning English speaking through mobile-based role-plays: the exploration of a mobile English language learning app called Engage. The EuroCall Review, Valencia, v. 21, n. 2, p. 27, 2013.

Edited by

Editor:

Prof. Dr. Rogério de Almeida

Iris Cristina Peláez-Sánchez

is a PhD student in Educational Innovation at Instituto Tecnológico y de Estudios Superiores de Monterrey. She has a master’s degree in Educational Technology at Instituto Tecnológico y de Estudios Superiores de Monterrey (ITESM) and bachelor’s degree in Pedagogy at Universidad Nacional Autónoma de México (UNAM). Her line of research is application of new technology in language teaching and digital gap.

Anabel Velásquez-Durán

is a researcher at the Instituto de Investigaciones en Educación de la Universidad Veracruzana and postdoctoral researcher at the Centro de Investigaciones y Estudios Superiores en Antropología Social (CIESAS-Golfo). General Director of the Medición Independiente de Aprendizajes - MIA Program (CIESAS and Universidad Veracruzana), and member of the PAL Network. Her lines of research revolve around technology-mediated learning.

Publication Dates

• Publication in this collection
  28 Apr 2023
• Date of issue
  2023

History

• Received
  22 May 2021
• Reviewed
  23 Nov 2021
• Accepted
  21 Mar 2022