By Tim van der Zee

(cover image Angelqiu122, CC BY-SA 4.0, via Wikimedia Commons

In 2015 Tim started as a PhD candidate at ICLON. Earlier he obtained a Masters in Human Learning and Performance psychology (with honors) from the Erasmus University Rotterdam. He studies how people learn from educational videos in open online education, such as MOOCs (Massive Open Online Courses), and focuses specifically on increasing the instructional design quality and educational value of these videos. His position is funded by CEL (Centre of Education and Learning) in the Netherlands, a collaboration between the universities of Leiden, Delft, and Rotterdam. In addition to his research on improving the educational quality of online videos, he is a vocal advocate of pre-registration, open science, and other efforts aimed at improving the quality of the scientific literature. You can follow him on Twitter at @Research_Tim and read his blog at www.timvanderzee.com.

In recent years, open online education has become very popular.  With a reliable Internet connection comes free access to a large variety of Massive Open Online Courses (MOOCs) found on platforms such as Coursera and edX. Millions of people from all over the world are enrolling in free courses from top universities around the world. The promise – global and free access to high quality education – has often been applauded (1).

The typical MOOC is a series of educational videos, often interconnected with other study materials such as texts, and regularly followed by quizzes. Usually these MOOCs are divided into approximately 5 to 8 weeks of content. In the picture below you see an example of Week 1 from the course ‘Improving your statistical inferences’ by Daniel Lakens.

MOOC participants really do come from all over the world (2), however the majority of these courses are developed in Western countries and are predominantly provided in English. What does this mean for non-native English speakers? Can they benefit from these courses as much as native speakers? If not, how can we reduce potential language barriers? This is the topic of a study I recently ran.

Availability versus accessibility

While online courses are available to anyone with a stable internet connection, they are not equally accessible to everyone. Language barriers are an important contributor to accessibility issues, as MOOCs are predominantly offered in English while most MOOC participants are not native English speakers. In our study (3), we had 125 non-native English speakers watch multiple educational videos in English. After each video, the students received a quiz for which they could score up to 10 points. By measuring the English language proficiency of the students, we were able to estimate the impact of having a low(er) level of English proficiency on quiz performance. For every scale point different on the 6-point language proficiency test, students scored on average 0.55 points lower on the quizzes. That means that if the most proficient students get a good grade (7 out of 10), other students are likely to score much lower (4 out of 10) due to their English proficiency.

To sub or not to sub

The Web Content Accessibility Guidelines (http://www.w3.org/WAI/WCAG20/glance/) prescribe that every audio medium should be accompanied by visual information, such as subtitles, to increase the level of accessibility. For educational videos (which are often the core content of online courses) this seems to make a lot of sense, as subtitles will help anyone to better understand the content. Right? Maybe not! Whether subtitles have a positive, or possibly even a detrimental effect on learning is a contested issue; While there is convincing evidence that subtitles are helpful when watching a language learning video (4), they appear to hinder learning from content videos where the goal is not to learn a language but to learn about a specific topic (5).

To test the effects of subtitles specifically for MOOC videos, half of the videos in this study had English subtitles enabled, while the other half did not. However, students did not score better when subtitles were enabled. Surprisingly, even the students with a low level English proficiency did not benefit from the subtitles. So, how can we improve how much students learn from videos?

Instructional design quality

We need to be very careful about how we design educational materials like videos, as they should be consistent with human learning mechanisms. For example, it is important to not to provide too much visual or textual information at the same time (6), because we can only process a limited amount of information simultaneously (7). If it is important content, you are better off segmenting it by slowly increasing the amount of information that is shown on the screen (8). In addition, it pays off to use visual cues such as arrows or color coding to guide students’ attention – this might free valuable limited resources which are required for learning (9). Another factor that contributes to (unnecessary) visual complexity is the physical distance between related information. For example, if you are showing a diagram with various labels, it is easier for students to process and understand it if the labels are in or near the points of interests. In the two images below you can see an example of an overly complex design, followed by a much simpler design.

By comparing videos that were identical in content but differed in instructional design, we noticed that videos with a simpler (or less taxing) instructional design appeared to be better for the students. That is, the students learned more from these videos even though the content was the same. In addition, all students learned more from these videos, regardless of their English level proficiency.

Conclusions

To summarize, the way videos are designed and especially the language ability of the student both affect how much students learn from educational videos. In contrast, our study found that providing subtitles in the same language as the video neither helped nor damaged students’ ability to learn from the videos. However, this does not lead to the conclusion that English subtitles should not be made available, as they are vital for students with hearing disabilities. Furthermore, students might prefer watching videos with subtitles for other reasons, even though this might not directly affect their learning of the content. The extent to which subtitles in the students’ native language might help them overcome lack of English proficiency to learn the content offered in MOOCs remains to be investigated. Another option would be to provide dubbed versions of MOOC videos to cater to more languages, but this is a costly intervention.

Want to read more about research on MOOCs? Read Tim's earlier blog post.

References:

(1) Daniel, J. (2012). Making sense of MOOCs: Musings in a maze of myth, paradox and possibility. Journal of Interactive Media in Education, 2012(3).

(2) Nesterko, S. O., Dotsenko, S., Han, Q., Seaton, D., Reich, J., Chuang, I., & Ho, A. (2013, December). Evaluating the geographic data in MOOCs. In Neural information processing systems.

(3) van der Zee, T., Admiraal, W., Paas, F., Saab, N., & Giesbers, B. (2017). Effects of subtitles, complexity, and language proficiency on learning from online education videos. Journal of Media Psychology.

(4) Perez, M. M., Noortgate, W. V. D., & Desmet, P.  (2013). Captioned video for l2 listening and vocabulary learning:  A meta-analysis. System, 41, 720–739.

(5) Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371.

(6) Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of EducationalPsychology, 93, 187–198.

(7) Baddeley, A. (2003). Working memory: looking back and looking forward. Nature Reviews Neuroscience, 4, 829–839.

(8) Mayer, R. E., & Chandler, P. (2001). When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? Journal of Educational Psychology, 93, 390–397.

(9) Boucheix, J.-M., & Lowe, R. K. (2010). An eye tracking comparison of external pointing cues and internal continuous cues in learning with complex animations. Learning and Instruction, 20, 123–135.