By: Cindy Wooldridge

We have covered the many benefits of retrieval practice, but we have often heard criticism that the evidence for retrieval practice often comes from randomized controlled trials in the laboratory using materials that are too simplistic. While there is evidence that retrieval practice works in the classroom (1), the effects are mixed (2). So, today we are providing new evidence – a series of studies using online quizzing in a large introductory course to show when quizzing boosts retention… and when it doesn’t.

Those of us who have taught introductory courses know the unique challenges involved in teaching a large class with a diverse population. It is difficult to tailor a course to meet the unique needs of students that vary in background and motivation levels and to keep the grading load manageable. It is for these reasons that Trumbo, Leiting, McDaniel, and Hodge (3) studied quizzing in large introductory courses at the University of New Mexico. In a series of experiments just published last month, they examined when and how quizzing can be implemented in an efficient way in college classrooms.

Here are the details of the methods and results, or you can skip to the bottom to read the conclusions for education.

Experiment 1

In the first experiment, students in two sections (722 students total) were given access to online quizzes outside of their normal lecture-based class. In one section the quizzes were optional, but in the second section, they were worth 45% of the course grade (required quizzes). The researchers were particularly interested in how the quizzes would help students on the exams in the course. Not surprisingly, students in the required quiz section took more quizzes and those students who took more quizzes tended to have higher exam scores. Overall the required quiz section performed 13.5% better on the exams on average than those in the optional quiz section, showing that taking the quizzes improved retention of the information.

Unfortunately, there were a few limitations in the first experiment. The quiz questions were identical to the exam questions and, because students were given the correct answers after submitting the quizzes, it is not surprising that those students who saw the answers to the test questions did better on the test. The instructor also could have been more encouraging in the required section vs. the optional section due to bias and students who were required to take quizzes may have been using other successful strategies as well.

Experiment 2

In Experiment 2, Trumbo and colleagues addressed some of the issues in Experiment 1 by having all of the students take quizzes, but varying the content that they were quizzed on. In a given unit of the class, the material was divided into two sections. Students were then randomly assigned to receive quizzes on Section A or Section B. The quizzes were required for all students and made up 33% of their course grade. The researchers also addressed the problem of identical questions by addressing the same concepts in the exam question but changing either the wording of the question or using different examples. Here is one example of the quiz to exam question changes:

There were no differences between students who received Section A questions vs. Section B questions on the number of quizzes they took or class performance, but students performed 16.2% higher on exam questions that were related to quizzed material.

There was a limitation in Experiment 2 as well. Because students were only quizzed on half of the material, they “studied” that material more than the other material in the course. Trumbo and colleagues. addressed this issue specifically in Experiment 3.

Experiment 3

This experiment looked almost identical to Experiment 2, except that instead of just being quizzed on one section of material, students also studied statements about the other half of the material. For example, half of the students were quizzed over Section A material and read statements about Section B. Here is an example of a statement related to the questions above:

There were again no differences on the number of quizzes students took or their class performance, but students performed 8.4% higher on quizzed vs. “studied” (statements) material.

Conclusion

The big take-away from these experiments is that students are often not aware of the best learning strategies. In fact, many students may only be interested in the minimum effort required to achieve the highest score possible (an admittedly efficient approach!). When students are unaware of the boosts in exam performance (learning) that come from retrieval practice, they are unlikely to engage in this material without some kind of extrinsic reward. If you are interested in implementing retrieval practice in your classes, here are some tips:

• To make grading manageable, use online quizzes outside of class
• Make sure you include immediate feedback
• Require quizzes by making them part of the course grade

And see our other blogs on retrieval practice for more suggestions!

• Learn How to Study Using Retrieval Practice
• I’m A Teacher Who Loves Quizzing: But Where Should The Quiz Questions Go?
• I’m A Teacher Who Loves Quizzing: But Does Quiz Format Matter?
• How Can We Help Our Students Love Quizzing? (Part 1)
• How Can We Help Our Students Love Quizzing? (Part 2)

References:

(1) McDaniel, M. A., Wildman, K. M., & Anderson, J. L. (2012). Using quizzes to enhance summative-assessment performance in a web-based class: An experimental study. Journal of Applied Research in Memory & Cognition, 1, 18–26.

(2) Bell, M. C., Simone, P. M., & Whitfield, L. C. (2015). Failure of online quizzing to improve performance in introductory psychology courses. Scholarship of Teaching and Learning in Psychology, 1, 163–171.

(3) Trumbo, M. C., Leiting, K. A., McDaniel, M. A., & Hodge, G. K. (2016). Effects of reinforcement on test-enhanced learning in a large, diverse introductory college psychology course. Journal of Experimental Psychology: Applied, 22, 148-160.