By: Yana Weinstein

This post is the second in a series where we delve into the nuances involved in retrieval practice. The first can be found here: I’m a Teacher Who Loves Quizzing: But Does Quiz Format Matter?

When I first started teaching at UMass Lowell 3 years ago, I didn’t know much about teaching. One thing I did know, though, was that my lectures were going to involve a lot of quizzing. I started off with two sections of Cognitive Psychology, which I taught in a fairly traditional lecture style, except that I always included a demonstration of an experiment in each lecture. I then faced a choice: should I put the quiz questions at the end of the lecture, or throughout? From what I knew of the literature, the choice wasn’t obvious.

On the one hand, most of the research on implementing quizzing in the classroom with positive results had focused on quizzes that were given after the lecture (1). Anecdotally, I also heard from colleagues in my field that this is what they did in their own classes.

On the other hand, I had recently done a study on face-name learning where quizzes were interspersed throughout learning (2). The results of this research strongly suggest that interspersing quizzes throughout learning helps not only learning of the face-name pairs that are tested, but also the learning of NEW face-name pairs that come later in the same learning session! One of the reasons for this could be that participants who are tested consistently throughout learning come to expect to be tested (3), and so pay closer attention to the material.

But at the same time, having students do the quiz at the end of the lecture could be more beneficial to later learning, because it would require more effortful retrieval, which helps learning (4). You can see how it was hard for me to pick the ideal place to quiz! I immediately saw the opportunity to turn this into an empirical question (5). 

The following semester, I picked 8 lectures and varied whether quiz questions were presented throughout the lecture or at the end of the lecture. Because I had two sections of the same class, I was able to counterbalance question placement so that if on any given day students in one section answered quiz questions throughout, the other sections had them at the end.

Here is what my PowerPoint slide sorter looked like when quiz questions (red slides) were interspersed, compared with at the end:

The quiz questions were simple, short-answer questions that could be answered with one sentence and fairly easily if students had been paying attention. To complete the quizzes, students simply wrote down the answers and turned them in at the end of class. As expected, students did better on quiz questions that were interspersed than those that appeared at the end. This was expected and unsurprising, because in the case of interspersed questions, the information had just been presented – so of course, it would still be fresh in the students’ minds. What was more interesting to me was to see how this tested information would be maintained in memory over a longer period of time.

A few weeks after the last of the 8 experimental lectures, I surprised my students with a midterm. What?? How did they not all get up and leave in protest, you ask? Well, I offered them extra credit for every single question they got correct. I did it this way so that I could measure the effect of quiz question placement without the “noise” of cramming that would inevitably happen if I had announced the midterm in advance. 

What I found was that weeks later, despite the initial advantage of interspersing, there was no longer any difference in performance between interspersed questions, and those answered at the end of a lecture. There are two ways to look at this: in one sense, performance during the lecture is better in the interspersed condition compared to the at-the-end condition; and then no worse than in the other condition on the final assessment. So in terms of overall lecture performance, interspersing wins. But another way to look at it is in terms of forgetting: with performance on the interspersed questions higher at first than on the at-the-end questions, but equal in the two conditions on the later test, this means that students forgot less after answering quiz questions at the end of the lecture.

What did you think the result were going to be? You could have predicted that the advantage of interspersing would carry over to the later test; it didn’t. You could alternatively have predicted that the extra effort involved in producing information at the end of a lecture would translate into an advantage on the mid-term, but it didn’t. Ultimately, those two advantages could be pulling in the opposite direction, resulting in no different on the mid-term.

When making pedagogical decisions, though, final exam performance is not the only important factor. When trying to decide where to put quiz questions, I find this table useful: 

So if you’re interested in keeping students engaged and raising initial performance, intersperse your questions; but if you care about the forgetting rate, and maybe want something that’s easier for you to implement, then just stick some questions at the end of the lecture. But if you’re only interested in long-term retention, it seems that the exact placement of quiz questions within a lecture doesn’t much matter.

References:

(1) Lyle, K. B., & Crawford, N. A. (2011). Retrieving essential material at the end of lectures improves performance on statistics exams. Teaching of Psychology, 38(2), 94-97.

(2) Weinstein, Y., McDermott, K. B., & Szpunar, K. K. (2011). Testing protects against proactive interference in face–name learning. Psychonomic Bulletin & Review, 18(3), 518-523.

(3) Weinstein, Y., Gilmore, A. W., Szpunar, K. K., & McDermott, K. B. (2014). The role of test expectancy in the build-up of proactive interference in long-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(4), 1039-1048.

(4) Pyc, M. A., & Rawson, K. A. (2009). Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory? Journal of Memory and Language, 60(4), 437-447.

(5) Weinstein, Y., Nunes, L. D., & Karpicke, J. D. (2016). On the placement of practice questions during study. Journal of Experimental Psychology: Applied, 22, 72-84.