By Megan Sumeracki & Yana Weinstein

This piece originally appeared in the Spring 2018 issue of Impact by The Chartered College of Teaching. You can read more about Impact here. Note the formatting of references and British English spelling was retained.

Retrieval practice, or reconstructing knowledge by bringing it to mind from your memory, has been shown by numerous researchers to improve student learning (see Roediger et al., 2011). Saying that retrieval practice promotes learning in the classroom is all well and good, what does this actually mean for teachers who want to implement retrieval practice in their classrooms? Teachers might wonder how successful students need to be for retrieval to promote learning. How difficult should retrieval opportunities be? Does the format of retrieval practice matter? How should I time the questions within a lesson? Some of these things may matter, while others may not. In this article, we briefly describe retrieval practice as a learning strategy and then we review research addressing these questions to help teachers find the best ways to utilise retrieval practice in their classrooms.

Practising retrieval improves learning compared to rereading information (Roediger and Karpicke, 2006) - a strategy that many college students report using (for example, Hartwig and Dunlosky, 2012). Retrieval practice can also improve learning compared to other study strategies thought to be beneficial, such as creating a concept map while reading (Karpicke and Blunt, 2011). Promoting retrieval practice in the classroom can simply involve giving students frequent tests or quizzes. In fact, the retrieval practice phenomenon was called the testing effect for much of the last century, but now is more commonly called retrieval practice because one can promote retrieval with activities other than tests or quizzes (for example, Karpicke, Blunt et al., 2014). Importantly, retrieval practice can help with both fact-based learning and meaningful learning and transfer (Butler, 2010; Carpenter, 2012; Jensen et al., 2014; Smith et al., 2016). Thus, retrieval practice is of significant value in educational settings, and research in live classrooms confirms that utilising retrieval practice improves student learning in multiple contexts. For example, a retrieval practice benefit has been shown for adult learners in college classrooms (Mayer et al., 2009) as well as for primary school classrooms with middle-school students (McDermott et al., 2014), among others.

How successful do students need to be during retrieval? 

To gain the most benefit, students do need to successfully retrieve a certain amount of the information during retrieval practice. Imagine if a student just stared at a blank sheet of paper and could not remember anything about what they had just read; this would be unlikely to produce learning. Some research has shown that guessing even when you do not know the answer can lead to improvements in learning (Kornell et al., 2009). However, it would not be good for students to always guess and never really know. 

Research we conducted with students aged nine to 10 years demonstrates the need for some success during retrieval (Karpicke, Blunt et al., 2014). We had children follow along as a research assistant read them a text that was selected from the school curriculum, so that it was at the appropriate level for the students. However, when the students were given a blank piece of paper to recall what they could remember, they were not able to recall very much. On average, the students only wrote down 9% of the information, with some children not being able to write anything. Unsurprisingly, the children performed very poorly on an assessment test four days after this learning activity. They remembered more in the control condition, during which the research assistant repeatedly read the text to them, which is the opposite pattern to that typically found in retrieval practice experiments. 

We realised that these younger students needed guides in order to retrieve the information. In another experiment, we gave the children question maps with the topic of the text in the centre, and then a few prompts around the centre to guide recall. For example, one text was about clouds and one of the prompts was ‘describe cumulus clouds (shape and colour)’. When students were first able to complete the map with the text in front of them, they were then much more successful at retrieving the information when they had to fill out the map without the text. This led to greater performance on a later assessment compared to the repeated reading control group. 

However, this is not just about children and adult learners. If university students are not very successful during retrieval, they will not benefit as much either (Kang et al., 2007). Regardless of the student’s age, some amount of success is necessary. 

How difficult should the retrieval opportunities be? 

While we want students to be able to retrieve some information, we also do not want retrieval to be too easy. A teacher could lead a student to successfully reproduce the information from a text by only presenting the information one tiny chunk at a time and then asking the student to retrieve just that little chunk of information. Repeating little chunks like this will likely lead to a high amount of information being produced, but it is still unlikely to produce durable learning. Students need to think back to a prior time when they learned information to reconstruct this information (Karpicke, Lehman et al., 2014), and some amount of difficulty is ideal during this process. The important thing is to balance retrieval difficulty and success. You do not want the retrieval to be so difficult that students fail to retrieve anything at all, but you do not want it to be so easy that they do not really have to think back and reproduce the information. Teachers will likely need to monitor difficulty and success, and adjust retrieval activities accordingly. For example, if students are very easily and quickly retrieving information and are retrieving almost everything, teachers might consider making the activity more difficult. If retrieval success is very low, such as below 50%, then providing prompts to help the students retrieve will likely be helpful. Teachers can also provide scaffolding to help their students achieve success initially, and then slowly make retrieval more difficult as the students become more comfortable with the material. Doing this has the added benefit of ensuring repeated retrieval, and continuing to retrieve information multiple times over a period of time is very beneficial to learning (Kapler et al., 2015). 

Does the format of retrieval practice matter? 

Two of the most extensively researched retrieval-practice formats are short-answer and multiple-choice questions. This is likely to be because these two types of questions are common for tests and quizzes in educational settings. But does it matter whether multiple-choice or short-answer quizzes are used? 

There are obvious practical benefits to multiple-choice questions over short-answer questions. Multiple-choice questions are easier to administer and mark. This may mean for many teachers that multiple-choice quizzes could be given more frequently, since they are less time-consuming, and subsequently may mean increased retrieval opportunities for students in the classroom. Multiple-choice questions are also more likely to yield success during retrieval practice. However, short-answer questions are more difficult to answer, and the effort involved may lead to greater learning benefits (see Kang et al., 2007). Teachers can also create a hybrid format on the computer, where students first answer a question in a short-answer format and then advance to answer the same question posed as a multiple choice. This hybrid format combines the potential benefits of both, but does require each student to have access to a computer. 

Thankfully, the exact format of the question may not much matter for learning. In a series of experiments, we (Smith and Karpicke, 2014) investigated the differences between short-answer, multiple-choice and hybrid formats. Students read a series of texts about different topics (for example, Venice) and then answered questions. The questions were either short-answer, multiple-choice or the hybrid format mentioned earlier. One group did not practise retrieval. All students were then given a sentence that contained the correct answer to each question as feedback. One week later, the students returned and took a short-answer test to assess learning. The results showed that, across all of the experiments, all of the retrieval practice formats increased learning compared to not practising retrieval. On the other hand, there were only negligible differences in later performance between different retrieval practice formats. More detail on this study and the results can be found at: http://www. learningscientists.org/blog/2016/3/18-1. 

Similarly, other researchers have found little to no difference between short-answer and multiple-choice quiz formats when feedback is given (for example, McDermott et al., 2014). Short-answer questions are more difficult, but multiple-choice questions lead to more success, and feedback is not always enough to make up for the success differences between these two. When multiple-choice questions are created to require students to actually think back about the answer rather than just relying on which answer looks familiar to them (see Little et al., 2012), multiple-choice questions can be just as effective at producing learning as short-answer questions. 

How should I time the questions within a lesson? 

Most of the research on implementing quizzing in the classroom with positive results has focused on quizzes that are given after the lecture (for example, Lyle and Crawford, 2011). However, one might rightfully wonder whether it is best to give these end-of-class quizzes, or might it be more effective to intersperse quizzes throughout a class? Recent research suggests that interspersing quiz questions throughout learning can reduce the interference that happens when we are trying to learn a lot of similar pieces of information in a row, such as long lists of words (Szpunar et al., 2008; Weinstein et al., 2014) or many face–name pairs (Weinstein et al., 2011). This research has also been applied to more educationally relevant information, such as video lectures (Szpunar et al., 2013). The results suggest that interspersing quiz questions throughout learning can help with learning information presented later on in the class in comparison to not quizzing, because the quiz questions help relieve some of the interference that typically builds up during a longer learning session. 

However, these studies did not test retrieval after a delay; they simply demonstrated that immediate recall on the information presented later during learning was better after previous information was quizzed than when previous information was not quizzed. Looking at long-term learning, in a set of three studies in the lab, online and in the classroom, we (Weinstein et al., 2016) did not find any differences between conditions when students were quizzed either throughout class or at the end of class. Importantly, students who were not quizzed at all did more poorly on the long-term learning tests. So, the take-home message is that it doesn’t much matter where you put quiz questions, as long as you do give students retrieval practice opportunities in as many classes as possible. 

So, how do we use retrieval practice in the classroom? 

Research has demonstrated time and again that reconstructing knowledge by bringing information to mind improves meaningful learning – but how can teachers best leverage retrieval in the classroom? Some variables are important to consider, while others do not largely affect learning benefits from retrieval. Teachers need to ensure that students can be reasonably successful when practising retrieval, without making the task so easy that students no longer need to think back and bring the information to mind from memory. However, the format of retrieval and the timing of the questions within a lecture do not much matter. The important thing is to promote frequent retrieval practice in their classrooms. 

References

Butler AC (2010) Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, and Cognition 36: 1118–1133. 

Carpenter SK (2012) Testing enhances the transfer of learning. Current Directions in Psychological Science 21: 279–83. 

Hartwig MK and Dunlosky J (2012) Study strategies of college students: Are self-testing and scheduling related to achievement? Psychonomic Bulletin & Review 19: 126–134. 

Jensen JL, McDaniel MA, Woodard SM et al. (2014) Teaching to the test… or testing to teach: Exams requiring higher order thinking skills encourage greater conceptual understanding. Educational Psychology Review 26: 307–329. 

Kang SHK, McDermott KB and Roediger HL (2007) Test format and corrective feedback modulate the effect of testing on memory retention. The European Journal of Cognitive Psychology 19: 528–558. 

Kapler IV, Weston T and Wiseheart M (2015) Spacing in a simulated undergraduate classroom: Long-term benefits for factual and higher-level learning. Learning and Instruction 36: 38–45. 

Karpicke JD, Blunt JR, Smith MA et al. (2014) Retrieval-based learning: The need for guided retrieval in elementary school children. Journal of Applied Research in Memory and Cognition 3: 198–206. 

Karpicke JD and Blunt JR (2011) Retrieval practice produces more learning than elaborative studying with concept mapping. Science 331: 772–775. 

Karpicke JD, Lehman M and Aue WR (2014) Retrieval-based learning: An episodic context account. In: Ross BH (ed) Psychology of Learning and Motivation, Vol 61. San Diego, CA: Elsevier Academic Press, pp. 237–284. 

Kornell N, Hays J and Bjork RA (2009) Unsuccessful retrieval attempts enhance subsequent learning. Journal of Experimental Psychology: Learning, Memory, and Cognition 35: 989–998. 

Little JL, Bjork EL, Bjork RA et al. (2012) Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological Science 23: 1337–1344. 

Lyle KB and Crawford NA (2011) Retrieving essential material at the end of lectures improves performance on statistics exams. Teaching of Psychology 38(2): 94–97. 

Mayer RE, Stull A, DeLeeuw K et al. (2009) Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology 34: 51–57. 

McDermott KB, Agarwal PK, D’Antonio L et al. (2014) Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. Journal of Experimental Psychology: Applied 20: 3–21. 

Roediger HL and Karpicke JD (2006) Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science 17: 249–255. 

Roediger HL, Putnam AL and Smith MA (2011) Ten benefits of testing and their applications to educational practice. In: Mestre J and Ross B (eds) Psychology of Learning and Motivation: Cognition in Education. Oxford: Elsevier, pp. 1–36. 

Smith MA, Blunt JR, Whiffen JW et al. (2016) Does providing prompts during retrieval practice improve learning? Applied Cognitive Psychology 30: 544–553. 

Smith MA and Karpicke JD (2014) Retrieval practice with short-answer, multiple-choice, and hybrid formats. Memory 22: 784–802. 

Szpunar KK, Khan NY and Schacter DL (2013) Interpolated memory tests reduce mind wandering and improve learning of online lectures. Proceedings of the National Academy of Sciences 110: 6313–6317. 

Szpunar KK, McDermott KB and Roediger III HL (2008) Testing during study insulates against the buildup of proactive interference. Journal of Experimental Psychology: Learning, Memory, and Cognition 34: 1392–1399. 

Weinstein Y, Gilmore AW, Szpunar KK et al. (2014) The role of test expectancy in the build-up of proactive interference in long-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition 40: 1039–1048. 

Weinstein Y, McDermott KB and Szpunar KK (2011) Testing protects against proactive interference in face–name learning. Psychonomic Bulletin & Review 18: 518–523. 

Weinstein Y, Nunes LD and Karpicke JD (2016) On the placement of practice questions during study. Journal of Experimental Psychology: Applied 22: 72-84.