By Cindy Nebel

Testing sometimes gets a bad reputation. This is perhaps unsurprising in the world of standardized testing, but it has led to some misconceptions that we come across frequently. For example, many people believe that retrieval practice is used to memorize facts only, a point that we have written about previously. Others confuse formative testing (testing for learning) with summative testing (testing for assessment) and believe that tests are only good for demonstrating memorization skills. Finally, there is an argument as displayed in the below tweet that we should do away with assessments that require students to retrieve knowledge because it is only application that matters.

There are several studies that have demonstrated the utility of retrieval practice for transfer or application of factual knowledge to new situations (1,2).  Today, I want to tell you the details about a new study that used retrieval practice to specifically enhance deductive reasoning (3). The researchers gave students 10 pieces of factual information and then half of the participants re-studied the facts while the other half were required to fill in missing words for each fact after which they received the correct answer. Once this practice session ended, half of the participants were asked to try to recall as many of the facts as they could. Finally, all participants took a multiple choice deductive reasoning test, where they had to use multiple facts to answer any one question. Here’s an example from the study:

Fact 1: If there are 10 or more people at a party, then Loners will not be in attendance.
Fact 2: Loners always wear a purple shirt, and no one else ever does.

Deductive Reasoning Final Test Question:
If there are 12 people at a party, might one of them be wearing a purple shirt?
a.       Yes
b.       No
c.       Maybe

Recall that these two facts are presented among 8 other facts that are not useful in answering this particular multiple choice question and these facts were presented in random order, making it a much more difficult task.

The researchers found that participants who practiced retrieval (via fill in the blank) performed a little bit worse than those who restudied both on the free recall test AND on the deductive reasoning test.

This might be because the fill in the blank test wasn’t actually a great way to have students retrieve the information. They focused in on tiny details instead of seeing how these items might fit together or even what the facts really meant. If students weren’t able to retrieve the factual information that they needed, they were pretty hopeless to complete the deductive reasoning task.

In a second study, they changed the methods such that participants either received the same fill in the blank questions described above or cued-recall questions where they were asked to recall all the information they could about, say, Loners (in the example above). All participants then received the free recall test and the deductive reasoning test, but unfortunately they found the same results. Participants performed equally on the free recall test, indicating that the cued recall test didn’t help them to remember the important information for deductive reasoning.

In a final effort, the researchers made a few methodological changes. Half of the participants had 5 minutes to restudy or take the cued recall test as many times as they wanted. Unsurprisingly, students were able to restudy an average of 14 times in 5 minutes, while they only took about 2 cued recall tests. The other half of the participants got to restudy or do the cued recall test exactly four times. The other big change was that half of the participants took the final tests immediately after restudy/cued recall while the other half took the tests two days later.

Here’s where the results start to look really cool. When they controlled for the number of times participants were exposed to the material (just four times whether restudying or doing cued recall), they still found no effect of the immediate test, but two days later there was a 19% benefit for students who had used cued recall instead of restudy. That’s the equivalent of two letter grades on an exam!

What does all this mean for you? Well, first of all, we’ve talked about how not all testing is created equal and certain types of retrieval practice will NOT result in big benefits (see this blog and this blog). While retrieval practice IS extremely robust, you do need to put a little thought into how you’re implementing it in your classroom in order to see the effects you want and students need to be encouraged to study using retrieval practice effectively.

But the really good news is that practicing retrieval of factual information DOES improve deductive reasoning. In order to apply information, you need to be able to retrieve it. Even though some folks might not like testing for factual information, that factual information DOES matter for students’ ability to use it later on in life. In this study we see that testing for facts helped students to apply information, but not right away. What seems to be happening is that the cued recall test helped to insulate students from forgetting the material. Those who had only restudied were unable to remember as many facts two days later. So, if students are studying for our tests (which probably include factual questions as well as application questions) earlier than the day of, they are best to practice retrieval. If they are cramming, practicing retrieval won’t hurt them… and they are more likely to remember that information later on (like on the final, in the next course in the sequence, or *gasp* later in life)!

 References:

(1) Smith, M. A., Blunt, J. R., Whiffen, J. W. & Karpicke, J. D. (2016). Does providing prompts during retrieval practice improve learning? Applied Cognitive Psychology, 30,544-553.

(2) Butler, A. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, & Cognition, 36, 1118-1133.

(3) Wissman, K.T., Zamary, A., & Rawson, K.A. (2018). When does practice testing promote transfer on deductive reasoning tasks? Journal of Applied Research in Memory and Cognition, 7, 398-411.