New Meta-analysis of 217 Retrieval Practice Studies
By Yana Weinstein
I have to hand it to Google Scholar – sometimes, their algorithms are spot on. This week, I was alerted to a brand new meta-analysis on retrieval practice (1) – just published on February 1st. This meta-analysis is by far the most comprehensive on this topic to date. It even includes 3 of our studies: two co-authored by Megan (2), (3), and one by me (4). As such, I think the meta-analysis will be of interest to many of our readers. In this blog post, I pull out the main conclusions, but the full meta-analysis is also freely available for download here.
Overall effect
Unsurprisingly, the news is good: retrieval practice was consistently found to be better than restudy (the most typical control condition), and much better when the control condition involved either no activity or an unrelated filler activity.
Type of test
Megan previously wrote about her series of experiments which showed that type of test does not make a very big difference to the effectiveness of retrieval practice. The meta-analysis suggests, however, that hybrid practice tests (e.g., ones that include both multiple-choice and cued recall questions) are most effective. Indeed, this echoes the trend that Megan reported in her study.
However, across the four experiments in Megan’s paper (5), most of the data suggest there isn’t a benefit of hybrid tests. In fact, Experiments 1 through 3 show no benefit of hybrid formats, and Experiment 4 shows a trend favoring hybrid. However, the authors did not include this paper in the meta-analysis, possibly narrowly missing it in 2014.
The meta-analysis also reports a modest impact of matching the practice and final test formats (matching was slightly more effective). In addition, significant benefits of retrieval practice relative to restudy were found not only for tests of retention, but also for tests of transfer.
Study setting and age of students
Hooray! Retrieval practice works just as well in “real” classrooms as it does in the lab (we already knew this from our own research and have described classroom research on retrieval practice in previous posts, but it’s nice to see it confirmed in a meta-analysis). Also, for those wondering whether retrieval practice works for school-aged children, or only college-age adults: the highest effects were for secondary school children.
Additional considerations
- Benefits of retrieval practice relative to restudy were stronger when time between study and test was longer than 1 day.
- The conclusion regarding feedback may come as a surprise: retrieval practice with feedback was only slightly more effective than retrieval practice without feedback.
- For those worried about publication bias of positive results, the authors conducted two fail-safe tests and deemed this area of research to be “resistant to the file drawer problem” (but you may need to take these fail-safe tests with agrain of salt).
References:
(1) Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017). Rethinking the Use of Tests: A Meta-Analysis of Practice Testing. Review of Educational Research, Online First.
(2) Karpicke, J. D., & Smith, M. A. (2012). Separate mnemonic effects of retrieval practice and elaborative encoding. Journal of Memory and Language, 67, 17–29.
(3) Smith, M. A., Roediger, H. L., III, & Karpicke, J. D. (2013). Covert retrieval practice benefits retention as much as overt retrieval practice. Journal of Experimental Psychology: Learning, Memory and Cognition, 39, 1712–1725.
(4) Weinstein, Y., McDermott, K. B., & Roediger, H. L. (2010). A comparison of study strategies for passages: Rereading, answering questions, and generating questions. Journal of Experimental Psychology: Applied, 16, 308–316.
(5) Smith, M. A., & Karpicke, J. D. (2014). Retrieval practice with short-answer, multiple-choice, and hybrid formats. Memory, 22, 784-802.
