The Negative Testing Effect
By Cindy Nebel
One way to quickly spot educational fads is when the seller argues that something will “always work” and that’s it’s “super easy” to implement. As educators know all too well, human minds are complex and there is no approach that will “always work”. Instead, we provide flexible guiding principles and do not promote a one-size-fits-all approach to educational reform.
Any reader of our blog knows that we love retrieval practice. In fact, we’ve written 83 posts with that tag! However, even our best recommendations do have some boundary conditions. We have discussed previously the difficulties associated with transfer of learning and writing high-quality multiple-choice tests. Not all retrieval practice is created equal. Today, I want to tell you a little bit more about one of the possible pitfalls associated with testing, but (spoiler alert!) I will also provide concrete suggestions for avoiding it, so don’t be too dismayed as you’re reading!
Image from Pixabay
The pitfall in question is sometimes referred to as the negative testing effect. Essentially, what that means is that in some situations, taking a test can actually hurt learning instead of showing a benefit. In one study (1), participants read lists of unrelated rhyming words (e.g. pickle-nickle; feel-steel). The participants didn’t know, but the words were grouped into categories (e.g. metals for the two previous examples). During the first study session, participants saw all the word pairs in random order. Then half of the participants restudied the words, now blocked by category, while the other half tried to recall the words. Those participants who restudied the blocked words performed much higher on a final test than those who were tested.
Image from cited source
You may be scratching your head a bit and wondering how this relates to education or the classroom. The materials and procedure are certainly not educationally relevant, but what if the same effect occurred with classroom materials? In a recent study (2), researchers sought the answer to this exact question. First, all participants studied the steps necessary to change the brake pads of a car. Half of the participants then restudied, while the other half were tested with fill-in-the-blanks for key words in specific steps. The final test then required participants to put the steps in the correct order and fill in the missing words. There was no difference between the two groups for the order reconstruction, but the ones who had been tested showed the typical testing effect for the missing words.
This is good news for educators. The negative testing effect did not persist with educationally relevant materials. But why? In this case, it may have to do with the materials themselves. There is a logical order to these steps and there is some research to indicate that testing can actually help with organization of material (3).
Image from Pixabay
Negative effects of testing occur for other types of material as well. During a multiple-choice test, a student may come across a question for which they do not know the answer. They may consider all of the options and choose one of the incorrect “lures”. When asked this same question later, the student is more likely to produce the lure response, now believing it to be true (instead of an educated guess).
This effect has been replicated in a few different studies looking at multiple-choice testing (4,5). However, there is also data to suggest that there are ways to avoid negative effects of multiple-choice testing. Perhaps unsurprisingly, as the number of lures and the difficulty of the test increases, so does the likelihood that students will choose an incorrect lure and create a false memory (6). But that doesn’t mean you should make your tests easy or that you should get rid of multiple-choice tests! Difficult, well-constructed tests improve memory. Rather, the best way to counteract the negative testing effect is to provide explanatory feedback to students (6), but it doesn’t have to be right away (even though that’s probably their preference!; 7).
Bottom Line
1) Consider your materials. There are some materials that lend themselves to a possible negative testing effect, but others that probably don’t. Consider what students will focus on in order to answer the questions you’re providing and whether believing their wrong answers could hurt them down the line.
2) Don’t give too many lures. If you’re giving a multiple-choice test, don’t overwhelm students with options. Provide a few plausible lures, but stop there.
3) Provide feedback. While it doesn’t have to be immediate, students who believe something incorrectly can actually overcome these false beliefs and remember the incorrect information (sometimes called the hypercorrection effect; 8).
For more information, read some of our related blogs:
http://www.learningscientists.org/blog/2017/10/10-1
http://www.learningscientists.org/blog/2016/4/3/weekly-digest-4
References:
(1) Peterson, D.J., & Mulligan, N.W. (2013). The negative testing effect and the multifactor account. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39, 1287-1293.
(2) Wissman, K.T., & Peterson, D.J. (in press). Investigating the replicability and generalizability of the negative testing effect. Journal of Applied Research in Memory and Cognition.
(3) Zaromb, F.M., & Roediger, H.L. (2010). The testing effect in free recall is associated with enhanced organizational processes. Memory & Cognition, 38, 995–1008.
(4) Roediger, H. L. III, & Marsh, E. J. (2005). The positive and negative consequences of multiple- choice testing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 1155-1159.
(5) Brown, A. S., Schilling, H. E., & Hockensmith, M. L. (1999). The negative suggestion effect: Pondering incorrect alternatives may be hazardous to your knowledge. Journal of Educational Psychology, 91, 756-764.
(6) Butler, A. C., & Roediger, H. L. III. (2008). Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Memory & Cognition, 36, 604–616.
(7) Mullet, H. G., Butler, A. C., Berdin, B., von Borries, R., & Marsh, E. J. (2014). Delaying feedback promotes transfer of knowledge despite student preferences to receive feedback immediately. Journal of Applied Research in Memory and Cognition, 3, 222-229.
(8) Butterfield, B., & Metcalfe, J. (2001). Errors committed with high confidence are hypercorrected. Journal of Experimental Psychology: Learning, Memory, & Cognition, 27(6), 1491-1494.
