By Virginia Clinton

Virginia Clinton, PhD is an Assistant Professor of Educational Foundations and Research at the University of North Dakota. She holds a doctorate in Educational Psychology with a minor in Cognitive Science from the University of Minnesota. Dr. Clinton’s research focuses on affective factors in student cognition and learning. You can follow her on Twitter @v_e_clinton and find her website here.

Diagrams can help students learn, especially when students connect verbal information (like words in the text of a lesson) with the diagrams (1). But getting students to make these connections can be tricky because you can’t look at a diagram and the text at the same time! When I was a postdoctoral research associate, my job was to test ways to help students make connections between diagrams and text in mathematics lessons. One idea I thought might work was based on a research study I helped with in graduate school. In that study, as they read, students answered questions that prompted them to make connections with the different ideas in the text. Answering these questions helped students with reading comprehension. I thought a questioning approach like this could work with helping students make connections between diagrams and text.

The questioning approach I was particularly interested in was elaborative interrogation. Elaborative interrogation is answering questions to make connections across different ideas in a lesson or between a lesson and a student’s prior knowledge. These are usually “how” or “why” types of questions, such as reading a science fact and asking – then answering – why it is true (2). Elaborative interrogation is considered an effective study method for good reason. Many studies have shown that helps students think more in-depth and make connections (see blog posts here and here).

The lesson I chose for my study was on probabilistic reasoning, specifically on a particularly challenging topic known as posterior probability (also called the false positive paradox or the base rate fallacy; Wikipedia actually has a decent explanation here).

I choose this topic because it’s important for the accurate understanding of screening test results. Also, it’s pretty challenging (I know I had a hard time with wrapping my head around it!), so techniques to help teach it would be useful.

I had three questioning conditions in an experiment where undergraduate students read a lesson about posterior probability (this was a lab experiment and they got course credit for participating, but they didn’t need to learn the material for a class grade) (3), as follows:

1) Elaborative Interrogation. In this condition, students answered how and why questions connecting ideas within the lesson (for example, “Why does including the base rate make the probability 47%?”).

2) Embedded questioning. In this condition, students answered simple who/what/when/where questions. The answers were stated right there in the lesson (for example, “What is the base rate?” when the definition was explicitly provided). Embedded questioning was used to see if just answering questions, even simple and shallow questions, would help with learning.

3) Read-twice. Reading the lesson twice was a control to account for the time students spent with the lesson. Immediately re-reading something doesn’t seem to help with learning in the long run, so it is a good way to account for some of the extra time spent answering questions while reading a lesson.

After they read the lesson, they had four posterior probability problems to solve that were different than the examples shown in the lesson. The elaborative interrogations were designed to help students understand the material so they could better solve these problems.

When I first looked at my results, I thought, “Yay, there was an effect! Elaborative interrogation must have helped with learning!” Then I looked more closely and realized that elaborative interrogation actually did the opposite of what I expected! Also, when I looked at how long they spent with the lesson, they clearly spent a lot longer writing elaborative interrogations than reading twice.

Basically, students who read twice spent about half the time with the lesson and did twice as well as students in either of the questioning conditions. Not at all what I was expecting.

This puzzled me—why did elaborative interrogation help so many times before, but seem to cause problems in this case?

I looked at the answers to the elaborative interrogations students wrote. I scored them based on how many ideas from the lesson were accurately included: an ideal answer had 3 ideas, an adequate answer had 2 ideas, and inadequate answer had 1 idea, and a missing, circular, or wrong answer had 0 ideas. Overall, the answers were pretty bad—86% were inadequate, missing, circular, or wrong. Only a few showed the types of connections the elaborative interrogations were intended to prompt.

Given that the elaborative interrogations themselves were not what I was hoping for, it makes sense that students weren’t learning more with elaborative interrogation!

The answers to embedded questions weren’t great either. On average, students only got about 50% of them right even though they were generally simple and the answer was stated in the lesson (students may have overthought them).

Another reason elaborative interrogation might not have helped probably had to do with the difficulty of the material. The topic of posterior probability is something many students find challenging and elaborative interrogation may have just made it too hard. After they finished the lesson, I had students rate how difficult the lesson was. Students definitely found the elaborative interrogations made the lesson more difficult. Sometimes difficulty can be desirable and struggling with material can actually increase learning (4). This was definitely not one of those times!

What does this all mean? Obviously, this doesn’t mean that elaborative interrogation is just a useless, time-consuming technique. There’s lots of evidence that elaborative interrogation usually does help. It just didn’t here. The science of learning is complex; even though a technique has been shown to really help in lots of situations, that doesn’t mean it will always help. As another example, writing to learn may not always be helpful (as pointed out before in this post). So, it’s important to remember there aren’t one-size-fits-all, sure-fire fixes in the world of learning science. 

I want to add that answer quality for the elaborative interrogations did increase with problem-solving accuracy. In other words, the better a student’s elaborative interrogations were, the more likely they were to solve problems correctly after reading the lesson. It could be that elaborative interrogation would have been more helpful in this situation if students got guidance on writing good answers through modeling or feedback. Plans to look into this are in the works!

References:

(1) Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York, NY: Cambridge University Press.

(2) Pressley, M., Symons, S., McDaniel, M. A., Snyder, B. L., & Turnure, J. E. (1988). Elaborative interrogation facilitates acquisition of confusing facts. Journal of Educational Psychology, 80(3), 268.

(3) Clinton, V., Alibali, M. W., & Nathan, M. J. (2016). Learning about posterior probability: Do diagrams and elaborative interrogation help? The Journal of Experimental Education, 84(3), 579-599.

(4) McDaniel, M. A., & Butler, A. C. (2011). A contextual framework for understanding when difficulties are desirable. In A. S. Benjamin (Ed.), Successful remembering and successful forgetting: A festschrift in honor of Robert A. Bjork (pp. 175–198). New York, NY: Psychology Press.