Questions in Class, Covert Retrieval, and Cold Calling

By Megan Sumeracki

Cover image by 정수 이 from Pixabay

In a previous blog post about retrieval practice, Cindy asked, is asking questions in class enough? She covered an experiment by Magdalena Abel and Henry Roediger (1) in which students studied Swahili vocabulary in a few different conditions. In one of those conditions, students graded another student’s retrieval practice, indicating whether the other student was accurate or not. The student practicing retrieval would see a Swahili word (like mashua) and would need to retrieve the English translation (boat). Then, the student who was grading would have to mentally recall the answer to determine the accuracy of what was retrieved. At least, that was the idea. When someone mentally recalls, and the answer is not explicitly produced, then this is called covert retrieval. It turned out that grading the other student like this did not result in greater learning of the Swahili vocabulary. In fact, this condition led to lower performance on a test 2 days later than just restudying! Abel and Roediger conducted another similar experiment, but this time they told the students to mentally recall the correct answer. So, they explicitly encouraged covert retrieval. Still, this learning condition was not particularly effective.

Cindy then asked in her blog post, what does this mean for educators? She said, “it is not enough to ask questions during lecture because students are unlikely to engage in covert retrieval unless they are pushed to do so.” There have been a few papers on covert retrieval that led to similar conclusions.

However, in today’s post I will talk about an experiment that I conducted with one of my undergraduate students, Johanny Castillo (2), where we showed benefits of asking questions during class and having students covertly retrieve (relative to not asking questions at all). The way we pushed students to retrieve covertly? Cold calling!

The Experiment: Rationale and Method

We often ask questions while we are teaching (at least I do), and there is not always time to stop and ask students to write the answers down. Certainly, I do this as often as I can, but there are times when doing so would disrupt the flow of the discussion in class, or we are short on time and we need to move on. In these cases, I call on one student to answer the question. Is that the only student who benefits?

In our experiment (it was actually the second experiment in the paper), we asked whether inserting questions during a lecture, and then asking students to covertly retrieve the answers, would benefit student learning. We compared this condition to overt retrieval where students stopped to write down the answers, and a no retrieval condition during which students just heard the lecture.

The experiment took place in upper-level research methods courses in psychology, taught by myself and another faculty member at a similar institution. The experiment was within subjects, so all students participated in all three conditions. We had one lecture during which students did not have any questions, and another during which students were given questions throughout the lecture and were instructed to either write the answer down (overt retrieval) or think of the answer (covert retrieval). Different sections of the class across a few different semesters completed the conditions in different orders and with different materials (i.e., we counterbalanced at the section level).

The main feature of our experiment that we reasoned would strongly encourage complete retrieval was the cold calling. After the students answered the question, either overtly or covertly, the instructor randomly called on a student to answer aloud. This way, all of the students had to prepare their answers as best they could, either covertly in their minds or overtly on paper, because they might be called upon to answer. Of course, if the student answering aloud was not accurate, the instructor corrected it in a kind and positive way, and the correct answer was provided.

(As an aside: From the research perspective, the questions for the specific students that were called on to answer aloud were “thrown out” and not analyzed in any of the conditions. Functionally answering out loud put those questions for those particular students into a different condition—the cold-called group—and there wasn’t enough data to actually analyze this as its own condition.)

Two days after the lectures, the students took a final assessment to measure how much they could remember.

The Results

On the final assessment, both retrieval conditions led to greater performance than the no retrieval condition, and there was no statistical or meaningful difference between overt and covert retrieval. In fact, performance in the overt retrieval group was 61.54% and performance in the covert retrieval group was 61.47%! Performance in the no retrieval group, on the other hand, was 36.41%.

Image created by author based on Sumeracki & Castillo (2022)

What does this mean for educators?

Does this mean that covert retrieval is always just as good as overt retrieval? Certainly not. If it is possible to have students write answers to questions, then encourage them to do so. From a metacognitive perspective, students who engage in overt retrieval may be more accurate at judging what they know and what they don’t know (2,3). Is just inserting questions enough, the question that Cindy asked in her previous post? No. However, when we are short on time, or our focus is on keeping a discussion flow, cold calling can be a viable option to encourage covert retrieval during class to help improve learning benefits from posing the question to the class. And, cold calling may actually facilitate discussion in interesting and important ways! A topic that I hope to cover in a future blog post.

References:

(1) Abel, M., & Roediger III, H. L. (2018). The testing effect in a social setting: Does retrieval practice benefit a listener?. Journal of experimental psychology: Applied, 24(3), 347. https://doi.org/10.1037/xap0000148

(2) Sumeracki, M. A., & Castillo, J. (2022). Covert and overt retrieval practice in the classroom. Translational Issues in Psychological Science, 8(2), 282-293. https://doi.org/10.1037/tps0000332

(3) Tauber, S. K., Witherby, A. E., Dunlosky, J., Rawson, K. A., Putnam, A. L., & Roediger, H. L. (2018). Does covert retrieval benefit learning of key-term definitions? Journal of Applied Research in Memory and Cognition, 7(1), 106-115. https://doi.org/10.1016/j.jarmac.2016.10.004