By Megan Sumeracki

Cover image by RDNE Stock project from Pexels

In my last blog, I covered a paper I published (1) about questions in class, covert retrieval, and cold calling. The conclusion from this paper, and the blog post covering it, is that inserting questions into a lecture, and encouraging students to covertly retrieve through cold calling, ought to improve learning from a lesson. If time does not permit writing out the answers (one form of overt retrieval), or doing so would disrupt the flow of a discussion, then students should still benefit from bringing the information to mind if they are encouraged to do so fully through cold calling. However, it is often quite difficult to get students to covertly retrieve, and some research shows metacognitive judgments—in this case how well students think they are learning the material—may be artificially inflated during covert retrieval. Overt retrieval probably should be used whenever possible.

When I’ve talked to educators about this paper, and cold calling generally, there is concern that cold calling will harm the students or the atmosphere of the class. This is a valid concern, and one that I had when I was conceptualizing this project. Thankfully I found some data addressing the concern!

The Study

In 2019, Elise Dallimore and colleagues (2) published a paper titled Leveling the Playing Field: How Cold Calling Affects Class Discussion Gender Equity. In their paper, they report an observational study from an undergraduate managerial accounting course. There were 16 sections of the course taught by seven different instructors, but all instructors taught from the same textbook and required curriculum. However, each instructor had their own natural style of teaching, and so there was variation in how often students were cold called across the sections of the course. These differences allowed the researchers to ask whether students’ attitudes toward participation in class discussions and observed levels of participation varied with the differences in cold calling across the sections of the course. They also examined whether the gender identity of the students was related to their attitudes and participation levels associated with cold calling. (Note this research was correlational and not a true experiment. This does mean there are some limitations, especially related to causality! More on this later.)

Students completed a survey at the beginning and end of the course about attitudes toward class participation, and recorded their gender identity. In addition, a research assistant who was unaware of the nature of the study observed two class sessions in which they recorded how often students volunteered to participate or were cold-called, and what appeared to be the gender identity of the student. (This is absolutely a limitation because we cannot know a person’s gender identity without asking them.)

Results

Participation Observations

The researchers found that when there were low levels of cold calling in a class, men tended to participate voluntarily more than women. When cold calling was frequently used within a class, students, both men and women, volunteered to participate more often. Further, the increase was larger for the women students than the men students. Thus, frequent cold calling within a class was associated with greater numbers of women volunteering to participate during class.

The researchers also analyzed how often men and women students answered questions in class. For men, it didn’t matter whether the instructor used cold calling more or less frequently. However, women answered significantly more questions when cold calling was frequently used than when cold calling was not frequently used. In fact, in classes in which the instructor used cold calling more frequently, men and women did not differ in the number of questions they answered. Thus, cold calling was associated with closing the gender gap regarding questions answered during class.

Survey data

On the survey administered at the beginning of the course, women reported they were less comfortable during class participation than men, and that they were less satisfied with their own class participation performance than men. (Note, for the survey data, each student’s gender identity was reported by them.) When the researchers analyzed data from the survey administered at the end of the course, they did not see any changes. Students did not report being more or less uncomfortable than they did at the beginning of the course, regardless of whether they were in a class that employed cold calling frequently or infrequently.

What does this mean?

In summary, when instructors used more cold calling, students more often participated in class. In particular, women participated more often and answered more questions when they were in classes in which instructors frequently used cold calling. Students’ attitudes towards participation and their comfort level did not change from the beginning of the course to the end of the course. This means that students were not made less comfortable in the classes where cold calling was used often.

However, this research is correlational (see this post for more details on different types of research methods). Because students were not randomly assigned to class sections, and most importantly instructors were not randomly assigned to use cold calling or not, we do not know whether cold calling caused increases in participation, especially for women. It could be that other factors related to whether an instructor tends to use cold calling led to greater participation, especially for women. It could also be that students who tend to participate more were more likely to enroll in classes with professors who use more cold calling. Instructors can develop reputations for teaching style, and students do talk to one another and/or share this information online. Students’ schedules, both for other classes they need to take and outside of school, can dictate which sections of a course they enroll in. However, when they have choice, they may pick courses taught by instructors who they think will teach in ways that they like, and avoid instructors they think will teach in ways they do not like.

So, from this study we cannot know whether cold calling causes changes in participation, or whether it causes participation gender gaps to close. In other words, if an educator begins cold calling in their classes, they may not see improvements in participation or reduced gender gaps. However, this study does suggest that students are not uncomfortable as a result of cold calling, and highlights the need for a true experiment to determine causality!

References:

(1) Sumeracki, M. A., & Castillo, J. (2022). Covert and overt retrieval practice in the classroom. Translational Issues in Psychological Science, 8(2), 282-293. https://doi.org/10.1037/tps0000332

(2) Dallimore, E. J., Hertenstein, J. H., & Platt, M. B. (2019). Leveling the playing field: How cold-calling affects class discussion gender equity. Journal of Education and Learning, 8(2), 14-24. https://doi.org/10.5539/jel.v8n2p14