GUEST POST: Using Checklists to Improve College Teaching
By Dr Javier Hidalgo
Dr Javier Hidalgo is a political theorist whose teaching and research interests center on ethics and international affairs, especially the ethical and public policy questions relating to immigration.
He teaches courses such as Leadership and the Humanities, Justice & Civil Society, and advanced courses on leadership in international contexts.
A Fulbright Scholar, he earned a bachelor's degree in political science and philosophy from Reed College and a master's and doctorate from the Program in Political Philosophy at Princeton University. He was a visiting scholar at Brown University prior to joining the Jepson School.
In The Checklist Manifesto, Atul Gawande describes how checklists save lives and prevent disasters in medicine, aviation, engineering, and other areas.
Here’s an example from Gawande’s book: the Boeing Corporation’s Model 299, the Flying Fortress. An early test of the Flying Fortress resulted in a crash and the deaths of two men. What happened? The pilot failed to release the elevator locks. But there was a bigger problem. The Flying Fortress was “too much airplane for one man to fly.” That is, the task of flying the Model 299 was too complex. In response, Boeing created a pilot’s checklist. By using a simple checklist that told the pilot what to do, the Model 299 became flyable.
Why might checklists help? Gwande points to two reasons. First, our memories and intellects are fallible. We suffer from cognitive overload. Second, we’re overconfident. We might skip some steps because we’re confident that we can succeed regardless. Checklists address both problems. They reduce cognitive load and simplify our tasks. If we stick to them, checklists can reduce the risks that accompany overconfidence.
Teaching suffers from these problems too. First of all, there’s so much to know about teaching and learning (1). There are the fundamental principles of learning. Even if you have those down, it’s hard to say how to implement them in any given context. And the science of learning is progressive. Scientists are publishing new results all of the time and it’s tough to keep track of them all.
Image from Pixabay
Moreover, it’s easy to be overconfident when it comes to teaching. Surely my students are learning, right? But take Eric Mazur’s story. Mazur is an acclaimed physicist at Harvard and a path-breaking teacher. Yet Mazur wasn’t always an effective teacher. He recounts his story here. Mazur reports that he got high teaching evaluations in introductory physics classes and he started to believe that “I’m the world’s best physics teacher.”
But Mazur says: “it was an illusion.” Mazur gave his students the Force Concept Inventory, a standardized physics assessment. They bombed. Despite the high student ratings, Mazur’s students weren’t learning well. This information led Mazur to pioneer effective teaching techniques. The moral of the story is that it’s easy to convince yourself that you’re doing a great job at teaching when you aren’t.
So, teaching confronts the problem of complexity and the temptation towards overconfidence--pretty much the same problems that pilots, structural engineers, and surgeons confront. Checklists have helped other professionals. For example, surgical checklists lead to better outcomes for patients (2). Can checklists also help college instructors to more effectively promote student learning?
Image from Pixabay
I thinks so. The challenge is coming up with good checklists. To find a good checklist, I first looked to Carl Wieman and Sarah Gilbert’s Teaching Practices Inventory (TPI). The TPI is a survey that measures whether teachers adhere to evidence-based practices in their science courses (3). It includes items like:
Check all that occurred in your course:
Students asked to read/view material on upcoming class session
Students read/view material on upcoming class session and complete assignments or quizzes on it shortly before class or at beginning of class
Reflective activity at end of class, e.g. “one-minute paper” or similar (students briefly answering questions, reflecting on lecture and/or their learning, etc.)
Student presentations (verbal or poster)
There’s strong evidence that adopting certain instructional practices, such as small group problem-solving, boost learning. The TPI can tell us whether instructors adopt these practices.
But there are problems with the TPI. The TPI is somewhat long and complex, which might explain why Wieman has struggled to get professors to fill it out (4). Also, the TPI is designed for lecture-based science courses, whereas I teach small courses in the humanities. It’s unclear how well it applies to my classes. Finally, the TPI doesn’t really tell you what teachers should be doing. It mainly asks instructors to describe what they in fact do.
I decided to modify the TPI for my own purposes. I took many of the items in the TPI and turned them into a checklist for assessing my courses. I made them specific to my particular institution: a liberal arts college where class sizes are relatively small and where there are no TAs. Furthermore, instructors should be able to fill this out in a few minutes. Finally, I made the checklist normative. It tells me what I should be doing, at least insofar as I’m concerned about student learning. I’ve done my best to survey the scholarship on teaching and learning to ensure that each item has empirical support.
You can find a copy of my checklist here. It covers most aspects of a class: from syllabus design to in-class activities and feedback. If you click on each item, it will take you to a GoogleDoc that describes the evidence for each practice and advice on how to implement it. If you don’t like some item on the checklist, feel free to modify it.
Is my checklist perfect? Far from it. There are teaching practices that boost learning that I’ve neglected to include in part because I couldn’t figure out clear ways of describing them in a sentence. Moreover, even a highly effective class won’t use all of the practices on the list. But perfection is the wrong standard. The right standard is: does following this checklist increase the odds that my class effectively promotes learning? I think that the answer to that question is “yes.” Like any useful checklist, it reduces cognitive complexity and dampens my overconfidence. It helps me to reflect on my classes and evaluate whether they support learning.
Image from Pixabay
Also, I’ve started work on checklists for planning effective class sessions and designing a learning-focused syllabus. As with my course checklist, I’ve appropriated material that others have come up with in order to formulate these lists. And keep in mind that I’ve created these checklists to solve my own problems. But many of the items should generalize to other instructional contexts.
Learning-centered inventories like the TPI can help with evaluating teachers too. The main way that universities evaluate instruction is through student evaluations. Yet student evaluations have many problems. The biggest one is that they don’t seem to correlate much, if at all, with learning (5). Wieman proposes that we use the TPI to measure teaching effectiveness instead of student evaluations (6). For now, though, a teaching checklist can serve as an optional supplement to student evaluations. You might submit your course checklists along with your student evaluations or annual review. These checklists can serve as evidence that you’re implementing effective practices in your courses.
You might object: “teaching is too complex to boil down to a checklist.” That’s true. But surgery, structural engineering, and piloting are also complex. They can’t be reduced to checklists. Nonetheless, checklists benefit practitioners in those fields. So why can’t checklists help with teaching too?
References
(1) Weinstein, Y., Sumeracki, M., & Caviglioli, O. (2019). Understanding how we learn: A visual guide. Abingdon, Oxon: Routledge.
(2) Bergs, J., Hellings, J., Cleemput, I., Zurel, Ö., Troyer, V. D., Hiel, M. V., Demeere, J.-L., Claeys, D., & Vandijck, D. (2014). Systematic review and meta-analysis of the effect of the World Health Organization surgical safety checklist on postoperative complications. BJS (British Journal of Surgery), 101(3), 150–158.
(3) Wieman, C., & Gilbert, S. (2014). The Teaching Practices Inventory: A New Tool for Characterizing College and University Teaching in Mathematics and Science. CBE—Life Sciences Education, 13(3), 552–569.
(4) Wieman, C. (2017). Improving How Universities Teach Science. Harvard University Press.
(5) Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42.
(5) Wieman, C. (2015). A Better Way to Evaluate Undergraduate Teaching. Change: The Magazine of Higher Learning, 47(1), 6–15.
