By Althea Need Kaminske

Despite the increased emphasis on the importance of a university degree, the current statistics on persistence in higher education is disheartening. In the U.S. only 54.8% of students complete a four-year degree within six years (1). It’s a big and very complicated problem that many educators are working to solve. At the Learning Scientists we feel strongly that part of the solution lies in sharing evidence-based learning strategies with teachers, students, and parents. There are a few different ways to approach this. One is through science communication, like we do with this blog, to raise the awareness of these learning strategies. Another is to work directly with teachers and schools to provide instruction and training to the people who have the power to affect the greatest amount of change: teachers. Another strategy is to work with students to get them to change their study habits.

Earlier this week I came across a research study that opted for the third approach: attempting to train students to adopt different study habits. This research has been made available as working paper entitled “The Remarkable Unresponsiveness of College Students to Nudging and What We Can Learn from It” by Philip Oreopoulos and Uros Petronijevic (2). It was done over the course of 5 years at the University of Toronto and included a total sample of nearly 25,000 students. It was a massive undertaking and provides some very interesting and compelling insights into what does and does not work when trying to get students to change their study habits. 

Before I give a more detailed description of the study I want to caution that this is still a working paper. This means that it has not gone through the peer review process. The peer-review process is in place to assure that research that is published is high quality. As the name suggests, this process involves a review by peers in your scientific community, wherein potential issues are critiqued and addressed before publication. We therefore tend to lend more weight to research findings that come from peer-reviewed journals and are cautious about results from non-peer-reviewed sources. That said, this paper is a summary of a very large body of work and sub-sections of that work have been published elsewhere in peer-reviewed journals (3, 4). Furthermore, while peer-review is incredibly important to the scientific method, it is also notoriously fickle and time consuming. It could be months, or even years, until this work gets published in the form of a peer-reviewed journal. So while I offer a word of caution that the final product isn’t 100% up to snuff, I still think it’s an important contribution that’s worth discussion.

In this study researchers partnered with professors of a first-year course in economics. Economics is a popular first-year course that is taken by roughly 5,000 students each year at the University of Toronto, making it an ideal course to implement a study intervention in. As a small part of their grade, students were asked to fill out an online survey. After filling out this survey students were randomly assigned to either a control group (which involved taking a personality survey) or a study treatment condition. Over the course of five years, researchers investigated six study treatments:

1. Goal Setting. Students were asked to think about their future goals and reflect on how their current behaviors might or might not help them achieve those goals.

2. Mindest. Students were told about the benefits of adapting a positive and resilient mindset towards academic challenges.

3. Online Coaching. Students completed online exercises where they were encouraged to think about the future they wanted and what steps they could take to achieve it.

4. Online Coaching with One-Way Text Messaging. Students completed online exercises as above, but received follow-up messages throughout the school year that offered advice, encouragement, and information about resources.

5. Online Coaching with Two-Way Text Messaging. Here the online coaching and texting were designed to emphasize study time. In this treatment group upper-year students who were academically qualified and had previous experience mentoring or tutoring were selected to act as coaches and follow up with students via text.

6. Face-to-Face Coaching. Students were offered personal coaches to meet with once a week during the school year. Again, coaches were upper-year students. Coaches were encouraged to follow up with students between meetings and were available via Skype or text messaging between meetings.

There were a number of measures of interest in this study, but for the purposes of this summary I’ll go over three: study time, student grades and persistence, and well-being.

Study Time

The researchers note that university students spend less time studying than they should. Many colleges and universities recommend studying two to three hours outside of class for every one hour spent in class. A typical course load is 15 hours of coursework a week, which translates to 20-45 hours of studying outside of class. Most students spend less than 15 hours a week studying outside of class (5). Therefore, low study time is often blamed for low graduation rates. Student simply aren’t devoting enough time to their studies. 

Of the six treatments in this study the Online and Two-Way Text Messaging groups showed some improvement in study time. On average they studied about 1.3 hours more a week. None of the other treatments produced any positive effect on study time.

Student Grades and Persistence

None of the treatments affected student grades. Regardless of treatment condition, final grades were the same as the control group. Extra coaching, weekly meetings, focusing on goals - none of it impacted grades.

Similarly, none of the treatments affected persistence. Students in the control group were just as likely to continue on to their second and third year as students in the treatment conditions. A student’s decision to stay and persist towards completing their degree was not impacted by any of the treatments.

Well-being

Here we finally get to some good news! The coaching conditions, particularly the Face-to-Face treatment, had positive effects on student well-being. Students who met regularly with a coach had better well-being, including an improved sense of belonging and university support. In the current environment with increased student mental-health issues, this finding is very important.

It is notable that the Face-to-Face treatment was by far the most expensive and time-consuming of all the treatments done in this study. The researchers set out with the express goal of finding a less resource demanding intervention to improve student outcomes. Which is perhaps why this intervention seems to have been developed last and was done with a substantially smaller sample than the rest of the treatment conditions (only 90 students were selected to receive Face-to-Face coaching).

Takeaways

One of the reasons I found this paper fascinating was because it highlights several of the challenges we face when we try to apply what we have learned from basic, laboratory research to real-world, educational settings. The researchers had solid theoretical and empirical reasons for designing the treatment conditions that they did. They worked closely with researchers from social psychology to design their Goal-Setting and Mindset interventions to make sure they were carrying them out properly. The experiments were well designed, using a large sample size and random assignment to make sure they could draw comparisons between groups of students. Despite all of that, it still didn’t work out the way they had hoped - and that’s incredibly valuable information.

Another takeaway from this research is that study time does not equal learning. The authors were particularly puzzled by this finding and devoted several pages to exploring this conundrum. As I mentioned above, the recommended study time is two to three hours for every hour in the classroom or 20-45 hours a week for the average university student. I have no idea who came up with this formula, but I can attest that it’s a fairly pervasive idea, at least in the U.S. university system. I ask my students every year how they should be studying and almost all of them have heard of this recommendation. I then use this as an opening to explore the idea that it’s not how much time you spend that matters, it’s what you do with that time that matters. Using effective learning strategies like retrieval practice, dual coding, spacing, etc. will get you more return on your time invested studying than less effective learning strategies like re-reading, sloppy highlighting, or texting your friends about studying but then watching Netflix instead.

Of course, you do need to set aside actual time to engage in all of these effective study habits. It’s not a terrible idea to encourage students to study more. However, I view it more as a possible first step rather than an end goal of a study intervention.

My final takeaway from this study is that changing habits is HARD. Before we start commiserating about how impossible and pig-headed students can be (“The Remarkable Unresponsiveness of College Students” sounds like the next best-selling book aimed at college administrators that will plague every faculty meeting and recruitment initiative for the next 10 years) I want to take a step back to talk about habits. When was the last time you decided to eat healthier, exercise more, or be more organized? Sure, you made some progress towards your goal. You read up on the latest diet fad, got a gym membership, or read “The Life Changing Magic of Tidying Up”. And you stuck with it! For a week. Or two. Maybe three. But, like most people, a month or so later your enthusiasm for your new way of life wore off and you went more or less back to your old habits. 

Student study habits and time management are like any other lifestyle changes. In this light, it makes sense to me that the interventions that showed the most promise were the ones that focused on social accountability and improved belongingness. What makes people more likely to go to the gym? Having friends who also exercise. Having an acountabilibuddy. Someone who cares how you’re doing and whether you show up. Simply telling someone that it’s good for them or introducing them to the concept does not seem to be enough. If it were that easy we would all be eating a whole-food diet, going to the gym 5 times a week, and coming home to our perfectly clean and organized homes. Living our best lives like the adults we always pictured we would be instead of the slightly frazzled, exhausted, shell of a person that we can become when we let our good habits slip away from us.

References:

1. Shapiro, D., Dundar, A., Huie, F., Wakhungu, P., Bhimdiwala, A., & Wilson, S. E. (2019). Completing College: Eight Year Completion Outcomes for the Fall 2010 Cohort (Signature Report No. 12c). Herndon, VA: National Student Clearinghouse Research Center.

2. Oreopoulus, P., & Petronijevic, U. (2019). The remarkable unresponsiveness of college students to nudging and what we can learn from it. EdWorkingPaper: 19-102. Retrieved from Annenberg Institute at Brown University: http://www.edworkingpapers.com/ai19-109.

3. Beattie, G., Laliberte, J., & Oreopoulus, P. (2018). Thrivers and drivers: Using non-academic measure to predict college success and failure. Economics of Education Review, 62, 170-182.

4. Oreopoulus, P., & Petronijevic, U. (2018). Student coaching: How far can technology go? Journal of Human Resources, 53(2), 299-329.

5. Farkas, G., Mazurek, E., & Marone, J. R. (2014). Learning style versus time spent studying and career choice: Which is associated with success in a combined undergraduate anatomy and physiology course? Anatomical Sciences Education, 9, 121-131.