By Blake Harvard

Blake Harvard is a high school AP Psychology teacher at James Clemens High School in Madison, AL.  He earned his B. S. and M. Ed. from the University of Montevallo.  Blake has a particular passion for cognitive psychology and its application in his classroom.  You can find him on Twitter @coachharvard.  

Below are several problems that vary in difficulty. Try to answer as many as you can.

(1)  A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? _____ cents

(2)  If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets? _____ minutes

(3)  In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake? _____ days 

Have your answers?  They might be wrong. They’re probably wrong; especially if you developed your answers quickly. But don’t feel down: Dr. Shane Frederick of Yale University administered this Cognitive Reflection Test to 3,428 people over a 26-month period beginning in January of 2003. The results?  The mean score was 1.24 questions correct out of 3. Of the respondents, 33% answered zero questions correctly and only 17% answered all questions correctly. Among those questioned by Frederick and his colleagues were students at MIT, Harvard, and Princeton Universities (1)*.

Why did students from some of the most prestigious universities in the world answer these questions incorrectly?  

They certainly don’t lack in IQ or educational background and opportunities; and, notice the instructions: there was no time limit. In his book Thinking, Fast and Slow, Dr. Daniel Kahneman discusses two systems for processing information (a “dual processing” theory). System 1, as Kahneman describes it, “operates automatically and quickly, with little or no effort and no sense of voluntary control.”  System 2 “allocates attention to the effortful mental activities that demand it, including complex computations.” He goes on to state that questions requiring only intuition (System 1) are answered quickly, with very little thought, and with “cognitive ease”. If you got them wrong, you probably answered the three questions at the beginning of this article using System 1. Questions that require more thought, or “cognitive strain”, activate our more deliberate and effortful system of processing (System 2).

So how do we know whether to use System 1 or System 2 when confronted with information? 

According to Kahneman, System 1 carries out assessment automatically and a decision is made regarding whether extra effort is needed to activate System 2. The poor scoring on the three-question Cognitive Reflection Test mentioned above is due to the fact that only System 1 was activated within many of the 3,428 students. Therefore, intuition was primarily responsible for the answers given. This can cause errors in thinking when confronted with such questions that appear to only require our automatic response, but actually require effortful consideration (2).

So how do we activate System 2 when information like that of the Cognitive Reflection Test is presented?  

A different set of researchers took the Cognitive Reflection Test to college students once again – though this time to a much smaller sample of 40. This time half of the students received the questions in a normal, easily legible font. The responses recorded were very similar to the original scores. For the other group of students, they made the questions harder to read. They decreased the size of the font and presented it in a washed-out gray print (2).

The results were striking: When the questions were presented in the harder-to-read font, 65% of students answered all questions correctly; an improvement of 48%. Of those who took the original assessment, 90% made at least one error. That number dropped to 35% with at least one error when the font was harder to read; a decrease of 55%. Performance was actually better when reading was made more difficult. Why? One explanation is the increased cognitive strain. The less legible font encourages the System 2 level of processing, which is slower and more deliberate, and is more likely to reject the intuitive answer from System 1 (3).

How could the results of the Cognitive Reflection Test be applied to the classroom?

As an educator, I am always looking for ways to improve my student’s cognition and retention of material. Could just changing to a less-legible font during presentations, on graphic organizers, and on quizzes/tests really improve my student’s scores?  One school district in Ohio experimented to find out.

Six different classes of children aged 15-18, ranging from the sciences to humanities were used in this study. The classes also ranged from regular classes to Advanced Placement classes. All teachers involved were blind to the hypothesis of the study and were required to have two sections of each class. One class would be used as the control and all handouts and PowerPoint presentations would be presented in a legible font. The experimental group would have the same handouts and PowerPoint materials presented in the following “disfluent” fonts:

The experiment lasted in the different classrooms from a week and a half to one month. The results? Students in the experimental group with the disfluent condition scored consistently higher on classroom assessments than those in the control group. Also, students in both groups were given a four-question assessment measuring the effect of fluency on motivation; no difference between fluency groups were found, suggesting that disfluency improved performance without affecting motivation (4).

Fluency interventions, like those used in the Cognitive Reflection Test study and by the high school in Ohio, has the potential to be be very efficient and cost-effective. This intervention could be applied to all levels of learners and requires minimal planning time and resources. If simply changing to a less-legible font to activate System 2 processing can improve problem-solving, imagine how effective it could be when coupled with other cognitive learning techniques such as spaced learning and retrieval practice?

Addendum: In an attempt to replicate the disfluent font effect with the Cognitive Reflection Test (CRT), researchers (including Dr. Frederick, who created the test) looked at data from over 7,000 participants who had solved these problems using fluent versus disfluent fonts – a much larger population than that of the original experiment, which had only used 40 participants. This investigation yielded rather different results (5):

Easy-to-read average score – 1.43 out of 3

Hard-to-read average score – 1.42 out of 3

According to this meta-analysis of 17 studies – including the original one that showed a large effect – the effects of disfluent fonts on problem solving are actually non-existent!  This blog post by one of the co-authors of the study, Terry Burnham @TerenceBurnham goes into more detail on the failed replication.

We would like to thank Dan @slrrrrp for bringing this new evidence to our attention.

References:

(1) Frederick, S. (2005). Cognitive reflection and decision making. The Journal of Economic Perspectives, 19, 25-42.

(2) Alter, A. L., Oppenheimer, D. M., Epley, N., & Eyre, R. N. (2007). Overcoming intuition: Metacognitive difficulty activates analytic reasoning. Journal of Experimental Psychology: General, 136, 569-576.

(3) Kahneman, D. (2011). Thinking fast, thinking slow. Farrar, Straus, and Giroux, NY, NY.

(4) Diemand-Yauman, C., Oppenheimer, D. M., & Vaughan, E. B. (2011). Fortune favors the Bold (and the italicized): Effects of disfluency on educational outcomes. Cognition, 118, 111-115.

(5) Meyer, A., Frederick, S., Burnham, T. C., Guevara Pinto, J. D., Boyer, T. W., Ball, L. J., ... & Schuldt, J. P. (2015). Disfluent fonts don’t help people solve math problems. Journal of Experimental Psychology: General, 144(2), e16.

*In case you’re wondering, the answers are: (1) 5 cents, (2) 5 minutes, and (3) 47 days.