The Dunning-Kruger Effect and Self-Assessment

“The first rule of the Dunning-Kruger Club is that you don’t know you are in the Dunning-Kruger Club”

-Various members of the Twitterati.

The Dunning-Kruger Effect is a phenomenon suggesting that people with low ability in a given field are likely to over-estimate their competence. The original paper by David Dunning and Justin Kruger, originally published in 1999, is available here, although, as they themselves acknowledge, the paper builds significantly on prior psychological studies. The authors describe the phenomenon as a “dual burden” for those with limited knowledge in a particular domain- “not only do [low ability individuals] reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it.”

Importantly, Dunning and Kruger suggest two conditions which are likely to be required for the phenomenon to hold. The first condition is when “knowledge about the domain confers competence in the domain”. If one has a significant knowledge of the mathematics then one is necessarily a competent mathematician. Compare this to something like sports where one might possess excellent knowledge of even the most technical aspects of the given sport but not have the physical prowess to execute the skills in the manner they know is required for exceptional performance (this is the position of many of the great sports coaches throughout history). In sports, as in many situations where physical skill is required, knowing what do do does not entail being able to do it.

The second condition that Dunning-Kruger is that some threshold knowledge is required of the subject in order for the the effect to hold. I am not likely to overestimate my ability to translate passages of text into Arabic as I simply would not be able to write anything down if asked to perform such a feat and I am all too aware of this fact. If however someone asked me to do the same in French, my very limited knowledge of this domain (largely unused since GCSE , much to my discredit), might allow me to generate some correct (or at least, vaguely plausible answers) and thus I might well overestimate my ability. In these cases, a little knowledge can be a dangerous thing.

It seems reasonable to suggest that Mathematics, along with a large number of other school subjects, fulfils both criteria.

Whilst this post focuses on over-confidence and a few possible implications for self-assessment, I am obviously aware that for many students a lack of confidence is a major barrier to success in school. This is something I am acutely aware of and it is just as important for teachers to be aware of this as over-confidence. In my experience however, teachers are generally more aware of students lacking confidence than they are of the over-achievers and they take this into account when  interacting with and assessing students.

It is reasonable to state that self-assessment is an often used tool by teachers in many schools. RAG, coloured cards, thumbs up, confidence scales from 1-5 etc. are all fairly common sights in secondary classrooms. Whilst unlikely to form the whole of a teacher’s or school’s assessment model, it is often used as a way for teachers to decide the ‘direction’ of a lesson or a sequence of lessons. “You guys are all ‘green for this? Great! Let’s move onto the next topic”. “Ok, lot’s of reds here, perhaps we need to take a while longer looking over this”. However, the Dunning-Kruger effect suggests that this is a poor approach to take as if students lack competence in a particular field, they have a tendency to overestimate their skill. Note that this phenomenon could affect older students as well as younger ones- when students encounter new material they are novices and are unlikely to know what they don’t know in that field.

A Few Suggestions- both conservative and radical

  1. Base any in class assessment as much as possible on data and your own observations and knowledge of the students rather than their perception of their competence. Rather than asking students if they are Red, Amber or Green for a particular topic, ask them how many questions in that exercise they correctly completed or perhaps give them a test on the subject to get some objective numerical data that you can then use to decide on your course of action. This is simple but it is amazing how often teachers still ask students for something based upon students’ feelings rather than something objective, valid and reliable approach.
  2. Be aware of the Dunning-Kruger effect when discussing learning with students. This is particularly important around exam periods where students inevitably have to make decisions about the areas on which they need to focus their revision. It might be that students need more guidance than one might think when it comes to supporting them in making decisions about where to focus their efforts. This is particularly the case if students are making decisions on the basis of ‘feel’ (in my experience this is the norm) rather than empirical data about their strengths and weaknesses. Interestingly however, Dunning and Kruger don’t discuss the ability of people to make relative self-assessments between similar disciplines. Perhaps a student might overestimate their competence in both vectors and calculus but it might be that they are at least able to make relative comparisons between the two so they can prioritise their efforts. I.e. they think they are better than they are in both subject areas but at least they are accurate in realising that they are generally better at calculus than vectors. Be aware however that some students might incorrectly consider themselves to be so accomplished in particular areas that they do not need to work on these areas at all.
  3. Give students more negative feedback. This is obviously a more radical approach and isn’t one that I have tried myself. Dunning and Kruger tentatively suggest that a possible explanation for their eponymous effect is that “people seldom receive negative feedback about their skills and abilities from others in everyday life”. I would certainly extend “everyday life” to the classroom. Most teachers I know (and I definitely include myself in this) are reluctant to tell students (and indeed parents) they aren’t very competent in particular areas, especially in such stark terms. Feedback is almost always given a positive spin and it might be that students are not even aware that they are being told that this is an area in which they need to improve. Obviously this is done with good intentions and is designed to preserve students’ self-esteem and confidence. However, perhaps this does more harm than good. Reading Dani Quinn espouse the virtues of competition and public sharing of results in the book Battle Hymn of the Tiger Teachers and listening to her interview on the Mr Barton Podcast made me think about this in more depth. Whilst Dani (and presumably other teachers at her school) accompany this with a carefully constructed narrative that focuses on effort rather than ability, this ranking of students means that they are certainly likely to have a better sense of their own ability and are presumably less likely to fall prey to the Dunning-Kruger Effect. Although I know this is something of an anathema for a large number of teachers, it is something worth considering further as there is at least some case to be made for this, even if it is not something that one ultimately agrees with.
  4. Just focus on teaching them subject content! Dunning and Kruger write that “[paradoxically] once [the participants in their experiments] gained the metacognitive skills to recognize their own incompetence, they were no longer incompetent”. If correct, for teachers to develop students’ self-assessment ability then they they could just focus on improving students’ mathematics. As they get better at maths, they will get better at self-assessment. This is not to downplay the importance of self-awareness as a component of expertise, rather it is to say that this awareness will come as one’s knowledge develops. Be patient with the development of this aspect of students’ expertise and don’t rush it.

Wrap up

I am well aware of the irony of me, a Maths Teacher with no background in psychology, writing an article on novices overestimating their ability. Throughout I have tried to couch my thoughts in the language of uncertainty given that this is not my area of expertise. However, I am confident that Dunning and Kruger are experts in this field so if this has made you do nothing more than read their papers and relate them to your own classroom practice then for me that is a job well done.

Advertisements