The Dunning-Kruger Effect and Self-Assessment

“The first rule of the Dunning-Kruger Club is that you don’t know you are in the Dunning-Kruger Club”

-Various members of the Twitterati.

The Dunning-Kruger Effect is a phenomenon suggesting that people with low ability in a given field are likely to over-estimate their competence. The original paper by David Dunning and Justin Kruger, originally published in 1999, is available here, although, as they themselves acknowledge, the paper builds significantly on prior psychological studies. The authors describe the phenomenon as a “dual burden” for those with limited knowledge in a particular domain- “not only do [low ability individuals] reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it.”

Importantly, Dunning and Kruger suggest two conditions which are likely to be required for the phenomenon to hold. The first condition is when “knowledge about the domain confers competence in the domain”. If one has a significant knowledge of the mathematics then one is necessarily a competent mathematician. Compare this to something like sports where one might possess excellent knowledge of even the most technical aspects of the given sport but not have the physical prowess to execute the skills in the manner they know is required for exceptional performance (this is the position of many of the great sports coaches throughout history). In sports, as in many situations where physical skill is required, knowing what do do does not entail being able to do it.

The second condition that Dunning-Kruger is that some threshold knowledge is required of the subject in order for the the effect to hold. I am not likely to overestimate my ability to translate passages of text into Arabic as I simply would not be able to write anything down if asked to perform such a feat and I am all too aware of this fact. If however someone asked me to do the same in French, my very limited knowledge of this domain (largely unused since GCSE , much to my discredit), might allow me to generate some correct (or at least, vaguely plausible answers) and thus I might well overestimate my ability. In these cases, a little knowledge can be a dangerous thing.

It seems reasonable to suggest that Mathematics, along with a large number of other school subjects, fulfils both criteria.

Whilst this post focuses on over-confidence and a few possible implications for self-assessment, I am obviously aware that for many students a lack of confidence is a major barrier to success in school. This is something I am acutely aware of and it is just as important for teachers to be aware of this as over-confidence. In my experience however, teachers are generally more aware of students lacking confidence than they are of the over-achievers and they take this into account when  interacting with and assessing students.

It is reasonable to state that self-assessment is an often used tool by teachers in many schools. RAG, coloured cards, thumbs up, confidence scales from 1-5 etc. are all fairly common sights in secondary classrooms. Whilst unlikely to form the whole of a teacher’s or school’s assessment model, it is often used as a way for teachers to decide the ‘direction’ of a lesson or a sequence of lessons. “You guys are all ‘green for this? Great! Let’s move onto the next topic”. “Ok, lot’s of reds here, perhaps we need to take a while longer looking over this”. However, the Dunning-Kruger effect suggests that this is a poor approach to take as if students lack competence in a particular field, they have a tendency to overestimate their skill. Note that this phenomenon could affect older students as well as younger ones- when students encounter new material they are novices and are unlikely to know what they don’t know in that field.

A Few Suggestions- both conservative and radical

  1. Base any in class assessment as much as possible on data and your own observations and knowledge of the students rather than their perception of their competence. Rather than asking students if they are Red, Amber or Green for a particular topic, ask them how many questions in that exercise they correctly completed or perhaps give them a test on the subject to get some objective numerical data that you can then use to decide on your course of action. This is simple but it is amazing how often teachers still ask students for something based upon students’ feelings rather than something objective, valid and reliable approach.
  2. Be aware of the Dunning-Kruger effect when discussing learning with students. This is particularly important around exam periods where students inevitably have to make decisions about the areas on which they need to focus their revision. It might be that students need more guidance than one might think when it comes to supporting them in making decisions about where to focus their efforts. This is particularly the case if students are making decisions on the basis of ‘feel’ (in my experience this is the norm) rather than empirical data about their strengths and weaknesses. Interestingly however, Dunning and Kruger don’t discuss the ability of people to make relative self-assessments between similar disciplines. Perhaps a student might overestimate their competence in both vectors and calculus but it might be that they are at least able to make relative comparisons between the two so they can prioritise their efforts. I.e. they think they are better than they are in both subject areas but at least they are accurate in realising that they are generally better at calculus than vectors. Be aware however that some students might incorrectly consider themselves to be so accomplished in particular areas that they do not need to work on these areas at all.
  3. Give students more negative feedback. This is obviously a more radical approach and isn’t one that I have tried myself. Dunning and Kruger tentatively suggest that a possible explanation for their eponymous effect is that “people seldom receive negative feedback about their skills and abilities from others in everyday life”. I would certainly extend “everyday life” to the classroom. Most teachers I know (and I definitely include myself in this) are reluctant to tell students (and indeed parents) they aren’t very competent in particular areas, especially in such stark terms. Feedback is almost always given a positive spin and it might be that students are not even aware that they are being told that this is an area in which they need to improve. Obviously this is done with good intentions and is designed to preserve students’ self-esteem and confidence. However, perhaps this does more harm than good. Reading Dani Quinn espouse the virtues of competition and public sharing of results in the book Battle Hymn of the Tiger Teachers and listening to her interview on the Mr Barton Podcast made me think about this in more depth. Whilst Dani (and presumably other teachers at her school) accompany this with a carefully constructed narrative that focuses on effort rather than ability, this ranking of students means that they are certainly likely to have a better sense of their own ability and are presumably less likely to fall prey to the Dunning-Kruger Effect. Although I know this is something of an anathema for a large number of teachers, it is something worth considering further as there is at least some case to be made for this, even if it is not something that one ultimately agrees with.
  4. Just focus on teaching them subject content! Dunning and Kruger write that “[paradoxically] once [the participants in their experiments] gained the metacognitive skills to recognize their own incompetence, they were no longer incompetent”. If correct, for teachers to develop students’ self-assessment ability then they they could just focus on improving students’ mathematics. As they get better at maths, they will get better at self-assessment. This is not to downplay the importance of self-awareness as a component of expertise, rather it is to say that this awareness will come as one’s knowledge develops. Be patient with the development of this aspect of students’ expertise and don’t rush it.

Wrap up

I am well aware of the irony of me, a Maths Teacher with no background in psychology, writing an article on novices overestimating their ability. Throughout I have tried to couch my thoughts in the language of uncertainty given that this is not my area of expertise. However, I am confident that Dunning and Kruger are experts in this field so if this has made you do nothing more than read their papers and relate them to your own classroom practice then for me that is a job well done.

Type 1 and Type 2 Fun

Railing against ‘fun’ has been done.

The case for the prosecution has been made in many cases including here, here and here. Here are a few of the salient points:

  • Students are likely to remember the ‘fun’ activity rather than the learning itself;
  • ‘Fun’ lessons are a way of trying to ‘trick’ students into enjoying the learning, rather than encouraging students to appreciate and value learning for its own sake.
  • Student engagement is a poor proxy for student learning;
  • David Didau quotes John Hattie in saying that the hard work of learning “is not always pleasurable and easy; it requires over-learning at certain points, spiralling up and down the knowledge continuum, building a working relationship with others in grappling with challenging tasks… this is the power of deliberate practice and concentration.”

It is also worth addressing the straw man in the room- that rejecting this idea of ‘fun’ does not entail planning dour and boring lessons, rather, it stresses that learning, and an engagement with the subject for its own sake rather than for any extraneous gimmicks, should be at the heart of one’s classroom practice. This relates tangentially to another of my pet peeves- contrived ‘real life’ maths questions in which the context serve no purpose (not the sort that encourage students to identify relevant information as part of the question- that is a huge part of a good mathematical education….another blog post for another time).

A discussion last summer with a friend of my father’s about cycling (obviously- it’s the subject du jour for men of his age) shed a different light on the concept of ‘fun’. He made the distinction between what he calls ‘type 1 fun’ and ‘type 2 fun’. I contend that we as teachers should almost always be striving for type 2 fun whereas type 1 is what leads to many of the pitfalls listed above.

Type 1 fun stems things that feel great when we are doing it. It is things that make us want to laugh and smile. Depending on one’s preferences this may include laughing with friends, dancing, sex or reading a good page turner. As one of the articles on the subject that I read put it, Type 1 fun is “fun to do, fun to remember”.

Type 2 fun occurs with things that aren’t that fun at the time but bring a sense of pleasure when one looks back and reflects upon them-‘. My dad’s friend used the example of a hilly cycle. From what I understand, cycling has it’s share of ‘type 1’ fun- the sensation of speed for instance or the views that might be experienced on the course of a particularly scenic ride or the sense of solitude one might have cycling on a quiet road in rural France. The essence of the enjoyment of these activities is based in the moment itself. However, much of cycling’s enjoyment is  ‘type 2’. The sense of accomplishment afterwards, reliving the challenges of what was at the time a quadricep-busting lactic-acid inducing climb that wasn’t enjoyable in any way whilst one with your companions after. In other words ‘not fun to do, fun to remember’.

Type 3 fun is also interesting- ‘not fun to do, not fun to remember’ BUT makes a great story while sitting in the pub or round a campfire! Often these are life or death situations and make great films (Apollo 13, Touching the Void, Everest etc.). We should probably avoid type 3 fun in the classroom (I don’t think this is what is meant when SLT talk about ‘taking risks….’).

Like most Maths teachers, I enjoy geeking out on a particularly good problem that is pitched just right for my level of expertise. But my feeling at the time I am doing the problem isn’t one of enjoyment. It is a feeling of being challenged, of curiosity and more often than not, frustration. But I persist because I know that the reward I get if I complete it and reflect on what I have done and look at different strategies will be huge! (As an aside I would be interested to know whether anyone feels that solving difficult problems is for then a ‘type 1’ rather than a ‘type 2’ activity). I suggest that we should be instilling and developing this feeling in our students.

‘Type 2’ fun in the classroom goes hand in hand with encouraging a lifelong love of learning. If enjoying and relishing the challenge of learning for its own sake is something we are seeking in our learners then perhaps dismissing ‘fun’ out of hand is too short sighted. Aim for type 2 fun, avoid type 1 fun (and definitely avoid type 3 fun).

 

If you are after an excellent selection of puzzles that you can use for school competitions each and every week that will help develop students’ appreciation of type 2 fun- head over to my colleague Andrew Sharpe’s excellent Puzzle of the Week Website http://www.puzzleoftheweek.com/

(Credit to the following-the ‘Three and a half types of fun’ from Teton Gravity Research and ‘3 types of fun’ from the Pebbleshoo blog).

 

 

 

Thoughts on teaching so-called ‘low ability’ Year 7

By now most teachers will have spent a week with their new classes. Inevitably some of you reading this have been throwing your hands up in despair about your new Year 7 class, complaining to colleagues that they are “the weakest you’ve ever taught/taught in years” and lamenting the fact that your primary counterparts have clearly fiddled the data and that the students must have been coached to death through their SATs exams without actually learning anything.

Sound familiar?

My knowledge of primary education is far too limited to even begin to comment on any of the above issues. However, what is indisputable is that a fairly large number of students, for whatever reason, arrive at secondary school ill-prepared for the demands of the mathematics curriculum.

Rather than trying to unpick why this is the case, I want to outline some of the considerations I find to be particularly important for these ‘low ability’ Mathematics students. Note that this term is imperfect and has far more negative connotations than I would like. To me it implies an inability to improve, something that is the polar opposite of what we as teachers work towards. However, I will use this term a. for lack of a better one and b. because I assume that most at least understand what the term means irrespective of whether or not they agree with my view that it has negative connotations.

Don’t teach them what you ‘should’ be teaching them, teach what they need to know to know now

If students are lacking basic skills and if they do not have certain facts committed to memory, they will not be able to make progress. Fact. Much of the philosophy behind the mastery approach to teaching is predicated upon this idea. Nail the basics so that they do not take up space in students’ working memory, thus allowing them to apply these concepts to more complex problems.

In Mathematics, I would go so far as to say that if the four operations and an understanding of place value and the decimal system is not 100% secure then that is what you as a teacher should be almost exclusively focusing. Trying to teach students to multiply two decimals together, to calculate area and perimeter or to carry out any sort of task involving algebra (as well as most other tasks on a typical Year 7 scheme of work) is likely to be to no avail without these in place first. The added cognitive strain of carrying out these operations whilst applying them to a new context will make committing any new processes to memory nigh on impossible.

And by basic I mean basic. Number bonds to 10, 50, 100. Place value. Chanted multiplication learnt by rote, basic inverse operations and not much else (For more on this, see Bruno Reddy’s account of how King Solomon Academy in London designed a ‘Mastery Curriculum’).

Many Year 7 schemes of work do not start with these ‘basics’. At the previous two schools I have worked in, ‘factors, multiples and primes’ has been the first topic covered in Year 7. Finding the factors of 48 without having multiplication tables committed to memory is an almost impossible task.

If you identify with this and your students are not 100% confident with the ‘building blocks’ of Mathematics, implore the powers that be to let you focus on that at the expense of all else. They will catch up later.

Complaining ‘they should know this by now’ is pointless. Make it your job to teach them this. Right now. And don’t rush. embedding these concepts takes time and once embedded must be periodically recapped to help commit them to long-term memory and to allow them to be effortlessly deployed as and when they are needed. This is a process that takes months and even years, not weeks.

 

Get them enthused

Even amongst the most disaffected students, it is rare to find a student who does not exhibit at least some excitement in starting secondary school. Harness this. For the love of God grab onto this and don’t let go. Keeping this level of excitement and enthusiasm for the entire year is a tough and seemingly impossible task but if you are constantly stoking that enthusiasm and creating a positive mindset, students will hopefully overcome many of the negative perceptions that they associate with Mathematics. New year, new you etc!

How to do this? Well that is the million dollar question. I only have one real guideline here- get students excited about getting better. Students like being good at things. They like this more than almost anything else. If you can show pupils how they are progressing and getting better (track quiz scores, table tests etc.) and if you can give them real, genuine, praise about their progress, they are more likely to ‘buy into’ your maths lessons. A generally positive learning environment with great relationships underpins this obviously, but you must get students excited about doing and progressing in maths, not about making posters, using technology or anything else.

 

Planning….or more accurately thinking and rehearsing

Obviously all lessons should be carefully and thoughtfully planned but when teaching lower ability students, meticulous planning is more important than ever. By planning I don’t mean full ‘Ofsted-style’ lesson plans. Rather, I mean taking the time to think through possible misconceptions, requisite prior knowledge and to  carefully rehearse explanations, focusing in particular on the language that you are using and the expectations for how the work should look on the page.

Given that the concepts being taught to these students are at the more ‘basic’ end of the spectrum, there is a tendency to almost ignore the explanation and to try to explain it off the cuff in class whereas when teaching  (say) a difficult A Level topic, many teachers spend more time considering this aspect of their practice. Given that these concepts are going to underpin the rest of the students’ Mathematical understanding and the fact that these students have almost certainly been taught these topics before but haven’t managed to retain the information, these explanations are the most important you will give as a maths teacher. Take time to get them right.

One area that I am very keen to develop in my own practice is the use of concrete resources to aid explanations. This is something I have never felt confident using myself having received a fairly half-baked 45 minute training session on concrete resources during my ITT year  and very little else since other than being told in my NQT year to “just use some Deans Blocks with them” (I imagine this is not uncommon among secondary maths teachers). From what I gather however, if these resources are used correctly, the impact on student understanding can be significant.

Feedback

There is no shortage of research on feedback and its importance. There is also no shortage of information telling you exactly how to give feedback. It must, of course be written (for parents, senior leadership, Ofsted etc), must give the opportunity for the student to give a written response and should involve peers…..

Whilst this might be applicable in some cases, my personal experience suggests that feedback for lower ability students should, by and large be immediate and is most effective when given verbally. Students who struggle in maths tend not to be able to effectively link the written feedback on a page with answers and solutions that they wrote down a day or two previously. Immediate verbal feedback given while circulating the classroom and mini whiteboard work combined with regular low-stakes quizzes to assess learning are far more effective than written comments in books for these students in particular.

Additionally, I find that, whilst students can be ‘trained up’ to be fairly effective peer markers, the opportunity cost of doing so (using time that could be spent doing maths) more than negates the relatively small benefits of being able to review another piece of work and almost all of the benefits derived from peer marking can be given through teacher produced model answers.

Scaffold the transition

One thing I am often struck by when entering a primary classroom (something I do not do enough of, especially given I work in a very large ‘all through’ school) is how different it is to secondary classrooms and in particular, my ‘unfashionable’ classroom layout with students seated in rows facing the board.

Whilst I am convinced that seating students like this has a positive impact on student learning, I am also of the view that for Year 7 students in particular, this sort of classroom environment can be something of a shock to the system. Similarly, having students work on questions from the board rather than on worksheets or listen in their seats rather than in carpet spaces can be very difficult for students who haven’t experienced it before. While the end goal for me is to get students to a position where they are able to work effectively in a traditional secondary setting, secondary teachers must acknowledge these challenges and try to wean students away from the primary mentality but must do so gradually.

If you have a space to do some work ‘on the carpet do it. Use worksheets. Sit down with students and work through problems together. Occasionally group tables together and move students around. Interleave ‘primary’ methods with ‘secondary’ methods and over time begin to move away from the former and towards the latter. Even if you are able to keep up this ‘primary-style’ approach, for better or for worse it is likely that your colleagues will not and students need to be prepared for the realities of secondary education.

Wrap Up

This obviously isn’t a ‘how to’ guide. It is just a set of reflections from my experience teaching so-called ‘low ability’ students in mathematics. I’d love to hear the experiences of others and what works and doesn’t work for you.

Teaching these classes can be challenging but, at the risk of sounding sanctamonious, can be the most rewarding classes to teach. The planning and delivery of these lessons requires patience and takes time to get right but, when the students start making steady and sustained progress, the sense of personal satisfaction is right up there with some of the best feelings in teaching.

 

More on modelling

I’ve seen a few blog posts on modelling in Maths this past few weeks so I thought I’d add my thoughts on this absolutely vital skill for teachers…

A perennial problem for Maths teachers is developing students’ ability to ‘see’ their route into the question when it is not immediately apparent which mathematical techniques they need to apply. This week I began teaching the first lesson of the Edexcel C2 Trigonometric Identities and Equations chapter to my Year 12 Class. For those unfamiliar with this chapter, it requires knowledge and application of the following identities:

Screen Shot 2016-02-21 at 7.06.54 PM.png

Normally when selecting the examples I am going to model with the class, I take a bit of time to carefully select the question I am going to use to ensure that the example contains an appropriate level of challenge whilst still keeping the concept that I am trying to teach at its heart. However, this week when modelling for the class I made a point of (somewhat theatrically) picking a question  that I had not reviewed beforehand and talking students through my thought process as I looked at the question. By chance the question I had chosen was a great one from Integral Maths:

Screen Shot 2016-02-21 at 7.13.22 PM.png

Obviously I knew that the question would involve the use of one or both of the above identities, however, the students also have this knowledge. What students don’t have is:

  1. A full mastery of the two identities including the ways in which the identities can be rearranged;
  2. Exposure to a sufficient number of questions that allows them to spot the ways into the question that examiners tend to use

In order to help expedite the process of spotting some of the commonly used methods required to solve these problems, I channelled my inner James Joyce and gave students a stream of conciousness style account of how I would look at and complete the question.

I started off by pointing out that the first thing I noticed was that the LHS was a difference of two squares, although not in the form students would be used to seeing given that this was their first foray into trigonometric equations. I then said that I looked at the RHS and spotted that there was no cosine squared on the right hand side. I then said that I know that the sine squared theta + cosine squared theta identity can be rearranged to give cosine squared theta in terms of sine squared theta and so a substitution would likely be the way forward with this question once the brackets had been expanded. I then continued to talk students through exactly what I was thinking as I solved the question. In the second example, students had to do a bit more of the ‘leg work’, spotting some of the interesting features of the question and doing more of the talking than in the first question.

Students really appreciated being explicitly told about how I went about looking at this question, especially given that I was doing it ‘live’ and so was, essentially in the same boat as them. Allowing students an insight into how an ‘expert’ thinks is one of the most valuable things any teacher can do and is one reason why didactic teaching is not the evil many seem to have argued. Discovery-based, project-based or inquiry-led learning just isn’t an efficient or effective way to develop students into competent Mathematicians. They need to ‘see’ how an ‘expert’ might see a problem, at least when developing their understanding of a topic.

When modelling all teachers do some work talking through how they ‘see’ a question, however, I contend that most of the time we can do more to ‘make the implicit, explicit’ (to borrow a phrase from David Didau) and really talk students through exactly how we, as Maths teachers, view and solve a the problems presented to us.

For F’s Sake….why I don’t teach F, Z and C angles

Throughout my teaching career I have only ever known teachers to teach alternate, corresponding and co-interior angles with reference to Zs, Fs and Cs. Obviously all teachers have stressed that they need to learn the mathematically correct names but invariably students have found this hard to do which hampers them in the long run. The letters, whilst intended as a scaffold have actually become part of the supporting structure. When they are removed the house falls down. I myself have done this time and time again and found myself frustrated by student’s inability to remember that ‘Z angles are alternate’.

Recently I have moved away from Zs, Fs and Cs completely. I have banned any mention of Fs, Zs and Cs from my classroom and focused more on the correct Mathematical words, their meaning and a slightly more conceptual understanding of why the angles are equal/supplementary. I have stressed that ‘corresponding’ means ‘in the same position’, ‘co-interior’ means ‘inside together’ and alternate means ‘one way and then the other’. I have also tried to encourage students to get a feel for the intuition as to why these angle facts exist- noticing for instance that because corresponding angles are in the same position on two parallel lines they have to be equal- they just couldn’t be anything else. Or that co-interior angles are supplementary because of the fact they combine the fact they know about corresponding angles with the fact that angles on a straight line sum to 180 degrees. Similarly for alternate angles- a combination of corresponding angles and opposite angles.

This is a long term approach. It isn’t quick and easy. It takes time, some well-thought out and clear explanations and a focus on long term improvement rather than rapid progress. However, I find that ultimately it improves student understanding and ability to identify and use the angle rules as well as providing students with an added layer of mathematical rigour.