The internet has many ‘laws’; perhaps the most famous is Godwin’s: ‘as an online discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches 1’.
A lesser known ‘law’ is rule 34: ‘if it exists, there is pornography about it’. I think a version of this rule applies to education research: if a claim exists, there is at least some supporting evidence.
The evidence may not be very relevant – or even particularly credible – but that need not stop a determined individual from supporting their pet project. Try it out yourself. I recently tried it with a group of ITT students and the longest it took was 20 minutes for us to find at least some evidence linked to even the wildest claims.
Although there is an increasing appetite for and engagement with evidence in education, my experience is that evidence is often used to justify, rather than inform decisions.
I commonly encounter this with the Pupil Premium where I’ve often been asked to find the evidence that supports pre-determined decisions. Various templates encourage school leaders to identify the evidence that supports particular investments. Ostensibly, this is a great idea, but unfortunately this is too often seen as an afterthought – something that has to be done for Ofsted – rather than an integral part of the decision making process.
Besides being a waste of time, I fear that using evidence to justify, rather than inform decisions risks undermining the growing engagement with evidence. At its best, evidence has the potential to democratise decision making, where ideas are judged on their merit, not who proposes them. I like the tongue in cheek contrast between evidence-based and eminence-based education.
To overcome this issue, it can help to:
- Be willing to change your mind – if you’re not willing to change your mind in response to evidence, then what is the point of looking at evidence at all?
- Be aware of your biases – this is often easier said than done, but applying the strongest scepticism to ideas you favour is good practice
- Be cautious with single studies – by looking at bodies of evidence, you can avoid the problem of cherry picking studies. For instance, one evaluation from the EEF about growth mindset is promising, but the wider body of evidence is much less promising.
- Look for both confirmatory and contradictory evidence – too often we start by looking for confirmatory evidence, but it is advisable to spend at least as long searching for contradictory evidence.
In short, while there usually is some evidence – often low quality – that supports almost any course of action, we need to be more critical consumers of research evidence.