The Christmas edition of the Economist contained an article called ‘Animal Factory: the evolution of a scientific meaning’. It was about the difficulties of conducting experiments on laboratory mice. I learnt that ‘Not all mice are equal, even if their genomes are.’ Two sets of littermates that have been raised apart will respond differently to the same experiment. Some laboratories only do experiments on male mice, but what works for the gentlemen does not always work for the ladies. In fact, what works for males in single sex groups, sometimes fails to work for males in mixed sex groups. Why is it so difficult to reproduce research findings? In part, because many journal articles omit ‘crucial details’ about the way in which experiments have been conducted. Carelessness explains some of these omissions. Sometimes, scientists fail to include information about conditions that are known to be significant. At other times, information is omitted because of an assumption that it is irrelevant. It is becoming increasingly clear that it isn’t, not least because attempts to reproduce research carried out by one laboratory in another often fail. This occurs even when all the scientists are using the same ‘reliably uniform’ mice from the same source – the Jackson Laboratory in Bar Harbor, Maine, one the world’s biggest suppliers of laboratory mice. If the mice do not get near identical treatment in their different labs, the results from the experiments will not be the same. In the past, it has been assumed that the precise details of how mice were fed and cared for was irrelevant to the outcome of an experiment. This is now increasingly being shown to be a misplaced assumption. For example, a 2014 study ‘showed that mice in pain … experienced extreme levels of stress if the researchers handling them were men, but not if they were women, a difference no-one had thought to look for, or report.’ Furthermore, there was a heated debate in the scientific community last year about the impact on studies of the ‘lab mouse’s microbiome, the bacteria that live in and on it.’
Why am I telling you about the difficulties with conducting and replicating experiments on laboratory mice? Because the first thing I thought when reading the article was, if it’s this complicated with mice how much more complicated it is with children? The second thing I thought was that this suggests that we need to have a lot of information about the circumstances in which educational research is being conducted, information which we would not always think to request.
I teach four Year 7 classes this year. All four groups are at the same school, they all contain a similar ability range, all are following the same curriculum, and all are taught by me using the same lesson plans but, as we all know, a wide variety of other things can have an impact on how well the individual lessons go. Is the lesson in the morning or the afternoon? (I teach one of my groups on a Monday and another last thing on a Friday.) What lesson and teacher have the pupils just come from? Who are the dominant personalities in the class? Are they broadly on side or do they wish they were elsewhere? What is the weather like? (A class that have been cooped up inside all day because of the rain will always be a little restless, and I find that windy days seem to send classes a bit bonkers but that might just be me.) The behaviour of individual pupils varies depending on what mood they’re in, how many doughnuts they ate at break time, what another pupil or teacher has just said to them, and what’s going on at home. And I don’t want to imply that I’m the one constant in the room because, however, hard I try, I am not perfectly consistent. The last interaction I had with a colleague, pupil or parent can make a difference to how I feel and probably affects the person I am in the classroom, however much I don’t want it to. I always try to teach as well as I can, but it is likely that my performance on a Friday afternoon compares unfavourably with my abilities on a Monday morning.
The important point is not that conducting educational research is fraught or that drawing conclusions about the effectiveness of different teaching methods is impossible. The fact that we can’t exclusively compare classes of identical twins at identical schools isn’t a reason to reject research. Conducting educational research is hard but the problems that come with it are well known, and I am sure that the best researchers work hard to control for them. Without research we would have no idea at all what works, and that is something on which we definitely need evidence. Once a teaching method has been tried, tested and proven we should be implementing it at every opportunity. The fact that we may implement it better at certain times and with certain classes than others is not a fault of the method itself. The important point is that it would be both interesting and helpful to know a lot more about the situations in which educational research has been conducted because children, like mice, are surely affected by all aspects of their environment. Has a particular method been shown to be effective within schools? If so, it isn’t enough to know if the schools in which the research was done were comprehensive or grammar, single sex or co-ed, in the west country or in London. We need to be asking a wider range of questions. Here are just three suggestions:
- What is the ethos of the school and how is this communicated and enforced?
- What are the school’s disciplinary systems?
- How is the school day structured?
We need to know not just what works in education but in exactly what circumstances it has worked. That will give us the best possible chance of implementing successful solutions that will benefit all our pupils.