Iain Mansfield’s controversial new research on grammar schools does not pull its punches.
It directs explicit criticism at the academic community which he accuses of having an “unconscious bias driving the [selective education] research agenda”. Yet it appears to be Mansfield’s own bias – presumably not unconscious – that leaves this research open to easy and extensive criticism.
Selective evidence
The report – The Impact of Selective Secondary Education on Progression to Higher Education – is selective in its citation of previous research on grammar schools, and stretches back over ten years to The Sutton Trust’s 2008 report to find supporting evidence for its hypothesis. More recent research that it conveniently disregards includes reports by Education Datalab (2018), Professor Stephen Gorard (2018) and Professor Simon Burgess (2017).
The report is also selective in the quotations it chooses. Despite highlighting polling data that “more than twice as many people support proposals to bring back grammar schools than opposed them”, the report ignores findings from the same poll that nearly two-thirds (of the same respondents) opposed the idea of selection and a warning that “views differed markedly between people according to their social and demographic characteristics”.
In addition, support for selective education was “significantly associated with being older, being male, voting Conservative, being more highly educated and living in certain regions.” Overall, it seems that Mansfield has fallen into exactly the trap that he accuses others of – “a failure to engage properly with evidence”.
FSM alternatives
This is also true of the methods that are used in the report’s primary data analysis. Previous research is criticised for having a narrow view of disadvantage by using free school meals (FSM) as a proxy. Whilst FSM is not a perfect analysis category, it ought to be acknowledged that this is the proxy used by all government departments, including DfE and Ofsted. There was an attempt, and even a consultation, to move beyond this to a new definition labelled ‘just about managing’ – but no reliable measure was found. Mansfield’s attempt at creating a new measure in order to make the data fit an argument is naive.
What about SEND?
The report also seeks to broaden the category of disadvantage by including specific disadvantaged groups, but again this is selective in its focus. While the analysis includes students with English as an additional language (EAL) and black and minority ethnic (BME) backgrounds, it does not include students with special educational needs and disabilities (SEND). Nearly a third of disadvantaged students have SEND, and this group are significantly under represented in grammar schools.
The percentage of SEND learners with statements or education, heath and care (EHC) plans is less than 0.04 per cent in grammar schools, compared to 1.7 per cent across all schools. A similar pattern exists for those SEND learners without a statement or an EHC plan, such as children with dyslexia, who are over 200 times less likely to attend a grammar school. This data, if included, could have significantly altered Mansfield’s conclusion that grammar schools provide a ladder of opportunity for disadvantaged pupils.
While academic debate is to be encouraged, and think tanks such as HEPI have an important role to play in this, sadly the only thing that we can say about selective education based on this publication is that it is selective in the evidence it offers.
It isn’t just that think tanks like HEPI have an important role to play in advising the debate over education policy, it is the fact that they have a DUTY to do so. Their utmost efforts have to be employed to make sure that any and all conclusions attached to their research are based only on the most robust, reliable and unbiased information at their disposal. Iain Mansfield has failed on all counts. What really puzzles me is why HPEI agreed to publish. My only thought is, they came to this decision in an attempt to expose its conclusions to review by peers. So far all the ‘evidence’ that I have unearthed from these said peers is damning of the conclusions drawn.
In my view it is time for HPEI to respond. Left unchallenged, reports like this have a profound effect on those education policy-makers (politicians) who direct education reform. Isn’t it already evident that these same ‘leaders’ have already done untold damage to generations of young people with their alleged evidence-based initiatives? In fact, too often, we find policy being made as a result of ‘cherry-picking’ that research which endorses already well-formed opinions. This resort to bias in policy-making decisions has to stop if we really want to evolve an education fit for our young people in the 21st Century.
Thanks John, You can accuse us of lots to things but you can’t (accurately) accuse us of not responding to the critiques. Iain, the author has done so on Wonkhe. I have done so for Schools Week. In addition, I seem to have spent most of the past week engaging with people about the report on Twitter (including with the author of this piece to which you are responding). We will have more to say in due course – perhaps even an event. But, for now, a few quick thoughts.
First, HEPI is a policy body and my frustration is that most of the responses, while interesting and powerful (though highly contestable, as Iain has pointed out on Wonkhe), have completely missed what I regard as the key policy questions raised by the issues in the report. These include: if selection is wrong at 11 but widely and (largely uncontroversially) practised at 18, when does it become wrong? (Jonathan Simons’s Wonkhe piece is a rare example of grappling with this.) The outstanding questions also include: if the evidence is so overwhelmingly against selection at 11, then why have the 163 remaining grammar schools proved so resistant to change? I am amazed that the firmest opponents of grammar schools who have taken against the report haven’t tried harder to grapple with this question. (One new response to the report from three academics at Bath, UCL and Bristol patronisingly puts the survival of grammar schools down to ‘Emotional arguments [that] might appeal to the masses’, which seems an odd way to engage the public and win them over.) Until we grapple with the question of why the schools have survived, it seems likely – to me, at least – that those who want grammar school provision to disappear will go one losing the debate about the remaining schools which they have already been losing for half a century.
Secondly, some of the responses seem to have been written by people who are angry that we haven’t published a report about entirely different things, typically things that are far beyond HEPI’s charitable objectives or remit. For example, Karen’s points about the low numbers of SEND children at grammar schools are very important, but nothing in the HEPI paper contradicts them and they are not so directly relevant to the main issue covered by the report, which is mainly on progression at 18 rather than progression at 11.
Thirdly, I am struck by the fact that the grammar schools report, which is – in truth – on a fairly limited educational question directly affecting only a limited number of people, has encouraged so much more debate than other, arguably, more important educational questions discussed by HEPI. I think for example of the report we published on comprehensive universities, which argues against our selective university admissions system (if comprehensive schools are right, why not comprehensive universities?), or even our recent blog on the huge inaccuracy in exam grades. These debates affect far more people and, if we really want to deliver a different sort of educational system, deserve to receive some of the energy that is currently being spent by people shouting across each other about grammar schools, in my view.
Thanks Nick for engaging and addressing some of the issues raised in response to the report last week. It’s certainly got people thinking! I’d like to just respond to a couple of things as you ask some important questions.
The question: “if selection is wrong at 11 but widely and (largely uncontroversially) practised at 18, when does it become wrong?” is a good one that I’ve thought about before but has largely been in the ‘must think more about this when I have a minute’ category! I still need to think about this further but my initial feeling is that the key difference is that at age 11 pupils have at least five more years in which they are required to be in education, therefore any system-wide effects of selection at 11 have an effect for everyone for almost half of the compulsory period of education. The evidence is pretty clear that there are negative externalities of the grammar system that impact on huge numbers of children (and disproportionately those from poorer backgrounds). As such, aside from any other arguments for or against selection at 11, it is an inefficient and inequitable set-up. At age 18, compulsory education has ended and young people are choosing the specialised training pathway that is best suited to their skills and interests and the system is looking to match students to providers. This is quite different to the common curriculum of general education that the state is to provide up to age 16. It is therefore perfectly reasonable to argue that selection is fine at 18 but not 11. The point at which it becomes OK is probably 16 as this is the end of the compulsory schooling period. As I say, would like to do more thinking on this.
The question: “if the evidence is so overwhelmingly against selection at 11, then why have the 163 remaining grammar schools proved so resistant to change?” is another good one. Perhaps though we could re-pose this: if grammar schools are such a good idea why have more than 1,000 closed since their heyday in the mid-1960s? It appears that it’s not the opponents of grammars that have been losing the argument for 50 years! For the record, our comment about “emotional arguments” that “appeal to the masses” was in response to Iain Mansfield’s emotive piece accusing academics such as myself and my co-authors of harming the opportunities for other people’s children. He made a case that grammars are a vital opportunity for the children of shop workers, warehouse supervisors and immigrants, and that grammar-opposing academics (whose children will allegedly do OK whatever) are therefore scuppering the chances for other people’s children. This argument is a direct pull on the emotions, as clearly nobody wants to harm opportunity for any child, and particularly those who start from a less advantaged position. Our response is that this is why we need to take a step back and analyse the big picture of what really harms opportunity for those from less advantaged backgrounds – and there the evidence is clear that selective education damages social mobility. That is not to patronise but to acknowledge that this is a complex area and requires rigorous analysis of the whole picture. While the story of the poor but bright child who goes on to great success through the benefits of a grammar school education is appealing to everyone, what needs to be understood is the extent to which this is an accurate picture of the typical experience of poorer children living in selective areas. Our evidence and that of others is that it is not, and in fact while there are these positive stories, the overall picture is that poorer children are adversely affected by the selective system. So academics opposed to selection are actually responding to evidence that for the majority of “other people’s children” the grammar school system will be harmful.
Why are grammar schools so popular in their areas and with the public is another important question and I don’t know the whole answer but part of it can be inferred from the above. The positive stories are easy to identify and celebrate, while the overall effects are not so easy to uncover and do not have such an appealing narrative and as such it is more difficult to convey the reality that overall selection harms the worse off.
I totally agree that HEPI raises a lot of important questions and think there should be more traction for other issues you raise such as questions around comprehensivisation of universities and exam grade inaccuracies. These things need to be highlighted, discussed and debated so I’m glad HEPI is raising them!
Thanks Matt. I am mulling all this over. I have been told that it is unnecessary and unhelpful for this debate over grammar school provision to be happening. But I don’t believe that to be the case. Debating such issues helps shapes opinions, especially when there is judicious use of data, evidence and research. So I welcome your thoughts wholeheartedly.
I am not completely convinced by your (tentative) responses to my questions though. The few people, including you, who have grappled with the question of when selection becomes acceptable have alighted on the age of 16, either because that is when compulsory full-time education ends and / or because the curriculum changes then. In other words, you seem to be saying selection at 11 is wrong because we have a rich evidence base to show its effects, and then move on to saying selection at 16 is okay simply because the education system is set up like that. One of these opinions seems very much more evidence based than the other.
Moreover, what happens whenever the end of compulsory education changes? Should we suddenly decide, overnight, that the the point at which selection becomes a positive thing changes too? If so, aren’t we at risk putting policy before evidence rather than setting policy on the basis of the evidence?
I remain surprised that some of the fiercest opponents of selection at 11 are so relaxed about it at 16. After all, if you do select some people at 11 (as we do) and if this puts some people on sub-optimal education tracks for them (as many believe) and if we then put the same people through another selection process at 16 (or 18), won’t it just compound the problem of sub-optimal choices rather than solve it? This is not just a point about grammar schools: other forms of selection (eg by house price) exist too. Isn’t the way to fix mistakes made at 11 or any other age to broaden choice out later on rather than to accept another selection process? And why do you assume relatively unquestioningly that the evidence for selection at 11 is unlikely to hold at later ages?
I take your point that some evidence suggests that overall outcomes lessen when selection is practised and I accept it. I am not particularly pro-grammars. But, equally, closing grammar schools is not the only possible response to the problem you have identified. Another would be to focus on the schools that under-perform in grammar school systems. I am not advocating that, I am not saying it is easy and I am not even saying it is my preference. I am merely pointing out that there is a big jump in the assumptions of those who oppose grammar schools: they move from saying the current data on grammar schools show them to be good for some children but less good overall to then saying therefore they must be closed. Perhaps it is so, but the other options need debating and considering too. After all, there are some brilliant comprehensive schools cheek-by-jowl with some of our grammar schools. I am not at all certain it is inevitable that other schools in selective systems must deliver so poor outcomes (even if the current evidence says it is often currently so).
In terms of the resilience of the remaining 163 grammar schools, I agree one can look at this issue through either end of the telescope. However, didn’t the mass closures take place before the political shift towards the right that occurred in 1979? So we have had nearly 40 years since the wider closures happened, which suggests to me the grammar schools that have survived are not so easily got rid of. My own academic research is on the tenacity of the independent school sector in maintaining their status – in the 1940s and 1960s, there were serious attempts to absorb independent schools into the state-financed school sector. The barriers against this were very big and very numerous. The story of the survival of these schools explains, in part, why I think it is so much harder than many of the opponents of grammar schools think to change the status of long-standing schools with impressive track records that enjoy strong support from parents and communities.