What’s the point of LEO in 2020?
Longitudinal Educational Outcomes data was always of questionable use to prospective students and policymakers. It makes for some corking headline findings – cynically, the main point of the release – but median values and a large time lag mean we need to be careful.
But, whatever I’ve said on Wonkhe over the years – which is a lot – LEO has become a regular release. We’ve seen new approaches, and benefited from a time series allowing us to understand how the graduate job market is changing. This latest tranche adds to that understanding.
It’s this time series aspect that presents the problem. I know from conversations with the team at the Institute for Fiscal Studies that accounting for the impact of the 2008 financial crash in the years afterwards makes working with salary data that stretches across that period a challenge. 2020 will be a similarly abnormal year for the graduate job market, and the longer term impact can only be speculated at.
You’re so bad
I always remember Sam Gyimah conflating “course” and “subject” at a press briefing on one version of these releases. It’s an easy (if frustratingly common) mistake to make, and you’ll look in vain at this release for the suggestion that data at a top subject level may well refer to courses and departments that have since closed or reconfigured.
I’ve worried in the past about the use that applicants are expected to do with details of what graduates in a related subject have done in the past. It is interesting, certainly, but the level of fidelity and statistical significance is so low that the only thing you can safely conclude is that most graduates get a decent job, eventually. Which is good to know, but hardly a distinguishing factor between courses to apply for.
I won’t back down
From previous releases we know that there are five aspects that have a significant impact on graduate salaries.
- Subject of study
- Provider
- Prior attainment
- Sex
- Where the graduate lives
For any meaningful historic comparison, four of these need to be controlled for while the fifth is shifted. Here we can control subject, provider, and sex. There is some information on where graduates live – the main release has a weighted median salary value, and there’s more (of slightly questionable value) deeper in the data. There’s nothing here on prior attainment, though we do get some data on POLAR it’s very much indicative rather than descriptive (and isn’t linked to individual graduates).
Of course, in setting my Kier Starmer-esque five tests I know that LEO can never do all of this without looking at sub-samples so small as to be meaningless. I’ve included the number of graduates involved in all the visualisations below – you need to be careful about drawing conclusions from small numbers.
Into the great wide open
Before we get to the usual tables and rankings, I thought I’d share something useful for understanding graduate destinations in a more literal sense. By selecting your institution on the filter at the top, you can see where graduates are living (and possibly working) three years after graduation, for four cohorts (the tax year slider on the right). The tooltip (run the mouse over the number on the map) will show you work/study status and median salary – though remember what I said above about low numbers.
There’s rankings of the number of graduates from each provider living in each region on the other tab.
Even the losers
You get a choice of subject area via the filter at the top, and can choose sex, and years after graduation (YAG) on the right, with the usual region and group filters and a provider name highlighter on the left. This data refers to the 2017-18 (latest) tax year only, is for all areas of residence, and is not weighted for the region of graduate residence.
The weighting is the hardest part
Available only for combined male and female figures, we also get a weighted salary that takes into account region of residence. You can look at the upper and lower quartile alongside the median for 1,3, and 5 years after graduation. Here I’ve coloured the unweighted numbers with dark blue outlines, with the teal representing the weighted figures. You can see where the unweighted figures over- or under-state the differences by provider (there’s no split by subject available) – this should give a reasonable person pause before making any grand statements about the main subject/provider graph.
Freefalling?
Given the coming Covid-19 issues with this time series, and given that for the first time the LEO data has been released without accompanying press releases and ministerial grandstanding, you could be forgiven for thinking interest in this fascinating but flawed dataset is waning.
But the opposite is true. The “further guidance” issued about student number controls and additional student numbers in England suggests that:
“the government has considered factors including subjects which relate to skills or professions at risk of shortage in the economy, or that generate positive economic returns for the individual and the taxpayer.”
This, to me, refers to IFS research that draws on LEO – the first time that government policy (rather than reports like Augar or nonsense like TEF “contextual data”…) has made use of LEO-derived information. I fear it will not be the last time.
This takes reporting to A Higher Place
Something odd happens when you pick Female + male on the second visualisation – median salaries for an institution jump up to £200k and more! Looking at Female or Male in isolation shows more credible median salaries, in the £25k range.
…this is when looking at All subjects rather than picking a subject.
Yeah, All works strangely in that visualisation. Use the one below.