A more refined employment metric

Employment metrics are forming an increasingly important part of league tables and the Teaching Excellence and Student Outcomes Framework (TEF), and understandably so.

Current and prospective students have a huge stake in their education, and one of the key factors in evaluating whether or not their investment is worthwhile is what kind of employment outcomes they are likely to achieve. But in order for the metrics to be helpful, they need to be reliable.

Imagine you were considering studying Law at university because you held a lifelong ambition to join the legal profession (not everyone who studies Law wants to join the legal profession, but for the purposes of this example let’s imagine you do). You’d think that the employment metrics would be helpful in informing your decision about which university would be the best to study at. But a recent “refinement” to the employment metrics by the Office for Students (OfS) means that, for the subject-level TEF Year Four pilot at least, you’d be wrong.

The TEF Guide to subject-level pilot data, published by OfS in October contained the following, rather technical, information relating to professional qualifications (paragraph 55):

Job titles and descriptions of duties are coded into the Standard Occupational Classification (SOC). Highly skilled employment is categorised as those jobs matched to SOC groups 1-3 (managerial and professional).

So far so good

All good so far. However, it continues…

Higher study is categorised as study that is reported by the student to be a qualification that is at a level higher than the one that they have recently obtained before completing the DLHE survey. This is a refined version of the metric used in TEF Years Two and Three, which considered outcomes in highly skilled employment or any further study.

So, given that the metrics require the follow-on qualification to be “higher” than the student’s previous one means that if our aspiring lawyer wants to pursue the obvious next professional step, this would be classed as a negative outcome in the TEF. A Masters would be a positive outcome, as would a PGCE – but anything that’s not regarded as a step up (including studying Medicine after a BSc in Biosciences) wouldn’t be.

Half an eye on the metrics

Careers services at universities spend huge amounts of time and effort supporting students to figure out what they want to do, helping them develop relevant skills and experience, and enabling them to be successful in selection processes. They also have – more than – half an eye on the employment metrics.

However, there’s sometimes a tension between helping students get to where they want to be when they graduate, and achieving as many positive destinations as possible. That’s because what a student/graduate wants to do (eg. “I want to get into TV production”) won’t necessarily be a positive destination as measured by the DLHE at the six month census point, or indeed the Graduate Outcomes survey at the 15 month census point (are you sure you don’t want to become an accountant?).

Contradiction

The recent refinement, however, makes this not just a tension, but in the case of our student wanting to pursue a career in the legal profession, a direct contradiction. This is because none of the professional qualifications required for you to join the legal profession as a lawyer (Legal Practice Course), a barrister (Bar Professional Training Course), or to convert to Law from another subject (Graduate Diploma in Law) are higher than the one that they have recently obtained before completing the DLHE survey.

This has two significant implications, one for universities and another for prospective students. For universities (in our example, Law Schools in particular), those who have a high number of students going on to professional qualifications that are classified at the same level or lower than an undergraduate degree will be penalised; their subject-level TEF ratings will plummet. For prospective students, they are at risk of being misled if they draw the understandable conclusion that a particular subject isn’t up to much because it’s adrift from its TEF highly skilled employment benchmark, little knowing that this could be as a direct result of a subject producing lots of professionally qualified graduates.

Blunt instrument

I assume this refinement has been introduced to keep in check the number of universities (I think a small number) who have made strategic use of GradCert qualifications in order to artificially inflate their employment metrics – something that does need addressing. However, the refinement is a very blunt instrument that has huge, presumably unintended consequences, for those universities that offer (or encourage their students to progress onto at another institution) professional qualifications that form part of a genuine career plan.

The one thing that encourages me in all this is that subject-level TEF is a work in progress; it’s still in pilot phase. Indeed the TEF is undergoing an Independent Review, the consultation for which closes 1 March. And the employment metrics at this current point in pilot phase are as follows:

highly skilled and higher study – currently derived from the DLHE;
sustained employment – derived from Longitudinal Education Outcomes (LEO) – salary data from HMRC linked to education data from DfE;
above median earnings – also from LEO.

The “highly skilled employment and higher study” metric is an important measure, but this recent refinement has significant drawbacks. Anyone concerned with the integrity of the data and its reliability in informing decisions should raise their concerns with the Independent Review. There’s a real danger that if the integrity of the highly skilled employment metric is undermined it could be dropped, and then we’ll be left only with the Longitudinal Education Outcomes. And that really would be a blunt instrument.