UK degree algorithms: the nuts and bolts of grade inflation

Concerns about UK grade inflation are now such a regular feature in the media that most of us are probably in the terminal stages of grade inflation fatigue. It is also the case that the term itself is interpreted in two different ways leading to two (intertwined) debates where both require a different policy response.

To this end, it might help to distinguish between grade inflation and classification inflation. In the process we can also look at one issue not being debated, namely how fairly (or otherwise) UK universities calculate their degree classifications.

Mark or grade inflation

A US student would call it their “grade” for a module; but the UK student uses the term “mark”. Fortunately, the interpretation of inflation is near universal. Thus, grade/mark inflation is the tendency of academic grades/marks for work of comparable quality to increase over time. However, in the UK what is being reported as “grade inflation” cannot be interpreted as mark inflation because we are not talking about changes in marks. The reported issue is around degree classifications awarded to students.

Defining classification inflation

Classification inflation, on the other hand is, the tendency for the number of higher degree classifications (1^st and 2:1 in the UK) to increase over time independent of any increases in the marks underpinning those classifications.

Much like the US Grade Point Average (GPA), the UK degree classification is not a “mark”: it is an average of marks. If a large number of UK universities change their degree algorithms, the proportion of 1^st and 2:1s will change independent of any change in the module marks. That is to say, there can be classification inflation without any grade/mark inflation.

Alternatively, if over time that average is calculated in a consistent manner across all UK HE institutions then any increase in the number of 1^st and 2:1s awarded could be taken as a reasonable proxy for mark inflation. In the US context, the widespread use of the GPA to summarise a student’s overall achievements generally meets this consistency criteria. When our US counterparts talk of grade inflation they are indeed referring to an inflation in the marks being awarded to students – marks that then inflate the calculated GPAs.

UK degree Algorithms

In the UK however there is no such consistency. The autonomy granted to UK universities means that for each university a 1^st or 2:1 is what they say it is. The result is significant differences in the degree algorithms used by UK universities. These generally differ in three ways:

• The number of years used in the calculation
• The weightings given to those years (or differential weighting)
• Whether lower marks are ignored (or discounted)

Bear in mind that UK students typically study 120 credits per year where the credits are evenly split between different modules For example, a student might study six 20 credit modules in each year. Therefore, those universities that that use discounting will only use the marks for the best 100 credits – or best 90 credits – in each year of study.

The table below lists the algorithms for 41 randomly selected universities. While the table cannot be taken as representative of the sector wide distribution it does illustrate this diversity in degree algorithms.

Source: Allen (2018)

We can immediately see the complexity of these different algorithms. In truth, most lecturers – let alone students – do not understand their own institutional degree algorithm. The vast majority of universities use only year two and three marks. The justification for the higher weighting on the last year is that it captures the student’s “exit velocity” or the standard that the student is performing at as they graduate from university.

The impact of this diversity in degree calculations

The impact on the classifications awarded is dramatic and best explained using a worked example. The marks for student X and Y are below, where both study on degrees comprised of 18 modules each one worth 20 credits (this makes the math a bit easier).

The right-hand panel shows the degree classification each student would receive from 5 different universities (A to F) each using a different algorithm (1, 3, 8, 9 and 14). The average of each year is shown along with the average of the best 100 credits (where the marks for modules 7 and 13 are discounted), these are the averages used by universities E and F (algorithms 9 and 14).

In this example, the degree marks and classifications are similar using algorithms 1, 3, and 9 – both students would achieve a low 2:1. However, had both students studied at university F student X would receive a 1^st, while student Y would receive a high 2:1.

The initial implication is that for both students the choice of university can play a big role in the degree classification they eventually achieve (all other things being equal). The ancillary concern is whether the 1st student X would receive from university F is representative of what most people think a 1st would ‘look like’.

But is the current problem one of classification inflation?

Concerns for equity apart (we’ll return to this below) if over the years a significant proportion of UK universities gradually changed their degree algorithms, for example moving from that used by University B to that of University F, then we could expect to see a rise in the number of 1^st and 2:1s (irrespective of any increases in the marks used by the algorithms).

And this gradual change in UK algorithms has indeed occurred. The Higher Education Academy (HEA) found that nearly half of the institutions it surveyed in 2015 (98 in total) changed their award algorithms in the previous five years so as not to disadvantage students in comparison to those in similar institutions. Reporting the HEA findings, HEFCE (2015) commented that those within the sector are rather comfortable with current approaches – adding that there is little evidence of an effective counter-narrative to regular media claims of “grade inflation” in undergraduate degrees. Some commentators are more forthright – Nick Hillman (director of HEPI) believes universities are essentially massaging the figures, changing their algorithms to improve the marks for borderline candidates.

Nevertheless, given that most universities have now changed their degree algorithms, classification inflation will probably become less of a problem in the immediate future, which leaves us with the issue of equity.

The potential for inequality

For UK universities who cherish their autonomy, the differences in the degree outcomes for student X and Y are irrelevant; the issue of fairness or comparability is not up for discussion.

Notwithstanding this, academics – including Woolf and Turner (1997), Curran and Volpe (2003), Yorke (et al) (2004), Yorke (et al) (2008) and more recently Allen (2018) – have examined the impact of this diversity in UK degree algorithms. The emerging view is that it is difficult to justify a system where the same marks will lead to a different outcome purely based on the student’s choice of university.

Methodologically, this research is straightforward: take the marks of an existing set of students and apply a range of algorithms to see what happens to the distribution of degree classifications. In the process, it is possible to estimate the proportion of students who would achieve a different classification. These proportions range from 15% to 38%, depending on the date of the analysis, the sample size, and the range of algorithms used.

So we must ask why some students could receive a different classification via a different algorithm – and others (the majority) will not. The answer relates to consistency in mark attainment, and how the different algorithms treat that consistency.

Discounting only the lowest marks accommodates those students whose module marks in any one year (for whatever reason) are not consistent in a negative way; however, the student with consistent module marks will not benefit from this discounting as their module marks are similar. Likewise, a higher weighting on year three marks will be of little consequence for the students whose year two and three marks are similar. In the above examples, student Ys marks could be described as consistent compared to student X’s marks, which is borne out by the difference in the calculated degree classification between university F and A (algorithms 1 and 14).

For student X, the potential of a 1^st from University F occurs solely because discounting has removed the lower marks in both years while the difference in weightings has exaggerated the impact of the year three marks. This is is a perverse outcome. While all university lecturers would welcome a student’s recovery in mark attainment (well done, student X!) it is likely they would not want to see this rewarded in preference to other students whose mark attainment is laudable for other reasons.

The policy response?

At one level the problem could be seen as one of a lack of standardisation in degree algorithms – as we’ve seen, the same set of marks can attract a different classification depending on which university a student attends. At another level, the problem is linked to the design choices within those algorithms – that discount and use differential weighting with the effect that they advantage one group of students over another.

The HEFCE (2015) consultation paper asked whether guidance should be developed on a sensible range of degree classification algorithms at the pass/fail and 2:1/2:2 borderlines. The inference is clear: the current arrangements with UK degree algorithms are not particularly sensible and a more standardised approach would be desirable for the majority of stakeholders.

In this respect, Andrew Wathey’s (2018) observation (as chair of UKSCQA) that a convergence in algorithms might be a solution to grade inflation, is a promising development. Nevertheless, if equity is a meaningful policy aspiration, any such standard algorithm should not involve discounting or differential weighting.

The response made by Universities UK (UUK) to the HEFCE consultation paper was blunt: UUK does not believe that it would be proportionate for the review to explore standardised degree algorithms. Instead, UUK would prefer to ensure that there are not patterns of gearing in degree algorithms that might be introducing grade inflation pressures. How this could be achieved, however, is not explained.

There is one simple solution to this potential (if not long running) impasse which also allows universities to retain their cherished autonomy. Universities can continue classifying their student’s achievements using the traditional classification, according to their preferred algorithm, but they would also be obliged to publish the student’s GPA score based on all the student’s marks across all years of study. The benefits are obvious and numerous and – to me – any suggestion that such a dual system might confuse people is disingenuous and patronising.