Three muddles that will undermine the TEF

The long awaited publication of the Government’s Green Paper Fulfilling Our Potential: Teaching Excellence, Social Mobility and Student Choice has finally provided a little more detail of the purposes and design of the proposed Teaching Excellence Framework (TEF).

In discussing the TEF, it is important to be clear that it is trying to do a number of important and worthwhile things by seeking to encourage high quality teaching and recognise outstanding teaching, provide useful information for students on the quality of their programmes, and recognise socially-inclusive institutions as well as the diversity of institutions across the sector.

There are also a number of strengths about the approach taken in Green Paper. In focusing on the quality of programmes the TEF avoids the tendency to overly focus on individual teachers which so often undermines attempts to support the development of high quality teaching. It recognises that such judgements involve aggregating a number of factors that cannot be captured by metrics alone. The TEF also attempts to take disciplinary differences in teaching and learning seriously by seeking to develop subject based measures of teaching quality.

However, there are three muddles in the Green Paper which are likely to undermine the TEF.

1. The selection of measures of quality is a technical issue

There is very little detail of which measures will be used to inform the TEF. We are told that a ‘technical consultation’ will take place in 2016, whilst the Office for National Statistics will review the robustness of potential data sources. The problem with this approach is that how we choose to measure quality will end up defining what quality is. This is because institutions will seek to improve their performance on these measures, which then become the embodiment of quality. Thus rather than being a technical issue the selection and use of measures will end playing a key role in shaping teaching across the sector. They need to be the subject of rigorous debate and discussion of the kinds of quality and institutional gaming that they are likely to encourage.

To give one example of why these debates are important, let’s look at the difference in the way that degree results and employment rates are treated as measures of the quality of student outcomes. There is huge resistance to using degree results because ‘everyone knows’ that, for example, a 2i from Oxford is not the same as a 2i from Chichester. This is despite having a higher education system that is intended to ensure that this is the case. In contrast, there is relatively little controversy in using employment rates to measure the quality of outcomes despite the fact that we know that many employers will only recruit students from ‘prestigious’ universities. Thus when quality measures wrongly reinforce the position of the powerful and the privileged this tends to be seen as unproblematic but the moment they might benefit the less privileged they tend to get summarily dismissed as not fit for purpose. Thus it is no surprise that the comparison of employment rates is included as a proposed common metric for the TEF, whereas the comparison of degree outcomes is not even mentioned even when discussing measures of student outcomes and learning gain. It is, however, deeply troubling.

2. The aggregation of quality to the institutional level

Whilst the Green Paper recognises that quality is a function of particular programmes rather than institutions, for reasons of bureaucratic efficiency it proposes that these should be aggregated up to an institutional award. This will determine what level of tuition fees can be charged by an institution. This will mean that the fees charged to students will not reflect the quality of their particular course. So a student studying a much lower quality Economics degree in Institution X could end up paying much higher fees than students on a much higher quality Economics degree in Institution Y. This looks more than a little messy and inequitable.

3. Four levels of TEF which do not exceed real term increases

The idea of the different TEF levels is that they will allow institutions to charge different levels of fees. This is to encourage a focus on teaching quality. The Green Paper mentions four levels of TEF, which will each be set a different maximum fee cap. However, the only increases in fees mentioned in the Green Paper are those that will not exceed real term increases. It is very difficult to understand how the four levels of TEF will be differentiated enough to act as an incentive without significant increases in fees in real terms. The only sense I can make of this is that once the Government has the right to increase fees without recourse to Parliament then the commitment to stick to real term increases will be argued to only apply once the maximum fees for the four levels of the TEF have been set. Thus those at Level 4 will be able to charge much higher fees. This speculation is informed by the almost complete lack of fuss from Oxford and Cambridge when it was announced that they would lose their special funding for their tutorial systems. In the past this prospect has led to all kinds of threats such as going private. The prospect of much higher TEF Level 4 fees is the only plausible explanation of this change in position that I can come up with.

If some institutions can charge much higher fees then it will mean that the issues around the selection of quality measures and the aggregation of quality to the institutional level become even more acute, with the latter likely to lead to student demonstrations and potential legal action. If it is not the case, then it is difficult to understand why institutions who have healthy levels of student recruitment would engage in the TEF particularly as they will be expected to carry the cost of their involvement in the process. Ironically the ways in which institutional prestige distorts perceptions of quality in higher education means that unless the self-proclaimed leading universities take the TEF seriously then it is unlikely to be seen as a rigorous measure of teaching quality.