I expected the Green Paper to use existing research to identify specific problems with university teaching and learning and then make recommendations addressing those problems. But the Teaching Excellence Framework (TEF) is only a little bit about teaching and not at all about learning. Its weak understanding of the object to be measured, its reliance on substitutes for it, its push toward job-based teaching, and its favouring of new providers will degrade rather than improve university learning—if the U.S. experience is any guide.
The trouble in the Green Paper begins with its nebulous core concept: “There is no one broadly accepted definition of ‘teaching excellence’. In practice it has many interpretations and there are likely to be different ways of measuring it” (A.1.15). What follows is a set of abstract principles (“excellence is the sum of many factors”) that offer little help, and are more pluralistic than they are reductive. This would not be a problem were the TEF to argue that teaching excellence must be defined by its practitioners—the teachers and students involved in subjective, context-specific, relationship-based cognitive efforts—and to work on how better to support it for the UK’s highly diverse student body.
Instead, it proposes a regulatory apparatus in which the measurement of teaching excellence is to be universal (A.1.16-18), quantified in “levels of TEF” (A.1.23), inclusive of various exceptions for students of different backgrounds, and yet standardised by a national assessment apparatus that will have to rely on proxies rather than on “direct measures of quality and learning gain” (A.3.13). The Green Paper evades technical questions of how a valid assessment can standardise while reflecting diversity, and reconcile these through the use of proxies for learning rather than through the definition and observation of learning itself.
When the Green Paper does become more concrete about teaching, it falls back on boilerplate: “TEF should reward and encourage teaching practices that provide an appropriate level of contact and stimulation, encourage student effort, and are effective in developing their knowledge, skills, and career readiness” (A.3.7). True, but what are such practices? They mean having “a strategic and effective approach” to understanding student engagement, having courses that are “effective in developing all students’ knowledge and skills” and so on. These are beginner’s comments, and are also circular. A TEF based on this low level of understanding cannot add any value to university teaching as it currently exists.
The paper’s premise is that UK university teaching is deficient. So where does it need improvement? The document relies in particular on the HEPI-HEA Student Academic Experience Survey 2014, which doesn’t make much of a case for teaching reform: general satisfaction is very high, 126 HEIs fall within a band of 10 points (90-80), and 96% of universities reviewed would have met the required criteria for TEF level one. Within the framework of student-as-consumer, the UK student-consumer is generally happy, which from the outset gives the TEF the feeling of a solution in search of a problem.
The deeper point is that the HEPI-HEA instrument is a customer satisfaction survey and not an analysis of student learning. It is a classic proxy that measures one thing—satisfaction—and not the other—learning gain. In addition, the nature of learning poses a problem for consumer response. Sustained learning, to oversimplify, requires repeated effort, particularly the effort of relearning something one has partially forgotten, and the effort of relearning makes people feel stupid, since they mistake it for having not learned in the first place. On the other hand, repeating what one has already learned makes people feel smart, even though learning has by that point ceased. Student learning will thus to some extent correlate with student dis-satisfaction. Although students are well-informed about their studies and their views must be taken seriously, their learning needs to be studied in itself, which the Green Paper does not propose to do.
Learning from the US debate
In 2011, the US media began to pay attention to books that hammered American universities for offering “limited learning,” in parallel to the UK government’s White Paper, Students at the Heart of the System, which also questioned teaching quality. But the US evidentiary base was different. The White Paper relied on the same student satisfaction data that the Green Paper does, while the best-known of the US critiques, Academically Adrift, used a qualitative skills test called the Collegiate Learning Assessment to quantify learning gains over several years in university. The headline result from the study was that a third of US university students learn next to nothing in four years, and that on average students now learn half as much as their parents’ generation during the 1980s. The authors’ methodology and conclusions have been critiqued, but they changed the public debate by rescuing a focus on learning gain from the morass of completion rates, rankings and reputations.
The study also found that in order to learn well, students do some fairly obvious things. They have to take many demanding courses, defined as those that require 20 or more pages of writing per term and more than 40 pages of reading per week. They have to spend much more time studying (say double the 12-13 hours per week US average, which is just below the UK’s). And they have to work directly with faculty members who have high expectations. Reversing limited learning is partly a question of refining the profession’s gold standard for instruction, but to a much greater extent is a matter of supporting the existing standard across the whole university sector.
How would that be accomplished? Not by adding more job-oriented courses. The US university system already abounds in practical majors like business economics, sports management, communication, marketing and medical technology. US students have migrated to these over several decades not only from the arts and humanities, but from the science disciplines too. When Academically Adrift predicted learning gains by overall discipline, they found that these vocational courses produced lower learning than did the liberal arts and sciences courses (e.g. Figure 6 here). In other words, a good way for a student to lower their learning was to shift from an academic to a vocational major. And a good way for a country to lower its overall abilities is to push its entire university system towards job-oriented courses.
The UK student is on to this issue: they favour learning conditions typical of liberal arts institutions, such as seminars and tutorials rather than large lectures (Figure 11), and individualised feedback. The American university system could probably close most of the gap between good 1980s learning and limited 2000s learning by moving all vocational students to liberal arts and sciences majors — or by raising all vocational courses to the more rigorous standard of the liberal arts and sciences. Something similar could be said of the UK, yet the Green Paper goes in the opposite direction. It is most excited about partnerships between HEIs and business, and favours new pragmatic providers that, if they resemble their US cousins, will offer near-term job training of such limited value that it will collapse without government-guaranteed student loan revenues. Exposure of quality problems intrinsic to the for-profit HE business model has shrunk the sector dramatically.
In the US case, these poor results are related to poor working conditions for most university instructors, who are now hired on unprotected short-term contracts and have no academic freedom. We already know what the conversion of professors to at-will employees looks like, and many have exposed the subprime tendencies of the UK analog. It is notable that the only social group not selected for nurture in the Green Paper is the university professoriate.
The remarkable audit burden on and public portrayal of UK academics as a coddled class has endangered their ability to generate the endless amounts of vital lucidity that their students need. My university system long ago established that the typical professor works about 50% more than a 40 hour week. Governments will eventually need to replace suspicion and surveillance, however politically advantageous, with trust in academic staff’s professional standards and motivations, or the workforce will not be sustainable.
Unfortunately, the Green Paper isn’t so much trying to improve teaching as it is holding universities responsible for raising UK productivity. It equates higher productivity with a better skills match, which it associates with more job-oriented courses. But not only do rates of productivity increase depending on an ensemble of socio-cultural and institutional factors, of which HE credentials are only (an important) one: the higher-order intellectual skills needed by advanced economies appear damaged by a systematic jobs orientation.
The larger story for Western universities is that massified higher learning has reached the end of the line. Standardised skills have been commodified and their earning power is weaker than ever. Our societies need creative capabilities from graduates on a mass scale. It’s hard to see how else the rising generation will address unprecedented problems in every domain, social and cultural as much as technological. We need graduates who can think independently, identify new problems, create customised research methods, interpret mixtures of data from different disciplines, identify a spectrum of solutions, and cope continuously with research failure, ambiguous information, and tacit knowledge. Creative graduates, in short, need individualised learning. And yet the UK government appears to be enforcing standardisation and pushing resources towards for-profit providers who can earn good returns only by limiting learning.
The Green Paper, in proposing outdated metrics, is a step backwards for student learning. Better learning requires investment, not assessment, but that seems to be the one thing not on offer.
