The National Union of Students call for a boycott of the 2017 National Student Survey is finding support on many campuses across the UK. The argument goes that once the NSS is used as a metric to inform the Teaching Excellence Framework (TEF) – as is now happening – it becomes implicated in the commercialisation of education. Hence by boycotting the NSS, students might undermine the TEF and derail the connected fee rises.
As a long-time lover of the NSS, I share the frustration that it should be monetised by TEF in such a crass manner. Yet the boycott is based on wonky (not Wonkhe!) logic, and will only hurt those that NUS represents. Students, please, don’t boycott the NSS.
When ‘fee rises’ aren’t fee rises
TEF, according to NUS, is a vehicle designed to increase fees. But really it’s the opposite: TEF is designed to suppress fees. Inflation means that the £9000 that a student was paying four years ago is not the same as £9000 today. Hence the government’s decision in 2010 that £9000 was a fair price for 2012 entrants leads logically – or should have done – to a principle of rises in line with inflation. Did anyone seriously believe that £9000 would remain the maximum price indefinitely?
The only reason we are where we are, in fact, is that the government in 2010 neglected to future-proof the system. That was stupid, but it was also politics. Today’s political fix is not a rise, nor indeed is it a commitment to maintaining funding levels in real terms. In fact, because only some institutions will be allowed to increase their fees in line with inflation, it’s a commitment to reducing net funding for universities. Going forward, those that receive ‘silver’ or ‘bronze’ ratings in the TEF will experience real-terms cuts since their students will pay less, in real terms, than those of previous years. Students and staff alike really should be angry about this assault on universities’ financial health.
Some people will argue that £9000 was not a fair price in the first place. Others see the TEF as a kind of stalking-horse, that will open the door to more fundamental deregulation of fees. Yes, very many of us regret the state’s withdrawal of public funds for higher education. But the fact remains that, given where we are and the system in which we’re working, what look like fee rises are really nothing of the sort.
Students can’t break the TEF; they can only make it worse
I don’t much like the TEF. I think it’s unnecessary because standards are already high, because it’s an instance of government fussing about stuff it doesn’t much understand, and because the infantile ‘gold’, ‘silver’ and ‘bronze’ tags will make us all look ridiculous. I also rather suspect it might collapse, or at least metamorphose into something quite different, under the weight of its contradictions.
But a boycott of the NSS will only make the TEF worse. Participation rates may drop in response to the NUS campaign, but not enough to trouble anyone at a senior level. TEF Chair Chris Husbands has already signalled the direction of travel by stating that panels will “not be overweighting the NSS”. Just about all the TEF data is already wobbly one way or another; this is a low-expectation environment. The data will also be less valuable, because some of the most politically engaged students will withhold their opinions. But probably the only people who will notice the difference will be those of us at department level who really, deeply care about the NSS because we profoundly value our students’ opinions. A boycott will hurt us, but the TEF will roll on regardless.
What’s so smart about saying nothing?
Plenty of people have argued that the TEF is not really assessing teaching quality at all. That’s fair enough at a theoretical level, and we can all see that satisfaction and graduate outcomes are not precise measures of teaching quality. But it’s nonetheless ridiculous to argue – as does NUS – that the NSS does not “have anything to do with teaching quality”.
In actual fact the NSS is a pretty good proxy for measuring quality. Moreover it has been the greatest agent of educational reform that I have known. Poor NSS results can be a catalyst for major reforms within academic departments, and even within those department getting quite good results, the NSS helps academics to rethink aspects of how we work.
I could give countless examples, but here are a few. The NSS has put contact hours firmly on the agenda across the sector. Ditto schedules for the return of feedback on assignments, and equally the form and quality of feedback. Now, thanks to the new questions on student engagement, we’re all thinking about the culture and communities within which our students are learning.
Of course universities use good NSS results for promotional purposes. But why does this become, for NUS, such a terrible thing? Good NSS results are the result of hard, successful work. There are still departments out there getting crap results, and boycotting the NSS will only give them an excuse to hide for another year.
So please, please let’s not boycott the NSS. There’s plenty to be angry about TEF, but I can’t believe we’ve reached a point at which saying nothing makes political sense. In fact this campaign feels to me like an insult to students who have waited for three years to have their say. I think those students are smarter than NUS’s campaign.
This article is republished from Andrew McRae’s Head of Department blog.
So much to agree with in this piece from my old colleague, Andrew. Save for the comment that “very many of us regret the state’s withdrawal of public funds for higher education.” The funds haven’t been withdrawn surely. The Government simply invented another way to fund a much expanded student population via so-called ‘loans’, at least 50% of which will have to be written off. As in so many other areas of public governance, our Leaders decided to kick the can down the road, and give the responsibility of tidying up the mess to future generations. How long it will take before Hammond decides to cut and run, selling the loan book at a huge discount to the boys in the City?
Couldn’t agree more Andrew.
Boycotting surveys that feed into the TEF will not stop TEF.
It will just make TEF itself less reliable and harm the large number of people who do good work that benefits students outside TEF with that data.
There is a strsnge perversion in the NUS boycotting something which helps inform the choice of future students, holds institutions to account and is immensely effective in driving positive change where needed within insitutions to better students’ academic experience at unvieristy. One might go further and suggest that as such, the NSS has been tremendously effective in giving students a real and powerful voice in a way that the NUS have rarely done.
“In actual fact the NSS is a pretty good proxy for measuring quality.” no-one with any undertstanding of opinon surveys or statisical methods would agree with this at all (actually, it does make sense as a senctence).
Where is the evidence that the NSS is statisitcally and methodogically robust enough to be able to ‘drive positive change’ or acheive ‘a better students academic experience’? Has everyone forgotten the McNamara falicy, or bad data in = bullshit out. If not the McNamara fallicy, then perhpas those that blindly think the NSS works should look up ‘Mid Staffs Hospital crisis’ and he use of patient survey data.
More specifially, we have examples of institutions that use NSS for bullying departments and don’t adhere to the NSS’s own stanards of acceptable sample size.
So, I would have sympathy with the article if the NSS was actually methodologially rigorous enough to be a sound evidence base for student opinions on degrees, but it is just not. The idea that students should be asked about their degree experiences and that those responses – if methodlogical sound and useful evidence – should inform change is uncontraverisal. The idea that the NSS is a sound method to acheive this is nonsense.
For example, from the article we have “The NSS has put contact hours firmly on the agenda across the sector. Ditto schedules for the return of feedback on assignments, and equally the form and quality of feedback.” What evidence is there of the cause and effect between the above and the NSS? Did no-one care about this before the NSS?
If this peice were submitted by a student to examine the use of survey data and the causal relatonship ebtween that data and it producing useful results It would fail a first year undergradaute submission.
I’m largely in agreement with you here Andrew. I suspect the NSS boycott may perversely simply drive up the response rate (which would be embarrassing for those advocating boycott), but simultaneously undermine a really important mechanism for student feedback. A survey which has been the source for innumerable positive changes in the HE sector to help improve the student experience. It is sad that a blunt, and ill-guided boycott may actually hinder improvements to the student experience.
The NSS is by no means perfect, but as measures in the sector go, it drives more positive change than most.
One area I’d critique your argument is that TEF will help keep fees down. In theory you may be right, but government is already convinced that fees need to increase for the majority of the sector, and TEF will help to enable this. But crucially an NSS boycott is not going to stop this.
This is perhaps one of the most ill thought out campaigns the NUS has ever run. There is no measure that this boycott can succeed on;
– over 90% of constituent members (local SUs) aren’t even taking part
– response rates will most likely rise, not fall
– even if response rates do fall, them falling to under 50% (the minimum to publish results externally) is highly unlikely
– the margin of error with a 50% response rate (for institutions >8,500 students) will be <1%, so projecting results to rest of student population is easy
– results could just be ignored by govt and TEF anyway!
So, what's the point?
1/ Fees are going up. It’s a fee rise. The net effect may also be a reduction in real-term funding, but that doesn’t negate the fact that students will be asked to pay more – nor justify it.
2/ A major reason Husbands was compelled to make his statement about overweighting the NSS was because of the attention drawn to it by the boycott. The government started consultations with an extreme position and it has taken (and will continue to take) strong voices and clear arguments to ameliorate that.
3/ Boycotting the NSS does not prevent students from giving feedback in other ways, and this emphasis on the NSS devalues the input of everyone who isn’t a third year undergraduate. That is has taken the blunt instrument of the NSS to prompt departments to act raises serious questions about their wider commitment to student engagement.
4/ The NSS is fundamentally flawed at both theoretical and practical levels. As research has shown, to raise your NSS scores simply hire attractive white lecturers with a generous approach to marking. If you’re working hard to achieve good scores you’re missing the point – what matters is quality not metrics.
Andy Hartley:
“– the margin of error with a 50% response rate (for institutions >8,500 students) will be <1%, so projecting results to rest of student population is easy"
This is nonsense. And anyway, the response rate is by degree course (via HEFCE funding report) not entire institution. If you can be bothered you can download all the data from the survey, and marvel and the innacuracy of it (courses that do not exist being listed…courses mixed up…non existant degrees getting good scores and the institutions saying nothing about it).
As Charlie says, the only reason this is back in the news IS because of the boycott.
Anyone who thinks NSS = improvement for students and better courses should find 'Table 1' in this paper: http://online.liebertpub.com/doi/pdf/10.1089/ees.2016.0223
Actually the inclusion on NSS data is based on a 50% participation rate institutionally, and additionally at subject level.
There appears to be a contradiction in the argument which simultaneously claims that (a) the NSS boycott will be ineffective, and (b) the NSS boycott will have a negative impact. You can’t have both – either it’s ineffective, or it will cause disruption. Unless, of course, the argument is that the boycott will be ineffective because there aren’t enough people taking part. But that would be an argument in favour of expanding the boycott.
Now, it is precisely the point of the boycott to cause disruption. It aims to disrupt market mechanisms, to place the TEF and the HE Bill on the negotiating table – like industrial action by a trade union. To say that it will cause disruption is not an argument against the boycott.
And it does seem to be raising eyebrows in high places. Government announced that it would use three years of NSS data, rather than one, soon after NUS and UCU national conferences voted in favour of the boycott. And Husbands announced that the TEF panel would give less weight to the NSS shortly after the boycott got under way – once it was apparent that Students’ Unions up and down the country (including at Sheffield Hallam, his own university) were taking action, and not merely posturing.
On the question of the NSS itself: besides the fact that it is a poor quality survey which, like comparable surveys in other HE systems, systematically discriminates against BME and female teachers, it is also a very poor substitute for a flourishing campus democracy.
One last point. TEF was introduced as a counterbalance to the effects of the REF. Attempting to raise the profile of teaching in universities is a laudable goal. Doing this by introducing more sticks to beat individual academics with, however, is not the way forward. Both TEF and REF should be abolished. If we must have a TEF, however, let’s have it as a stick to beat university managers with. Give a Gold rating to universities which offer the most paid time for marking scripts and preparing classes.