Visualising the national student survey 2019

It’s been great to see the Office for Students itself getting in to producing interactive graphs for this year’s National Student survey results. The approach in Nicholson House has been to highlight performance against benchmarks – I’ve taken some other routes.

First up, here’s the numbers that will end up in the league tables and press releases. I’ve long argued that NSS by institution only isn’t helpful for prospective students or others – you include so many different student experiences l that an average doesn’t offer much help for understanding how your experience may compare. Averaging out the experience of fine arts students and nursing students tells us very little.

NSS 2019 – whole institution

[Full screen]

My other mainstream NSS bugbear is the focus on “percentage agree” as the gold standard. A while back I took a look at “percentage disagree” – focusing on the ones and twos rather than the fours and fives. This approach has been included in all the visualisations presented here, and in future I think I’ll use it as a default. It takes a lot for a student to go for the lower end of the scale – for me this conveys more meaning about how annoyed students have been than the top end.

You can use the filters to toggle between “registered” (all students on a course run or validated by a provider) and “taught” (all students taught at a provider). I’ve also added the ability to choose which question to look at, or which range (new for 2019). The ranges are as follows:

NSS 2019 – by subject

[Full screen]

Here I’ve presented the 2019 results in the form that allows you to get as close to results for an individual course as the data allows – using the Common Aggregation Hierarchy Level 3 list of subject. As well as letting you look at “your” subject in “your” institution (or anybody elses), this also gives us the ability to look at average dissatisfaction by subject and level of study. It turns out polymer and textile students are not a happy lot.

NSS – time series

[Full screen]

The survey, of course, looks at a different cohort each year, but because we assume cohorts are broadly similar in their attitudes a time series can show changes in institutional performance over time – so here’s the results from 2015 to 2019.  The difficulty comes with the changes to the survey between the 2016 and 2017 release – some questions can be mapped across this gap, and others cannot. The default views on this visualisation look at the “overall satisfaction” question (old Q22 and new Q27), but using the filter you can look at any single question or multiple question (I suggest you chose one from the old questions and one from the new).

For the keen, here’s my best attempt at a mapping:

Other thoughts

We should always be aware of the limitations of every survey instrument – both in design (the NSS looks only at final year students, though there are issues faced by part-time students, students on placement years, and non-standard length/format courses that prevents full coverage) and use (the NSS has become increasingly politicised, and you don’t have to look so far to see evidence of incentives to raise the rate of return among engaged students). Looking at the NSS cannot tell us directly about the “quality” of HE provision, but it opens an important window on how it is experienced.

Many responses, therefore, could be said to deal with “hygiene” factors. Issues with access to academic feedback and resources are always a concern in this survey, 2019 is not an exception. And while we should welcome the small average rise in satisfaction (and even more so the small average fall in dissatisfaction, now at the lowest level since 2015) it is the fine grained analysis of the data that will benefit future cohorts.