Two new “boots on the ground” reports bring concerns

Two more assessors reports have emerged from Office for Students investigations into business provision – with both the University of Wolverhampton and Buckinghamshire New University being the subject of documented concerns.

These reports are the latest outputs from what is universally recognised as the “boots on the ground” initiative – kicking off in May 2022. The assessors findings, as detailed here, are presented separately from any regulatory action that may follow. Eight reports in and we have yet to get a handle on the consequences that may follow.

The concern at the University of Wolverhampton is around the vexed issue of assessment retakes. Some students on the recently revalidated BA (Hons) Business Management got to resit module assessments up to six times, others (who may have initially taken the module later in the year) did not benefit from the same number of opportunities. There are really two issues within this: equitable treatment of students, and the underlying academic quality issue of allowing a large number of resits.

Interestingly, the review started in examining reasons for a low continuation rate – something that you may imagine multiple resits would exist to address. The university had recently revalidated the course as part of a wider pedagogic refresh – bringing larger (30 credit, rather than 20 credit) modules and a focus on business practice, following general concerns over the quality and consistency of the student experience.

But back to those six resits – the academic regulations allow for two attempts at an assessment within an academic year, or three “where scheduling permits” (which in practice is linked to the point in the year at which an individual student starts their course). This includes things like the 8,000 word report for the “capstone” module in year three, with the assessment team considering this as evidence of a lack of consistency. Students are also able to resit a module if they fail it, allowing again for up to three attempts – around two per cent of all students managed to get to five or six tries.

At Buckinghamshire New University the assessment team identified five areas of concern:

• The university was not consistently providing a high quality academic experience, with particular concerns around the content and delivery of teaching.
• A lack of educational leadership
• A lack of consistency in the quality of materials on the virtual learning environment
• Student academic support needs were not consistently identified
• A variability in teaching skills in academic staff – with appropriately qualified staff not sufficient in number

It’s quite the charge sheet – easily the most troubling set of concerns raised in one of these visits.

Like many providers, Buckinghamshire New is expanding recruitment – with nearly 800 students added between 2020-21 and 2021-22. To complicate matters, we have a “specific international cohort” from a single country, all recruited outside of university processes and with less stringent than usual checks on prior attainment and English language skills. The university was also attempting to expand staff numbers to match – at the time of the first assessment visit, there were 12 teaching post vacancies in the School of Business and Law.

Alongside this, September 2023 saw the introduction of a redesigned curriculum, with improved access to placement learning and optional modules. So at the time of the assessment visit, the School was teaching two different versions of the curriculum (previous cohorts are being taught out on previous designs) with a reduced complement of staff supplemented by temporary lecturers.

If you work in a provider you can probably guess why the assessment team identified so many issues – this is very much a perfect storm. However, the regulator is interested in the experiences of students (rather than what we all know happens to a department in transition), and in this case it feels like students were getting something of a raw deal.

For instance, 78 per cent of that big and complex international student group failed their first assessment. You can split the difference between “there were students that maybe shouldn’t have been on the course” and “these students did not get the support they needed to succeed” – either way, it reflects badly on the university (and, indeed the Daily Mail, who had recently made Buckinghamshire New its “University of the Year for Student Support”).

Concerns were raised multiple times about low engagement (or indeed, failure to attend) leading to a failure to submit assessed work. Bucks does have an attendance monitoring scheme and a dedicated engagement and retention team – the report intimates that the latter focused on general issues and centralised support.

However, the impression that the assessment team were given was that the quality of provision may have been a factor in low engagement. It is a world of long, content-heavy (but, surprisingly, academic reference -light) lectures with limited opportunities for students to actively engage. The report is particularly scathing about a particular observed instance:

When the students were discussing the question set, as they had been instructed, the lecturer told the class to ‘keep it down’, which had a negative impact on discussions.

Which was followed by a lack of “challenging and clarifying facilitation” from the teacher in question.

Lecture focused pedagogy stands or falls on the quality of resources – and unfortunately the assessment team had concerns here too, with many instances where materials for sessions “lacked what they deemed to be adequate engagement with appropriate academic literature”. Though it is possible this information was supplemented during the sessions themselves, for a cohort already identified as disengaged (and likely with multiple responsibilities outside of their studies) the state of the materials is an issue.

There are plans in place (including the aforementioned curriculum redesign) to address some of these problems – indeed, some were already running at the time of the assessment visit. However, the assessment team felt that failings in educational leadership and academic governance was having a lasting impact on the academic experience of some students. A key role – that of School Director – had been inconsistently filled during the assessment period, and was in any case only a fractional (0.8 FTE) role. The team identified “no adequate plan” for covering these responsibilities while a director was not in post. There had been a pattern of poor retention among senior school staff, to the extent that an “internal review” of quality and compliance had recommended an increase in academic oversight of quality-related issues (including the high withdrawal rate, and the quality of teaching).

One interesting thing we spotted in the section on “missed opportunities” to identify academic support needs was that the university had killed off end-of-module student surveys, which collected anonymous student feedback – instead moving to monthly meetings with student (course) reps. It’s interesting because on the SUs study tour we led earlier this month, we tended to see the opposite – universal and in some cases compulsory module feedback (both on teaching and support and on students reflections of their own independent study) and a smaller group of more influential student reps influencing decisions where they’re made. Jim has a write up of what we found on the site.

Uniquely among these reports the assessment panel has made recommendations for improvement at Buckinghamshire New – though it is the Office for Students that will have the last word on any regulatory sanctions. The panel suggests

• A new monitoring system to ensure baseline expectations about the use of the virtual learning environment
• Consistent recording of attainment data, as an evidence base for teaching quality enhancement
• A temporary reduction in the size of new student intakes, at least until staff numbers catch up
• Robust processes for teaching observations and critical academic review of curriculum and resources – which should feed in to HR processes like probation, performance review, and promotion
• Senior academic staff providing additional input to ensure the role of Director of Education is adequately fulfilled.

Seasoned OfS watchers will note that third bullet point in particular.

You’ll be aware that at Wonkhe we are also interested in the (very sparsely documented) OfS process around these investigations.

Buckinghamshire New’s investigation started back in May 2022 along with the main group of investigations, whereas the decision to investigate Wolverhampton was taken in October 2022. So far every other investigation of a business school was taken back in May, suggesting that the monitoring and decision points are ongoing. Wolverhampton has also enjoyed what is so far the shortest investigation period – Buckinghamshire New by far the longest.

Provider responses

Gill Knight, Pro Vice-Chancellor Academic Leadership and Student Experience at the University of Wolverhampton, said:

We welcome the thorough findings of the Office for Students (OfS) assessment of our Business School course offer. Prior to, and since the assessment team visited in early 2023 we have been reviewing our academic regulations with a view to ensuring they remain both robust and supportive in delivering student outcomes. The review has brought about one recommendation for a change to our reassessment regulations that directly addresses the finding in this report, and this will be considered by our Academic Board this academic year. We continue to focus our efforts on providing a high-quality academic experience for our students with courses that are up to date and relevant.

Nick Braisby, Vice-Chancellor of Buckinghamshire New University, said:

BNU’s historic mission is to extend opportunities to those traditionally denied access to, and under-represented in, higher education. We are proud of our diverse student community and we work hard to understand, and support, the increased complexities and challenges that many of our students face. We will always place great emphasis on the importance of ongoing quality enhancement to ensure our students have the best experience possible.

A longer statement from the provider expressed surprise that OfS had not divulged the precise reasons for the assessment, noting BNU’s strong showing for business students in the 2023 National Student Survey, and the recently awarded TEF Silver. It also noted the internal review of business and management courses, and claimed inconsistencies discovered therein had been addressed.

Buckinghamshire New University has a strong reputation for the quality of its programmes and we always take quality extremely seriously. The Office for Students (OfS) instigated their recently published assessment of quality in May 2022 but, in spite of our requests, would not divulge to us the precise reasons why they had done so.

We therefore conducted our own review of our business and management courses. Alongside much good practice, our review also found some inconsistencies in teaching delivery and in the provision of electronic resources. Since then, we have worked hard to make sure these issues were properly addressed. Our actions, including new appointments within the University’s executive team and within the School of Business & Law, have delivered excellent results.

The OfS assessors rightly paid much attention to the National Student Survey but we are perplexed that the assessors did not consider our results for NSS 2023. Published in August 2023, these data were based on responses students made between January and April 2023 when the assessors were still working on their report. Students studying business and management at BNU were extremely positive about the quality of their programmes. BNU’s business studies students had the 4th highest positivity about their course in the UK (average of all questions). 95% were positive about the quality of their teaching, the 5th highest score in the UK. 94% were positive about their learning opportunities (3rd highest) and 97% were positive about academic support (3rd highest). We believe these demonstrate the success of the actions taken by University staff and its leadership to address the past challenges noted both by the assessors and our own review.

We are further encouraged in this belief by the OfS’s assessment of overall academic quality at BNU. In December 2023, a panel of expert assessors, appointed by the OfS, awarded BNU Silver for both Student Experience and Student Outcomes in the Teaching Excellence Framework 2023. Across all 13 features of the framework, the panel found every single one at BNU was of either very high or outstanding quality. Moreover, the panel’s detailed report, which has been shared with the University, positively and frequently referenced BNU’s courses in business and management.