To comply with section B4 of its conditions of registration, the Office for Students requires providers to retain students’ assessed work for up to five years after the completion of their course.
Previously on Wonkhe, Paul Greatrix highlighted problems with this requirement:
This represents an enormous, staggering burden requiring institutions to establish new systems to collect, collate, organise and store securely many hundreds of thousands of items, of various forms, each year. A recent survey estimated a minimum cost in the range of £270,000 to £1m per institution depending on size.
Here, we want to explore the practical implications for teachers, students and the OfS inspectors themselves – especially for authentic work-based assessments in creative disciplines (although many of the points apply more generally).
Branching out
To support employability, creative disciplines are adopting authentic assessment methods which allow students to negotiate their response to an assessment brief, including its form and format. This student-led approach is intended to connect the dots between academic study and the “real world” of work, allowing them to develop their professional skills by proposing multiple solutions to real-life problems.
For students studying creative writing, an “authentic” response to a negotiated assessment brief might include a creative artefact, along with a practical element that demonstrates the work’s potential market value to a specific audience or outlet.
The creative artefact might be a “traditional” piece of creative writing such as a short story, collection of poems, or script. But increasingly, students are branching towards outputs which reflect the modern workplace in which they, as future creative arts graduates, might find themselves: podcasts, apps, blogs, short films, live events, poetry installations, even stories told via a series of linked social media accounts – all are fair game.
Credibility scores
When it comes to making such work futureproof, several problems are apparent. First, in a fast-changing market, what was once cutting-edge can quickly become a cliché – advances in technology would make the games designer’s work of a decade ago look hopelessly outdated today. OfS inspectors will therefore need to be not just discipline experts but also have awareness of what represented “credible” work for each specific past year they review. Otherwise, how will they be able to evaluate what represented excellent, good or poor work at the time the work was submitted?
It doesn’t end there. Often, students choose to respond to an authentic assessment brief by producing a professional portfolio containing many different files and file types. Some of the content may, out of necessity, be hosted online, or on cloud storage. Students can submit a link to such files for assessment, but then what? Will the university have responsibility for ensuring ongoing access to systems which may become obsolete or move from free to subscription payment in the ensuing years?
Even if it is accessible, then work is unlikely to look as it once did: links may be broken, and file formats may be unsupported. A crucial part of grading such work is rooted in its functionality – the ways in which the form of the work relates to its content – and it is precisely this which may be lost.
One possible solution could be a narrowing of assessment scope – encouraging the reversion to traditional assessments on easy to store Word documents. This almost guarantees easy future access, complies with OfS’s preference for assessment anonymity “where possible”, saves money from cheaper storage – as well as lessening carbon footprint, we might add. However, formal academic texts answering standard questions are under challenge from AI generated capabilities and have a limited potential to fully recognise students’ ability or develop their broader employability skillset. While academic writing is important, it is only one of many attributes required for success within the graduate market.
They do it with mirrors
Given the practical concerns of large-scale storage, are there other workable solutions? It’s probably worth reflecting on what the OfS is hoping to achieve – valid and reliable assessments supporting credible qualifications. And who wouldn’t want that? Though we might also want to add “fair and inclusive” to the mix while we’re at it.
However, by requiring the retention of assessments for so long, there is a danger that this becomes an exercise in rearview mirror driving. If the OfS considers that grading was too generous for marks given several years ago, then does that invalidate the work of anyone who has graduated since?
If essential to their expanding remit, then the OfS could focus on evaluating assessment practices in the present. We already do this through the external examiner system, so if this is a matter of concern, shouldn’t this be the area for OfS’s expertise? Could the OfS select a sample of work from each year’s students for review and discussion? This might not resolve differing viewpoints of what makes an effective assessment, but it would save universities second guessing what may or may not be acceptable and accessible in many years’ time.
