Last summer, an academic was puzzled by why their students were suddenly acing the end-of-semester assessments. The usual quizzes were being returned faultlessly. Confident that the inclusion of images would thwart AI tools, the academic mentioned this to a colleague. The colleague’s reply landed with a thud: “You know large language models can work on images as well as text, right?”
Universities face a significant challenge, with tech giants poised to deploy AI-powered learning experiences that could substantially alter the current model. As access to expertise becomes widespread and traditional assessment models are tested by large language models (LLMs), what will distinguish and ensure the value of a university education? Success for universities will hinge on three things: clarity in the teacher-learner relationship, a well-defined institutional mission, and strategic approaches to learning delivery.
Piling up in the streets
We all know that in the era of generative AI and large language models, the usual assessment suspects are dead. The essay, still beloved of many, is perhaps only the most obvious corpse. If we’re serious about preventing widespread plagiarism, many other traditional assessment methods are also no longer viable. Of course, this potential for content generation is currently obsessing academics and managers — how do we stem a flood of “good honours” and the inevitable grade inflation that will catch the eye of the Office for Students? Yet, it’s almost certainly a mistake to approach this challenge from a purely deficit perspective. A more balanced, and perhaps more “academic” approach would give equal emphasis to generative AI’s potential for advancing knowledge and inspire us to develop authentic, AI-resilient modes of assessment.
A cottage industry is rapidly developing, promising to translate traditional assessments into supposedly generative-AI-proof formats. While freely available AI adaptive toolkits offer some superficial solutions, they greatly underestimate the power of current LLMs. For instance, replacing an essay with a task that asks for an AI-generated answer and a reflection on the AI’s usefulness naively assumes the AI can’t also generate the reflection.
Even that supposed bastion of plagiarism-proof assessment, the multiple-choice questionnaire, has been hacked. Websites like Virtual Professor offer a shockingly simple workaround: students merely hover over online test questions, and the correct answer magically appears. This underscores a key challenge — generative AI can significantly enhance students’ learning and assessment, but only if we move away from oversimplified, traditional modes and adopt more thoughtful and engaging frameworks.
Education first
Faced with the rise of generative AI, how must current higher education proactively rethink learning and assessment? The answers here are tentative and provisional, but they approach the challenge from an educational and strategic rather than purely technological perspective. This approach, we think, offers a productive path for institutional and educational planning.
The first cluster of propositions focuses on transparency in the relationship between academics and students. This involves an open compact between learner and teacher about learning goals and how they will be achieved. It entails clear expectations around students’ use of AI technology, matched by academics being equally transparent about their own use. This could mean learners documenting their “technology map” as part of their assessment — a map shared with teachers and potentially with peers. LLMs can also be used to assess students’ level of understanding and engagement, providing aggregated data at both programme and module level.
The second cluster of propositions centres on a holistic generative AI framework for learning and assessment, emphasising the relationship between institution, learners, and assessment. In this framework, content acquisition becomes less central, while formative assessment and discovery-led learning gain importance. Peer interaction and dialogue with teachers become key to the learning process, finally fulfilling the full promise of the flipped classroom. Learning with LLMs can be inherently engaging, encouraging students and teachers alike to explore through well-crafted natural language questions. Social learning becomes equally important as individual knowledge acquisition.
The third cluster of propositions centre on integrating LLM techniques, methodologies, and models across the university curriculum. LLMs must become a core part of the educational experience, understood by learners, teachers, and everyone involved in supporting the learning process. This imperative grows with each new development in text, image, and voice generation — even the potential for AI-driven lessons delivered by simulated celebrities.
Fundamental overhaul
This formidable challenge finds most universities unprepared. It requires a fundamental overhaul of education strategies and the entire student experience. Success demands clarity of strategic intent and execution, along with significant commitment, investment, and time — resources many universities currently lack. Educators must rethink their approach, as students will demand nothing less. Staff will need comprehensive support in understanding and deploying LLM methodologies, a task that also requires commitment, investment, and time in the face of rapid development. Ignoring the transformative power of LLMs risks the sustainability of the current higher education sector, as educational experiences are easily replicated by tech giants like OpenAI, Microsoft, AWS, and Meta.
In this context, a critical question arises: in the age of generative AI, how will universities provide unique value to learning? The answer lies not in replicating teaching methods or assessment techniques, as these can be easily matched by other providers. The true value lies in expertly guiding interactions between learner, teacher, and LLM. Universities, with their expertise and commitment to knowledge advancement, are uniquely positioned to lead this transformation. The continued relevance of higher education hinges upon their success.
The article argues we need to move beyond traditional assessment formats and develop “AI-resilient modes of assessment” but doesn’t outline what these might look like.
The article suggests that “the usual assessment suspects are dead”. But is conspiciously absent in mentioning the most traditional and (along with the essay) the most widely used assessment in education – the exam, which also happens to be the most AI-resilient assessment we have.
To make the case we need novel assessment formats, you need to outline what these might look like, and how they would be better than exams.
And I suppose “Higher Futures” shall lead this fundamental overhaul of higher education, shall they? This article is a great example of why, pace Twain, your reports of the essay’s death are greatly exaggerated. Next week I shall expect a piece about various qualitative subjects facing their ends because AIs will soon do it better. I will not be holding my breath by the way.
The piece also does not articulates any new, innovative assessments which fold in “AI”, nor does it offer any proof that the essay format has been definitively struck down by ChatGPT or any other LLM. In fact I am unsure what, exactly, it offers, much like a typical GPT query answer.
Also, at no point does this article address the predisposition of trained AIs to lie, to make stuff up, based on the reams of data they have been trained on and the endless queries they have been set. It says students of the future will get to focus “less on content acquisition”, as if somehow “content” is bad in academic disciplines as opposed to their general foundations, and suggests the “flipped classroom” has made a return when: a. it never left, and b. that model relies on students doing work themselves.
In sum I am quite glad that “universities of the future” will almost certainly not look like what this piece describes, and that if they continue to exist at all it will be as genuine sites of knowledge creation and testing, not merely as “facilitators” of a particular kind of communication.
This is an exceptionally poor “analysis” of generative AI and higher education assessment policies. It repeatedly and emphatically asserts without adducing evidence (not an approach compatible with most university disciplines) and fails to provide meaningful answers to its own impoverished questions. Perhaps consulting with academics (and students) on the frontline of teaching and learning would add a useful perspective? Might subject benchmark statements (https://www.qaa.ac.uk/the-quality-code/subject-benchmark-statements# ) perhaps be relevant to this discussion? Virtually all UK universities have ‘net 0’ policies or aspirations. What would the environmental impact of their wholesale adoption of generative AI by universities for both teaching and research be on these? We don’t need an AI-generated ‘celebrity’ environmentalist to engage our students on this front at present. Must do better than this Wonkhe on such a vitally important issue.
I wouldn’t be so quick with the criticism here, the authors are in fact on the front line of teaching and learning, and do consult with students. I am one of those students and we have been consulted. The universities though are lacking in their response. The authors haven’t provided answers because they don’t have them. Every single attempt to integrate AI, or work around AI, has been a blatant failure. I know this because we have had meetings in which it has been discussed how easy it is for me as a student to cheat in every attempt to adapt to modern technology.
The point of the article is to first get the universities on board. Because the real problem is when the professors are at a university that doesn’t even want to acknowledge the depth of the problem, coming up with implementable answers is impossible.
I’m sceptical about three-step solutions at the best of times, and right now even more so. In their book Radical Uncertainty, John Kay and Mervyn King instruct us that the best response in radical uncertainty is first to accept that we don’t know the answer, and then to determine to make sense of what is really happening here. There is unlikely to be one or even three answers at this point. Gen AI will continue to advance and evolve, so any responses we can come up with today will likely be redundant soon. Rather than reacting, we should be sensemaking, experimenting, probing, informing ourselves (I mean really hands-on learning and skilling ourselves). Embrace the mess and the uncertainty. And beware potted solutions. By the way, are we really sure that ‘teaching methods or assessment techniques … can be easily matched by other providers’ such as OpenAI and AWS? That doesn’t seem to me to be a very informed view.
This article seems almost to be rooted in a “post-truth” view of the world. In many (most?!) academic subjects, there are facts and non-facts, and perhaps the single most valuable skill we impart to our students is the ability to differentiate the one from the other. This both conceptually by developing reasoning skills, and in specific cases by imparting knowledge/content. As other commenters have observed, LLM models remain fundamentally incapable of distinguishing truth from falsehood. Until they can do this it is hard to see a legitimate role for them in subjects with any degree of intellectual rigour, unless perhaps a bit part as a generator of nonsense examples for the students to practice debunking.
I am curious how you see the future direction given what you have said. Both in regards to (1) assessment structure and (2) course structure?
In regard to (1) assessment structure, there is a rapidly growing base of research evidence that shows how well these models do on current assessment, and with the recent papers published by OpenAI etc., this rate of improvement isn’t slowing down anytime soon. So as a student, I am concerned about how easy it is to cheat, and thus how I can stand out from those who aren’t actually interested in the learning process, and just want the degree/job (I am sure you are aware of the amount of students who take this trajectory).
In regards to (2) course structure, it appears that if a course isn’t structured around the world I will graduate into, then what is its use. Current courses are the equivalent of teaching how to use a fax machine. Critical skills are a part of what is taught, but writing skills and memory recall are given a much higher level of focus. You are right, in a world with AI present, critical thinking skills have an even higher importance. But if the attitude of lecturers is to imply that AI isn’t good for much, then you are missing the real concern. An approach that has been used by at least one of the authors is to get students to write an essay with AI. This way things like writing style etc. are openly worth little. But falsity and poor thinking are marked down heavily. Another way is to train AI on the course content, and use that as a learning tool. This is something that has been empirically tested, and works. So I am not sure the attitude that the article here is “non-factual” is grounded in anything other opinion. These things have been tested.
Ben, are you implying that there is a world waiting for you where “style” in writing (a broad term, and one that likely fails to capture the rhetorical savvy good writers have practice with) is devalued? I ask because your description of the assignment of “writing” an “essay” with AI makes it appear as if the assignment privileges informational correctness above all else. This is my fundamental issues with people who think they are using AI to “write,” and my misgivings become nightmares when people purport to teach writing that takes the rhetorical situation and the choices that lay within into little account. And let’s not even get into the idea that writing as an epistemological act dies on the altar of AI.
Too many claims have been made that we have to get out in front of the latest tech by people—no matter how well-meaning they may be—who profoundly misunderstand the current technologies and theories behind them.
I completely disagree that teaching how to use AI is a new imperative. AI is increasingly user friendly. 5-10 years from now, everyone will have ready access to AI-driven assistants that make using AI exponentially easier than it is to use now (and it’s already pretty easy to use now). No, the imperative is that schools counteract the tendency for this technology to make us rely less and less on our own minds for producing intellectual products and to thereby badly under-develop our own intellectual capacity. We face a significant risk that AI systems will begin to approach human levels of wisdom, not because of advances in Artificial Wisdom research, but because of a rapid decline in humans who have developed the capacity for critical thinking and moral and prudential judgment. In an age of AI, schools have really only one imperative: to vigorously resist the natural tendency for AI to make us stupid.
We are currently in the position where Universities are unable to spot and prosecute students who are making adept use of AI in order to produce essays that are fairly good and for which it is not possible to identify how AI has been used. Some folks advise that we ask students to just let us know how they are using these tools – but why would they bother if we are not going to be able to discern it? Unless we are witness to the whole creation process of an essay, (and who has the time for that??) I think the essay IS dead. It’s dead because as tutors we are not so interested in the essay as a product but the process of knowledge generation. If we want to judge what students have learned we need to go back to using, at least in part, timed and closed book exams. If we want to judge what skills they are attaining, we need to watch them using them. Unfortunately, university systems are so slow to adapt I think we will have a couple of years where we should be graduating students with the postfixes BSc (Hons) [AI], BA (AI), MSc (AI) and PhD (AI). As much as anything else, I don’t think universities are paying much attention to the extra workload that uncovering new forms of cheating (or working out whether it is acually cheating) entails. Some folk don’t bother asking the question, so students who are using AI inappropriately will get through. The first question I now ask myself when I read a piece of student work is not “What have they learned”, it’s “How have they cheated”. It’s horrible.