A challenge for senior leaders to act on evaluation evidence for improving student outcomes

Evaluation really does show what works when it comes to students' outcomes, but that doesn't mean that's what happens. Liz Austen and Stella Jones-Devitt call on policymakers to take the pledge

Liz Austen is Associate Dean Teaching and Learning (Social Sciences and Arts) at Sheffield Hallam University


Stella Jones-Devitt is Professor of Critical Pedagogy at Staffordshire University and Director of the Staffordshire Centre of Learning and Pedagogic Practice

Higher education institutions are constantly being told to make higher education better for students, and these demands are steeped in political rhetoric.

In the world of vacuous policy promises and accompanying public indifference, we offer some spin-free enhancement points for HE policymakers to action, in getting them productively across the finish line for student success. Sounds easy: so why does achieving the holy grail of student success appear so problematic? In this blog post, we imagine a world in which key decision-makers boldly go where they very often fear to tread. Into explicit evidence-informed effective action for enhancement.

We suggest that senior policymakers with responsibility for learning and teaching enhancement at organisational and sectoral level need to have more explicit and strategic ownership and accountability for envisioning what great evaluation looks like, rather than being exasperated when such enhancements don’t cohere or provide desired outcomes.

Drawn from evidence gleaned from our near decade-long work as HE evaluators, such as this longitudinal Evaluation of 20 Years of Enhancement Themes for QAA Scotland published earlier this year, we offer five key promises that these strategic policymakers need to make, alongside those in wider HE funding roles who might benefit from adopting some of these promises, too.

The promises are translated into a set of “we will” pledges for policymakers to adopt, if they truly want to make a positive difference for, and with, students. We ask you to consider why “we will” invariably becomes “we won’t” or “we can’t.”

Pledge 1: We will align all enhancement work within an overarching organisational framework.

Within higher education institutions, maximum effectiveness will be achieved by aligning all enhancement-led projects and activities within an overarching framework which will demonstrate clear assessment of contribution made to strategic ambitions. Activities will be rated against a set of shared and transparent outcomes. The alternative is an activity focus rather than an outcome focus – well-meaning but lacking an evaluative mindset.

Pledge 2: We will ensure there is a well-resourced “critical mass” in place to drive enhancement work and organisational success.

All enhancement work will be aligned with strategic change to ensure that a critical mass for momentum is established. Ensuring strength in numbers within an institution will drive effective resourcing, better engagement, understanding and, most importantly, impact. This avoids silos and lone early-adopters.

Pledge 3: We will ensure that enhancement work is integrated into a change process for maximum impact.

Enhancement work is more effective when planned, monitored, evaluated and implemented within a strategic change process that can make a difference. We will consciously involve a range of stakeholders in any decision-making to routinely reflect on impact and any future revisions, scaling or stopping.

Pledge 4: We will focus on impact, not number of outputs

Rather than only encouraging the production of a plethora of outputs (toolkits, practice guides, reports, case studies etc), we will ensure consideration of how activity drives benefit and impact, especially concerning the contribution made to achieving positive student outcomes. The production of a hundred plus outputs doesn’t necessarily add up to impact.

Pledge 5: We will prioritise improving student outcomes by having clear indicators of success from the start.

All enhancement work will include projection of impact on a range of student outcomes. Success will be considered as a long-term venture relative to the planned activity. Long-term student outcomes relating to improving the metrics on continuation, attainment, completion and progression will be a priority when institutions develop strategic enhancement approaches. Intermediate outcomes are our friends – they can ensure continued enhancement funding whilst we wait for the evidence of assumed longer term impact.

So, there you have it. Five key evidence-informed policymaker promises concerning enhancement work that should make a real difference to achieving positive student outcomes. Simple, right? Unfortunately, our near decade of evaluation experience, along with a burgeoning evidence-base, indicates that very few enhancement approaches go beyond planning enhancement work premised on emotional desirability. This inevitably involves lots of well-meaning activity and outputs, in which everyone participating has a jolly time, yet characterised by a lack of focus concerning the difference any of this makes for improving student outcomes.

Changes to access and participation plans (in England), TEF and the emerging Tertiary Framework in Scotland are more explicit than ever about the connections between institutional activities and impact on student outcomes. These pledges therefore become evaluative reminders for effective policy enactment. How can your institutional policymakers move from a “we won’t” or”‘we can’t” model when engaging in enhancement work for student success?

Leave a Reply