As three years of HEFCE funding comes to an end, we have an evidence base to inform UK policy approaches to learning gain.
Right now, the Office for Students is drawing together findings from the 13 pilots through the evaluation work of Camille Kandiko-Howson. She defines learning gain as a change in knowledge, skills, work-readiness, and personal development, as well as enhancement of specific practices and outcomes in defined disciplinary and institutional contexts.
One of these programmes of research is LEGACY (Learning and Employability Gain Assessment CommunitY), a consortium of 18 Russell Group universities (and Sheffield Hallam University) with four strands of research:
- The development of a tool to measure learning gain over time, led by the University of Cambridge (Vermunt, Vignoles and Illie)
- Assessing the use of positive psychological approaches to supporting employability through the Realise 2 Strengths diagnostic, led by the University of Warwick (Wilson and Behle)
- Engaging “hard to engage” students in employability through career adaptabilities measures, led by the University of Nottingham (Thambar and Wright)
- Assessing the impact of international experiences of developing employability skills and capabilities, led by the University of Birmingham (Jones and Wright)
There are five key messages from LEGACY which were presented at our Learning Gain: An Agenda for Change conference in September.
- Learning is non-linear: Learning gain is often described as the “distance travelled” between points A and B. However, our research reinforced how students make progress in their knowledge and understanding. They can also have set-backs and concerns. Accepting the pedagogic staple that learning is non-linear is essential. Otherwise measurements at particular points in time will not accurately reflect a student’s final level of development.
- Subject level is more significant than institutional level: Subject is a more significant differentiator in respect of outcomes than institution. There is a lesson here for the subject-level teaching excellence and student outcomes framework (TEF) in terms of how “signature pedagogies” produce particular kinds of learning outcomes and how learning gain tools might be designed.
- Methodological rigour is important: LEGACY has used a variety of methodological tools and techniques from small scale interviewing to control groups, large-scale surveys and data tracking. This has generated a wealth of significant data as well as greater understanding of “what works” methodologically. There is, though, more to be done on how broader institutional and subject practices impact on outcomes, as well as the impact of peers.
- Student engagement is incredibly challenging: LEGACY research has involved around 7,500 students over the past three years. Yet getting students to engage has been an enduring theme across the whole of the learning gain programme. This is why it is seriously good to know that OfS have commissioned Stella Jones-Devitt, Liz Austen and colleagues at Sheffield Hallam University to do more work on improving our understanding of students’ perceptions of learning gain.
- Within LEGACY we have also had our challenges: Any successes have been due to persistence, commitment and collaboration. It is also because we have focused on making our work relevant to students. Our analysis indicates that to be effective and ensure engagement, learning gain needs to be embedded in the curriculum or as part of developing a set of capabilities and skills that students recognise as necessary.
What Next for Learning Gain?
Learning gain, if nothing else, is about putting the students’ educational progression first. Yet we need to ask whether we making it too complex for policy. In response, the key messages from the LEGACY conference fall into two crucial action areas:
- Collective vision – those working on advancing the understanding of learning gain in HE need to pool collective knowledge, have confidence in key findings and outline a vision for the future. As professional educators we have to own the policy problem. The role of the OfS, in a regulatory capacity, should be to outline the rules of the game in which this vision fits. There is a risk, if the sector does not collectively move to fill this brief, that an enforcement of measures may fill the gap. As an immediate task, learning gain researchers should outline key principles for measurement, which can be contextually defined in a manner which reduces the academic complexity for policy makers.
- Sector ownership – for a fuller consideration of the implications of learning gain measurement, the sector should apply a more inclusive lens. This would include further work with students (how do you understand your own learning gain?) and graduates (how have you demonstrated learning and how could this be developed?). There is also more work to be done to include the perspective of professional bodies and importantly, employers, to understand the potential demand for learning gain measurement from HE providers.
There is a growing body of work around learning gain, most recently collated in the journal Higher Education Pedagogies’ special edition on learning gain. Many of our speakers at the LEGACY final event have published their findings here, including Vermunt, Illie and Vignoles from our LEGACY Cambridge strand.
Let’s not let it die
Around the sector there has been some discussion that the agenda may disappear. That would be a shame. Appropriately designed, learning gain is a powerful tool and one we should be using. As one of our student evaluators expressed it:
This is actually much more useful than perhaps the student survey. Because it’s much more holistic … it’s a much more personal way of finding out about you rather than being just a student of the university of x, and what was your experience in that institution.
