Using Theory of Change in the Iterative Evaluation of the ASPIRE Intervention to Address the Black PhD Leaky Pipeline in UK Higher Education Institutions

This session will outline the evaluation process of the Accomplished Study Programme in Research Excellence (ASPIRE), a unique intervention to address the black PhD leaky pipeline in the UK. ASPIRE is a reciprocal teaching programme, designed to provide research mentorship and wellbeing to improve graduate outcomes and access to doctoral study for Black and Black heritage students. This session will explore the evaluation of the programme over three years and three cohorts of scholars and supervisors, sharing best practice in evaluation and the ways this was used to advocate for policy solutions and the engagement of decision makers. This session will be jointly delivered by the ASPIRE programme lead and the independent evaluation lead, showing how evaluation can be used to advocate for transformational change. It will involve a presentation followed by discussion and Q&A.
The presentation will outline the Theory of Change (ToC) process, including the ways this fed into the evaluation of the ASPIRE intervention across a three-year period, from cohort 1 to cohort 3. A ToC framework followed the four stages of diagnosing, planning, measuring and reflecting. These stages were used to create a logic model, describing the need the programme was trying to address, the changes it aimed to make (ie outcome and impact), as well as how it planned to achieve those (ie activities), providing a basis for the evaluation. In this session the logic model will be shared, alongside the ToC process, which led to its creation.
Following this, a mixed-methods approach was adopted, including a variety of data collection methods which evolved during the three-year process, including interviews, listening rooms, surveys, and diaries with scholars, supervisors, mentors and ASPIRE intervention leads. This provided insight into what makes a successful intervention, such as what worked well and aspects which required adaptation or change, as well as the impact of ASPIRE. The process of collecting and adapting the data will be shared with delegates in this session, detailing the ToC, evaluation process, the results of the evaluation, and how these fed into the iterative evaluation plan.
Finally, this session will share how the evaluation has been used to involve and influence decision makers and other key stakeholders including MPs, university senior leaders, sector leaders such as the Office for Students (OfS), as well as university staff and students:
• Guiding 46 scholars through mentorship, culminating in 10 securing fully funded PhD positions.
• Empowering participating scholars to overcome impostor syndrome.
• Fostering a stronger sense of belonging among the scholars involved.
• Spearheading significant policy reforms across partner universities to dismantle barriers hindering Black students' pursuit of doctoral studies.
• Establishing dedicated scholarships for Black students.
• Advocating for the project's potential on a national scale, including discussions with Members of Parliament.
• Garnering global recognition, notably featured in Forbes during Black History Month.
• Engaging with senior leadership within regional universities on anti-racist HE agenda.
• Pioneering anti-racist education initiatives within the sector.
• Disseminating the project's impact through various channels, including academic journals, conferences (both academic and EDI practitioner based), BBC Radio Sheffield, The Conversation, and Wonkhe.
• The development of the ASPIRE podcast to engage a wider audience.
ASPIRE's journey exemplifies the transformative potential of targeted interventions, underscoring the power of evaluation in driving meaningful change within educational landscapes. This session will support others delivering programmes which aim to address inequalities as well as their evaluators. It will share best practice in evaluation for transformational change, detailing the use of ToC and a longitudinal, iterative evaluation process.