Building the evidence base: Insights from recent widening access evaluations

In this post, Rianna Elmhirst, Access & Participation Impact Manager, reflects on two recently published evaluation reports.

How do we know whether widening access activities are making a difference? 

As universities continue to strengthen their approach to access and participation, there is a growing emphasis on building robust evidence about what works, for which students, and in what contexts. At the University of Bath, evaluation forms a key part of our commitment to delivering a comprehensive intervention strategy and evaluation framework for our access work, as part of Bath’s Access and Participation Plan. 

Over the past year, I have led two evaluations exploring the impact of our post-16 widening access programmes and residential summer schools. While the evaluations used different methods and examined different outcomes, both sought to answer the same overarching question: how can we better understand the contribution that outreach and recruitment activity makes to students’ access to Higher Education (HE)? 

What we evaluated 

The two evaluations focused on different stages of the student journey and used complementary approaches (Table 1). 

The first explored the impact of our Year 12 widening access programmes, Discover Bath and Pathway to Bath, on students’ HE knowledge, HE expectations, academic self-efficacy, sense of belonging, and application behaviour. The evaluation combined pre-post programme survey data with admissions analysis. 

The second explored the longer-term impact of Bath’s summer school programmes, using Higher Education Access Tracker (HEAT) data to compare outcomes for participants and non-participants. This allowed us to explore progression to HE, progression to high-tariff institutions, and entry to the University of Bath. 

Table 1: Overview of evaluation approaches 

What we learned from Post-16 programmes 

1. Confidence and belonging are important outcomes in their own right 

Students who participated in Discover and Pathway reported increases in HE knowledge, academic self-efficacy and sense of belonging. By contrast, HE expectations remained broadly unchanged. 

For practitioners, this provides a reminder that widening access programmes do not always need to increase aspirations to be effective. Many students engaging with targeted outreach programmes already expect to progress to HE. The benefits lie in helping students to feel confident, informed, and prepared to navigate the transition to HE successfully. 

This reinforces the importance of measuring intermediate outcomes. While application and HE entry data remain essential indicators of success, intermediate outcomes can go towards explaining how programmes influence student decision-making. Understanding changes in confidence, belonging, and knowledge can help practitioners identify the mechanisms through which interventions support longer-term outcomes. 

2. It is not simply whether students participate, but what that participation looks like 

Application rates to Bath were higher among programme participants than comparable non-participants. However, the strongest outcomes were observed among students who engaged most fully with programme activities, particularly those completing the academic elements of the Pathway programme. It is important to note that the Pathway programme is designed to encourage deep engagement, with residential participation contingent on completing the academic component. 

This suggests that participation alone may be a relatively blunt measure of success. The way in which students engage with a programme and its content may be just as important as whether they attend in the first place. 

For practitioners, this raises important questions about programme design and participant engagement. How can activities be structured to encourage sustained participation? Which components add the greatest value? For evaluators, it highlights the importance of moving beyond attendance data and considering engagement as a meaningful outcome measure and using implementation and process evaluation methods to understand why some programme components generate greater engagement. 

What we learned from Summer Schools HEAT data 

1. Residential experiences can have long-term impact 

Using longitudinal HEAT data and propensity score matching, we found that summer school participants were more likely to progress to HE and more likely to enrol at Bath than similar students who applied to attend but did not participate. The analysis also suggested positive outcomes in progression to high-tariff institutions. 

These findings provide further support for residential outreach as a valuable component of a widening access strategy. It also demonstrates the value of systems such as HEAT in enabling the longitudinal tracking of outreach participants and their progression into HE. 

2. Understanding who benefits is as important as understanding why the programme works 

While overall outcomes were positive, impact was not experienced equally across all groups. Some of the strongest effects were observed among students from areas with historically low rates of HE participation (POLAR). This is encouraging, given that students from POLAR Quintiles 1 and 2 have historically been included within the targeting criteria for summer schools. The findings suggest that the programme may be supporting one if its intended beneficiary groups. 

By using HEAT, we were able to pool multiple summer school cohorts between 2015/16 and 2018/19, providing a larger sample size for the analysis. The increased sample enabled us to explore a wider range of student characteristics in greater depth and identify which groups appeared to benefit the most from the programme. For practitioners, this can support the refinement of targeting, review of delivery approaches, and ensure that interventions continue to support those facing the greatest barriers to participation. 

Reflections for practice 

Taken together, these evaluations highlight the value of combining different sources of evidence to understand multiple aspects of an access intervention. 

Survey data can help us understand short-term changes in confidence, knowledge, and attitudes while admissions and longitudinal tracking data enable us to explore progression outcomes over time. Additionally, by using HEAT we are able to review the level of engagement of participants on Bath activities as well as tracking where those student progressed to, not only whether they progressed to Bath. 

What next? 

We are continuing to work closely with Widening Access, Outreach and Recruitment colleagues to strengthen the evaluation maturity of our large-scale interventions. This includes increasing the use of validated measures in surveys, improving consistency in outcome measures across interventions, further developing our use of HEAT to support monitoring and tracking, and creating greater opportunities for cross-programme analysis. 

Over time, this will allow us to build a stronger evidence base about which approaches work best, for which students, and under what circumstances. 

• Want to read the full reports? Access the University of Bath evaluations. 
• Want to see what else we are working on? Explore our other Evaluation projects. 
• Want to reach out to the evaluation team? Contact appimpact@bath.ac.uk.  
• Want to learn more about Widening Access at Bath? Explore Widening Access and Outreach opportunities