Our Director of Programmes, Laura Bruce, outlines the evaluation of our digital delivery in 2020, and what we can learn to improve digital access in the future.

A year on from the first national lockdown and digital delivery has become embedded in our everyday working. This time last year The Sutton Trust team were contemplating how to manage a rapid transition to digital delivery, how to support our partners, and ultimately, how to position ourselves to quickly transition back to face-to-face delivery when it was safe to do so. That was wishful thinking!

Fast forward a year and we have launched our Sutton Trust Online platform, supported over 6,000 young people to access it and are now preparing for a second summer of digital delivery for up to 10,000 students. Alongside delivery, we are also considering our long-term delivery models, with a focus on blending face-to-face and digital elements. Evaluation is always at the heart of the Trust’s decision making; understanding if and how student outcomes have varied given the digital format is extremely important to inform our next steps.

Our overall aim remains the same: to support students into leading universities and professions. Whilst we have a longer wait to access the university and career destinations of students who took part in digital delivery this summer, we have been able to evaluate the shorter-term outcomes of changes in attitudes and behaviour, through our partnership with The Bridge Group.

Their research analysed quantitative data from surveys, engagement data from our platform and qualitative data from interviews with programme participants to compare the impact of our face-to-face provision in 2019 with digital provision in 2020 on participants’ attitudes, understanding and ambitions.

Detail on the key findings of the report is shared below; overall, the research found similar levels of positive change in 2020, indicating that the digital programme was a good alternative to face-to-face programming.

In particular, we saw activities based on sharing factual information and advice, for example personal statements and students finance, match or exceed the experiences of students in previous years.

We did however see little or no positive change in areas relying on experiential learning and ‘soft knowledge’, for example students’ confidence to move away from home for university or students’ seeing themselves in the workplace. This has reinforced our previously assumed position that some aspects of programmes are easier to replicate than others in digital format and that face-to-face programming is still very much needed to achieve certain outcomes.

In terms of the positives, participants welcomed the flexibility of digital provision and there were some particularly positive stories of individuals being able to better access our programmes due to this flexibility.

This is balanced with overall attendance on the digital programme being lower than on a face-to-face intervention. Typically, once a student is at the university, they stay for the full session or week of activities and do not dip in and out as we have seen in digital.

We are now implementing changes to our programmes based on the positives of digital delivery: creating more flexibility and accessibility and looking to build on the success of sharing knowledge and information via digital means.

All students on a Sutton Trust programme this year will gain access to Sutton Trust Online, which they will continue to access until the end of Year 13/S6. This means that existing programme intervention, which will return to face-to-face when possible, will be supplemented with digital resources and will allow students to revisit the topics that they need the most. We are also adding a programme of digital webinar support to engage students throughout these years and will focus these on sharing knowledge and information, for example, delivering personal statement support via webinars.

Given the relative success of the digital intervention, we will also be inviting up to an additional 6,000 eligible students to engage with Sutton Trust Online on a purely digital basis. For these students, we hope the webinar series and redesigned content structure will boost engagement and keep students returning to the platform throughout their university application journey.

There is no doubt that we still have much more to learn and our continued focus on data and evaluation will allow us to approach delivery in an iterative manner, but it is reassuring to find that all the work so far has made a difference to students who may have otherwise missed out entirely on outreach activities.

This is evidenced by one key theme that came out of the research, the theme of students being immensely grateful for the opportunity. After a year of change, disruption and coping with everything the pandemic has thrown at us professionally, it is a great reminder that the work we do has impact and provides opportunity to students who need it the most.

The Sutton Trust would like to thank Bloomberg LP for their generous support of Sutton Trust Online, which you can learn about here. We would also like to thank the Bridge Group for their evaluation and our platform partners, Komensky, Filtered, Causeway Education and The Access Platform.

Key points

  1. The digital programme provided a different experience to a face-to-face programme, but it still proved valuable for students.
  2. Students were grateful for the opportunity and were really impressed with quality of the programme.
  3. It was noted that students may have had lower expectations, given the pandemic and immediate response to it, which may have made them more grateful than in a typical year.
  4. Students would have preferred face-to-face provision, but were happy with the digital programme in the circumstances.
  5. The programmes informed what students wanted to study rather than where.
  6. Students reported increased confidence in their decisions and interacting with others online.
  7. Pathways students particularly valued the opportunity to speak to industry professionals.
  8. Digital work experience gave participants the opportunity to understand what the profession was, being a lawyer for example, but not an insider experience. In previous evaluations the insider experience was highlighted as the most valuable aspect of the programmes, so an area to still consider for face-to-face provision.
  9. Students saw the flexibility of content and being able to revisit sessions as major advantages of digital delivery.
  10. Students saw the overall disadvantage of missing “experiential” learning – this was reflected in students becoming less confident in moving away from home, for example.

Learning for outreach providers

  1. Digital provision is a good alternative.
  2. Digital provision will not achieve the same experience of being on campus or on a work experience placement.
  3. Providers should monitor whether digital provision affects rates of applying to leading universities.
  4. Providers should look to add the flexibility students described to existing programmes to improve the quality.

Caveats and questions

  1. It is difficult to compare the impact of this year of digital delivery robustly with previous years, since students have been in the exceptional circumstances of lockdown. Will students rate digital provision as highly, if there is an offer of face-to-face experiences with other programmes?
  2. The findings above have the additional caveat that we do not know how representative survey and interview findings are of all students admitted to the programmes. It is likely that those who engaged with surveys and interviews had also engaged with the programme. We will be analysing this next year.
  3. Engagement was lower across Summer Schools: for example, just over one-third of admitted students completed 40% or less of summer school sessions, while only 19% completed all sessions. By contrast, when summer schools are held as face-to-face residentials, all participants normally attend all sessions.
  4. It is noted that universities used the Sutton Trust Online platform differently, with some hosting content off the site, for example on Microsoft Teams, which may have impacted the completion rates on the site. Universities that used the platform as their primary delivery method saw higher completion rates than those who used multiple platforms.
  5. Engagement varied across Pathways programmes and the year groups within them: for example, the proportion of admitted students attending each event increased from 64% in 2018-19 to 73% in 2019-20 for Year 12 students on Pathways to Banking and Finance, but halved for Year 13 students on Pathways to Medicine. Figures across the two years are not directly comparable, as 2019-20 figures include students who attended only part of an event.