Our Director of Programmes, Laura Bruce, discusses our latest report Measuring Disadvantage.

How to ensure that initiatives reach the right students and work is targeted effectively is a regular topic of conversation across all areas of our work at the Sutton Trust. It sits at the core of our internal discussions on how to select students for our programmes, with our university partners on contextual admissions and with our employer partners when discussing early talent recruitment. 

It is not a simple question, how do you judge the disadvantage of one person compared to their peers, and the subsequent impact that disadvantage may have had on their education, support networks and decision making?  

As today’s report, based on research by Professor John Jerrim, highlights, in the absence of access to data on household income over time, proxies such as postcode markers are predominantly used to answer that question; to make the best estimate as to how someone’s educational outcomes may have been impacted by their circumstances. As has been well discussed across the sector, proxies inevitably lead to discrepancies, biases, false positives and false negatives, and so we very much welcome today’s practical guide on the advantages and limitations of different measures to inform our practice. 

As practitioners at the Trust, we have aimed to strike a balance of ensuring that, as far as possible with the data we have available, we do not miss students that one individual marker may exclude, whilst also attempting to limit the number of false positives – students who are captured by individual markers but who are perhaps not as disadvantaged as their peers. To achieve this, we have used a basket of measures including free school meals, postcode markers, school demographic and first generation and weighted each of these criteria equally to give applicants an eligibility score. This decision was based upon not previously having the empirical evidence and confidence to weight one form of disadvantage more highly than another. Our own internal monitoring showed that the more criteria that a student met was a good predictor of their reported household income, with the average Sutton Trust student meeting 3.6 out of 5 criteria.  

Having this report and evidence today will help to strengthen our targeting going forward. Highlighting free school meal eligibility and ACORN as the two most reliable markers when mapped to household income will enable us to weight these markers more highly in our programmes’ selection criteria to ensure we identify the most disadvantaged students, continue to take a holistic approach and limit the false positives within our data set. 

For universities, who are making higher stakes decisions in identifying students for lower grade offers through contextual admissions, it is even more important that the most reliable markers are used so that the most disadvantaged students can access support and universities can work towards levelling the playing field.  

Our concern for the sector continues to be that free school meals data is not currently shared with universities to support their decision making. If FSM data is not available, the research shows priority should next be given to ACORN. However, ACORN is a commercial data set, that can be invested in, but is not readily available for comparable use across the sector. The methodology behind it is also not openly published. IMD is another good option for an area level marker where ACORN cannot be used, and has the benefit of being publicly available, but does also itself have limitations. No measures are perfect; our advice to universities is to make use of the most robust measures available to them.     

Universities are currently assessed on their intake by the POLAR postcode marker, which looks at higher education participation by area. This is understandable as it provides a consistent way of measuring progress through a readily available data source. POLAR however is not designed as a marker of socio-economic disadvantage and the research found that it had little correlation to household income. The research also identified biases in the data sets, most notably in the POLAR postcode marker, which is particularly concerning. These biases skew against students who live in cities and most notably against black students, renters and students of young mothers – all of whom are young people we want to be targeting to ensure equality of opportunity.  

Given these limitations, we are advising universities not to use POLAR in contextual admissions for individual students. The Sutton Trust are also calling for universities to be granted access to free school meals data and suggesting that the Office for Students consider the inclusion of these markers, and review the role of POLAR, in the next round of Access and Participation Plans. 

Of course, today’s report has looked at how well markers assess socio-economic disadvantage and focuses on levelling the playing field based on economic disadvantage. Whilst there are intersections across underrepresented groups, it is also important to recognise other forms of under-representation in higher education and the employment market in their own right, for example race, disability and mature learners. These issues of underrepresentation need further consideration and focus to ensure equality of access across the broadest spectrum of society. 

With this in mind, we will continue to look holistically at our work and individuals that we support through our programmes, now with a more refined approach to the data sets we are using.