Simon Burgess and Dave Thomson unpick their new research for the Sutton Trust on the impact of GCSE reforms on disadvantaged pupils.

New research we’ve carried out on the impact of GCSE reforms on disadvantaged pupils has been published today by the Sutton Trust.

Specifically, we looked at the impact of the change from the old style A*-G GCSEs to the reformed 9-1 versions. This is the latest in a series of changes to school accountability since 2010, following the introduction of the EBacc, the Wolf review and the introduction of Progress 8.

Before numerical grades came in there were concerns that these would increase educational inequality. In this research we found that disadvantaged pupils are fractionally worse off as a result following this reform.

Grades awarded before and after the reforms

Our analysis looked at reformed GCSE subjects for which awards were made in 2018. English and maths were the first subjects to be reformed, with 9-1 grades awarded for the first time in 2017. A range of other major subjects[1] followed in 2018.

We analysed the change in attainment in those subjects between 2016 and 2018. We deliberately chose 2016 as the start for our analysis as previous reforms (Wolf review reforms, Progress 8) had already taken effect. We included all pupils in state-funded schools, mainstream and special.

The new 9-1 grades offer greater differentiation of higher attaining pupils. The previous A*-A range is now stretched over three grades: 9-7. Similarly, the previous B-C range is stretched over three grades: 6-4. Lower attaining pupils are now less differentiated, with the previous D-G range compressed into three grades: 3-1.

We would not expect to see much change in attainment nationally at the key grades of 7/A and 4/C due to the process of comparable outcomes that Ofqual applies when exams are reformed. However, even if attainment does not change overall, the disadvantage gap can change.

The chart below shows the grades awarded before and after reform for disadvantaged pupils and non-disadvantaged pupils.

Some slight post-reform improvement in the B-C/6-4 range can be seen for both groups if you look at it for long enough.

This is likely to reflect changing entry patterns for lower attaining pupils (e.g. more of these pupils entering subjects in which higher grades tend to be awarded), since the comparable outcomes approach is designed to maintain grade standards provided there is no change in the prior attainment profile of pupils entering a particular subject.

Where in the distribution has the gap widened?

It’s hard to tell whether things are any better for disadvantaged pupils using the chart above, so in the following chart presents the data differently.

On the horizontal axis it shows the percentage of non-disadvantaged students achieving at least a particular grade. The vertical axis shows the same measure for disadvantaged students. The graph plots these numbers for the pre-reform grades and the post-reform grades. At the points highlighted in the chart – grades A/7 or above and C/4 or above – a direct comparison can be made between the two grade systems.

The gap between each point and the 45-degree line shows the size of the attainment gap at that grade. The further away from the 45-degree line, the greater the attainment gap. For example, at grade B: 48% of non-disadvantaged students achieved at least a B, compared to 26% of disadvantaged students.

It is clear that the overall pattern of the grades in both regimes are very similar: the dashed lines by and large overlap each other. However, we can also see that the post-reform line is further away from the 45-degree line at grade 5. At this part of the distribution there is a slight widening of the overall attainment gap.

Lessons from this

What can we conclude from this?

Despite the intention to make GCSEs much more rigorous, the percentage of awards at the key threshold of grade 4/C has increased. It seems likely that the operation of comparable outcomes meant that the overall distribution of grades was not dramatically different.

However, the reforms have been accompanied by a change in the language used to describe attainment at the grade 4/C boundary. While this has always been the expected standard for 16-year-olds, the government now refer to it as a “standard pass”, ignoring the fact that grades 3-1 are also pass grades.

To emphasise the greater challenge of the reformed GCSEs, the “strong pass” (defined by grades 5 and above) was also introduced and is more in line with the international PISA average .

It is at the grade 5 boundary where most of the negative effect of the reform on disadvantaged pupils occurs. This will matter if grade 5 rather than grade 4 becomes the expected standard for progression to post-16 courses or even in university admission.

 

[1] Science subjects, computing, modern foreign languages (French, German, Spanish), geography, history, religious studies, citizenship, art and design, dance, drama, music, PE, Greek, Latin. Food and nutrition was also newly available in 2018.