Big variations in England’s education rankings in global league tables can be misleading and should be treated with caution, the Sutton Trust said today as it published a new report explaining apparent inconsistencies in international rankings.

According to the new Sutton Trust report, Confusion in the Ranks, the rankings also obscure the true challenge facing English schools: the consistent success of many East Asian schoolchildren.

The latest OECD PISA league tables place England equal 23rd in reading, equal 27th in maths and 16th in science out of 65 countries, whereas a recent global index prepared for Pearson by the Economist Intelligence Unit placed England 6th of 40 countries.

Research by Professor Alan Smithers, of the Centre for Education and Employment Research at the University of Buckingham, shows that apparent differences in performance between different global tables, including PISA, TIMSS, PIRLS and the new Pearson index, are the result of three key factors:

  • Different countries are included in the different tables – many of those that were ahead of England in the OECD PISA survey in 2009 did not take part in the TIMSS survey in 2011, including Germany, Switzerland and the Netherlands.
  • League tables exaggerate the importance of raw test scores, where a country that is just one or two points ahead of or behind England has probably got the same level of performance.
  • Some countries do better on one survey than another, perhaps because the surveys test different aspects of literacy, numeracy and science.

In addition, a third of the score in the Pearson index, which placed England sixth in the world, was the result of data on secondary school completion and university graduation rates, which may owe more to policy decisions than attainment. When the latter data are excluded, England drops to 12th position, and could be lower if five other countries ahead of England on PISA, but not included in the Pearson index, were added in.

Professor Smithers concludes:  “When omissions, the distorting effect of including graduation rates, and differences in the aims of the tests are taken into account, PISA 2009 and the Pearson Index 2012, despite appearances, give more or less the same result.”

The report concludes that there are five countries or territories that generally perform better than England in all the tables: Hong Kong, Japan, Singapore, South Korea and Taiwan.  On PISA, Belgium, Germany, the Netherlands, Germany and Canada outperform England. Finland also usually performs better than England, though did not do so in the 2011 TIMSS maths test.

Politicians place considerable weight on apparent success or failure in the tables – Labour celebrated a good result in the 2000 PISA study, although the sample later proved to be skewed to high achievers; the Conservative-Liberal Democrat Coalition has used an apparent subsequent dip in performance to criticise school standards to justify its reforms.

Yet the differences in rankings reflect an increase in participating countries and a more representative sample in English schools rather than any change in performance.

Sir Peter Lampl, chair of the Sutton Trust and of the Education Endowment Foundation, said today: “Global education tables have become an increasingly important tool in the political debate in Britain as well as in other developed nations. But league table rankings are not always what they seem, hence the see-sawing in the rankings that we have seen over the years.

“Whatever the average ranking of English education, we need to focus on reducing social segregation which is greater in England than almost all other OECD countries. We also need to improve teaching standards across the board and not focus so much on structures if we are to match those countries that consistently outperform the rest of the world – not just places like Hong Kong and Japan, but successful European education systems – and use their achievement as our benchmark.

Professor Smithers said: “Although it looks from the media coverage as though there are wide differences in the results of PISA, TIMSS and the Pearson Index, there is, in fact, an underlying consistency.  If there is a lesson to be drawn from these analyses it is: don’t leap to conclusions from the league tables.

“These league tables are designed to enable education systems to be compared, and it is easy to assume that any differences in pupil performance are due to the education system. But this is not necessarily the case.

“The superior performance of Asian pupils has been attributed to a culture of hard work and effort, the personality trait of quiet persistence, and distinctive parenting. After all, Chinese children also shine in England’s education system. There may not be a magic bullet from these countries which can be incorporated into England’s education system, and we may do better to look at those European countries that do well in PISA to learn the lessons of their success.”

NOTES TO EDITORS

  1. The Sutton Trust is a foundation dedicated to improving social mobility through education. It has published over 120 research studies and funded and evaluated hundreds of programmes for young people of all ages, from early years through to Access to the Professions.
  2. The report Confusion in the Ranks: How Good are England’s Schools? is available on the Sutton Trust website at www.suttontrust.com/our-work/research/
  3. There are three main sources of global comparisons. The OECD tests secondary school pupils in its Programme for International Student Assessment’ (PISA).  PISA tests in pupils’ native language, maths and science are designed to assess to what extent students an apply their knowledge to real-life situations.The Boston-based International Association for the Evaluation of Educational Achievement (IEA) tests both primary and secondary pupils in its ‘Trends In International Mathematics and Science Study’ (TIMSS) and primary pupils through the  ‘Progress in International Reading Literacy’ (PIRLS).  Recently Pearson commissioned the Economist Intelligence Unit to amalgamate the various tests and add data on adult literacy and graduation from school and higher education to form into a Global Index of Cognitive Skills and Educational Attainment.