When Accountability Measures Eat Themselves: League Tables & Measuring Performance for a Range of Abilities

by

9th January 2012

Consequently, Rebecca Allen and Simon Burgess wrote an excellent paper in 2010 showing how the five A*-C school pass rate is not enough information to know how a school will provide for your own child. If your offspring are precocious and enters secondary school with an above-average level of knowledge you might expect them to go on to achieve all As – but picking the school with the highest pass rate could conceal the fact that no students at all achieved A grades in that school. In that case, you don’t want to send them there. Equally, if your child is most statistically likely to get Ds given their KS2 performance the 5 GCSE ‘pass rate’ isn’t telling you how likely your child is to secure those Ds as opposed to Fs or Gs.

One of the metrics Allen & Burgess suggested as more useful for parent choice is a ‘percentile’ measure.  Each school would report the average results for its students who enter with KS2 scores in the 20-30% range (low), 45-55% range (medium) and 70-80% range (high).  The scores should be in grades so parents can easily compare. For example:

 

Low: EEDDD

Medium: CCCCCCC

High: BBBBAAAAAA

 

Publishing this information in the league tables could therefore be considered a way of ensuring all schools support and challenge students across the ability range.

BUT, BE WARNED! The DfE Measure does not follow this advice. The Statement of Intent (p.4) shows ‘Low’ will be pupils below Level 4 (the average score at KS2), ‘Middle’ will be pupils at Level 4 at KS2, and ‘High’ will be pupils above Level 4.

MEDIA HOUNDS TAKE NOTE: The changing of this measure means it cannot be used to compare across schools.  ‘Low’ at one school could mean a much higher number of students entering on Levels 1/2 where another school only has students on Level 3 in that category. Comparing scores for ‘low scoring’ students when those students have such significantly different entry scores is mathematical gibberish.

EVEN MORE IMPORTANT: The thing no-one has mentioned is that Allan & Burgess found that making a school choice based on these ‘ability pass rates’ didn’t actually help parents make better choices. In fact, in the case of bright kids it led to worse decisions.

Allen & Burgess also note: “.. (T)he best performance information is only slightly more useful in school choice than a school’s composition, measured by the average prior attainment of pupils entering the school”.  That is, the best indicator of how well a kid will do when they leave school is how many bright kids they entered with.  Allen & Burgess say this is not because of a ‘peer group effect’ but because schools with high scoring kids tend to have the most resources, the most voluntary help, more stable teacher turnover, and more experienced higher-quality teachers.

The measure will therefore not support parents in their decision-making at all. The only reason it is being added is to ensure schools focus on students of all abilities – which is definitely something they should do.  But this is a measure being introduced to deal with the perverse effects of another measure.  Does anyone else get the feeling this could go on indefinitely?!