A Simple Test for Gove’s “Soft Bigotry” Claim
14th November 2012
There are likely many ways to test for this bias but here’s a simple test for which the data is readily available: Use exam board data to determine which schools had out-of-tolerance controlled assessment marks due to ‘undermarking‘ and determined via statistical testing if these schools have significantly more ethnic minority students than schools who either marked within tolerance or who over-marked. If teachers systematically under-mark ethnic minority students then one would expect to see schools with high levels of ethnic minority students over-represented in the ‘under-marking category’ and under-represented in the ‘overly generous’ category.
- The CMPO Bristol study used to support Gove’s point does not say that all ethnic minority students are equally under-marked. In fact, some are over-marked. There are also local variations that matter – e.g. biases appear to occur more in areas with fewer ethnic minorities – and these variations may then influence the results of our test. However, the DfE knows the ethnic minority breakdown for each centre meaning these variations could be controlled for. Other checks or variables could also be considered – e.g. subjects, ethnic minority variance by area, etc.
It may be that teacher’s prejudiced under-marking is only slight and ‘within tolerance’. I.e., if students are only under-marked by, say, 5% then the mark will be accepted by the exam board even though the moderator would give a higher mark. However, if the issue is as systematic and unconscious as Gove suggests it is highly unlikely that teachers carefully calculate their biased undermarking to be within such strict parameters. It is much more likely that bias runs through all results and would still be picked up by the test.
The data to do this is definitely available. Ofqual’s Appendices of the English GCSE Report gives the number of centres in each exam board who were under-, over- and within- tolerance for the controlled assessments. Every examination series will have similar figures to hand. The centre numbers for the schools in each category should be easily extractable and could then be run against cohort demographic numbers held by the DfE. It may not be the easiest job for someone, it might take several weeks to get the data in place and thoroughly check the statistics, but what could be more important?
It’s also possible that this test isn’t so robust – if you read the CMPO paper you’ll see how complicated things can get when looking at assessment data. So let’s get into it, if the suggested test isn’t good enough, let’s really examine the numbers and find out if the claim is correct. And if this test won’t do it let’s find one that will. Because if Gove really believes that controlled assessments are being undermined by racism (and I don’t think it’s an entirely outlandish claim) then he MUST provide strong evidence otherwise we are going to end in a spiral of ‘you are doing this’ and ‘no we’re nots‘, all the while forgetting that there is a serious potential here for teachers to be systematically (albeit unconsciously) undermining the life chances of some students.
Read our blogs? Please let us know their impact and how we can improve by taking our quick survey here: https://www.