How is it possible that no-one in England knows what a C-grade answer looks like?
11th September 2012
But an even bigger part of the puzzle needs to be answered: How is it possible that no-one in England can agree what a C grade English GCSE answer looks like?
As teachers we mark hundreds of pieces of work each year, often giving a grade suggestive of what a student might expect to get at GCSE. On what do we base our decision? It may surprise non-teachers to know that we don’t really look at student’s work and then ‘grade’ it as one might expect. The way exams and controlled assessments are measured means that a student gets a set of numbers for achieving different tasks and those numbers add up to grades. Hence, for most controlled assessments (and lengthy exam questions) a teacher scores the work according to a mark scheme (almost a checklist for scoring). For example, if an essay in history includes a certain number of sources, are evaluated in a certain way and show analytical skill the student gets 40 out of 50. With fewer sources and less evaluation they might only get 30. [I can see the teacher-readers nodding at this point]
However there is no mechanism for turning this number into a ‘grade’ other than using the published boundaries from the exam board. So, if I know that in the previous year 40 marks gained an A then I can tell that to the student and so 40 marks becomes the quality standard for an A, if a 30 was a B then my other student gets a B.
If I was a non-educator I might say here: Why turn it into a grade at all? Why not operate in numbers and leave the grade-turning-bit for the exam board? Good question.
The problem is that the student’s college wants a predicted grade, parents want to know what grade their child is achieving at, Senior Leadership want to know what grades my students got in their most recent assessment and so teachers are constantly having to convert numbers – that don’t mean anything – into grades even though we have very little insight into how grade boundaries are actually decided and we don’t have any tool for doing that other than the prior grade boundaries.
What the GCSE fiasco has done has shown, in quite a unique way, that the only way we can ‘know’ the likely grade for a student is by looking at grade boundaries from the past and if – as happened this time – those are allowed to vary wildly and even within the same exam session, then teachers are left without any way of accurately grading students. But given that grades are a vital part of our accountability, our data systems and – depending on future reforms – our pay, then the idea that we can no longer have any confidence in where grades will be is a problem. Even more of an issue is that it means we cannot figure out which student we need to target for support. Schools have been pushed in the direction of data-driven decision making and so most students use the teacher predicted grades to target resources at students most likely to fail. But if the predicted grades are based on inaccurate grade boundaries this throws the entire system and so, it seems, because teachers could not accurately assess students’ likely grades the whole data-driven decision-making has been affected.
What is even more concerning than teachers being unable to accurately say what grade quality of their students work is that the exam boards professed not to know where the grade boundaries were. As a teacher you are expected, even in the first year of marking an assessment, to accurately assess your students’ work using only a mark scheme and the work of your one (maybe two) classes. Those marks will not only go to the exam board in their raw numerical form, you are also expected to convert them into grades and hand them into school for evaluation purposes. We all do this and although teachers are sometimes out it is usually only by a small percentage.
In January though while many thousands of students had their work assessed by the exam boards they (and Ofqual) maintained that the numbers could not be accurately turned into grades. This has to be incorrect and, increasingly, the evidence looks like it simply wasn’t true. Remember: the examiners who run qualifications are – in general – highly competent at what they do and AQA in particular are a thorough and well-run operation. It has always seemed highly suspicious that the exam boards simply ‘did not know’ what a C grade looked like but – if it turns out that they didn’t – and no outside pressure was applied then there really is a very important question here: How on earth is that possible? How is it possible that even the most senior assessors (all of whom will have been teachers) did not know what a C grade answer looked like? Have we really lost grip of education so terribly that NO-ONE knows what a C grade looks like?
Somewhere Gove is sitting waiting to pounce on this to make a reform announcement and I am sure we will rake over his changes time and again. But among all that analysis I hope that someone asks him this: How will the new reform help teachers know what C (or B, or A) grade quality is? Because if teachers don’t know then we are the blind leading the blind, and no matter how ‘rigorous’, or dull, or comparable-to-Singapore our new qualifications are, there is still the same risk of this happening all over again.
Furthermore, we also need to ask questions about the children currently in our classrooms who cannot wait for his reforms. So, if you get the chance, ask this: From now, on what basis should teachers estimate a student’s grades? If we can’t use published grade boundaries, what should we use? Don’t accept answers about ‘experience’, because some teachers are new and they need to know. Don’t accept answers about ‘professional judgement’, because we need to know on what evidence that judgement should be wielded. And don’t accept answers about ‘in the future we will…’ because it simply not acceptable that the teaching profession does not know how to grade its students – too much of their future relies on it for it to become something that we no longer understand. In the end, I don’t care whose fault it was that our ability to understand grades has been swept away – whether it was Ofqual, Gove or Balls – the aim now must be getting it back so that we all have at least some concrete basis on which to proceed.
2 thoughts on “How is it possible that no-one in England knows what a C-grade answer looks like?”
Great post, I think you’ve put your finger right on the reason why criterion referenced tests are so much better than norm referenced ones. It’s a shame that there isn’t public or press understanding that grades on criterion referenced tests will naturally increase on average over time as teaching improves.
As you saw from this morning’s session this is a bit like swimming through treacle with a lead weight attached to both your legs!
I suspect Gove will very diplomatically suggest that teachers are only doing their best for their students but that to ensure standards are maintained for everyone then the regulator has to be perceived as being tough and that this has worked as Ofqual did not require any Awarding Body to change its grade boundaries. However they all know it has powers to remove their licences to operate and this is a completely different environment from a few years back.
I agree that the Committee now needs to grill Awarding Bodies and I suspect they will have a long list of reasons why GCSE English grades were under pressure to rise this year. The debate will then move back to the bigger issue of how many Awarding Bodies there should be and what powers should Ofqual (and Parliament/Ministers) have.