Analysis of the papers can take many angles and I don’t hope to cover them all in this series of blogs; instead, I intend to use analysis of the reading SATs to allow for insights into our current teaching practices in the hope of further developing and sharpening classroom pedagogy.
If I miss anything of particular interest, please do get in touch, and I will try to explore that avenue.
No change
The pass mark of 28 remained the same as in 2018. In real terms, this means that children need only have had a good stab at texts 1 and 2 to gain the marks necessary to meet the Expected Standard (EXS). Full marks on the questions relating to text 1, plus just over half marks on questions relating to text 2 would have reached the magic number.
With this in mind, securing high scores on these two texts becomes paramount for those pupils who you know are quite capable of reaching the EXS, if not quite ready to achieve GDS. It might be argued that encouraging these readers to focus more time and attention on the earlier texts – making sure that they give themselves enough time to read them well and answer questions with care – is preferable to challenging them to rush through the paper in order to get to the end. Lost marks due to rushed reading and careless mistakes on texts 1 and 2 may be a challenge to recoup through hard-won marks gained later on in the paper.
Does word count matter?
To answer this question, it is worth exploring how this year’s test compares with past papers.
According to Tim Roach’s (Twitter handle @MrTRoach) analysis of word count, we are right to feel that this year’s test was a weighty beast.
As his word count analysis shows, the 2019 test comprised of the most words ever presented in a KS2 reading test paper.
Can this increase in word count account for the drop in attainment at EXS at national level from 2018 to 2019 (a decrease of 3 percentage points)? And, does this national dip signify a real decline in reading standards?
In response to my first question: I would argue ‘yes’.
In response to my second question: I would argue ‘no’.
If we stick with the notion that pupils who we expect to reach the EXS need only make good ground on texts 1 & 2 in order to meet the necessary pass mark, in 2018, children had to do substantially less reading than in 2019.
To help exemplify this point, I have taken the liberty of amending Tim’s analysis grid, adding in the total word count for texts 1 & 2 only.
Now we see, with burning clarity, the reading demand made by the length of texts 1 & 2 in 2019 compared to 2018. In 2018, children had to read far fewer words (491 fewer to be precise) in order to be in with a chance of gaining enough marks to meet the EXS, than in 2019. In reality, this means that if you consider an average fluent reader – and we assume that they read a text at 150WPM (obviously this is a contentious figure and can fluctuate depending on task, context and child) – then, in 2018, a child would have had the luxury of over 3 minutes longer to spend on answering questions. This may not seem all that significant, however, any teacher will tell you that when it comes to the reading test, every second counts. Moreover, when we factor into the equation the visual impact that fewer words would have had on the confidence and enthusiasm of a reluctant or less test-savvy child, then we begin to see that the 2018 paper may have presented a more accessible challenge to most pupils. This in itself may go some way towards explaining the 3% point drop in national attainment at EXS from 2018 to 2019 – essentially, we are beginning to see that the children had to work much harder in 2019 to reach the EXS than in 2018.
So, if looking at data for EXS for the 2018 and 2019 tests doesn’t offer a fair comparison, where can we look to see if the children are getting better, or worse, at reading?
To support with this, we can turn our attention to the word count for the first two texts in the 2017 reading paper. We note from the grid above that in 2017, pupils aiming for EXS would have had to read a similar amount of words as in the 2019 test (with a difference of only 56 words). If we attribute challenge to word count alone (I am sure we wouldn’t but in fact, my analysis shows that the texts 1&2 from 2017 and 2019 were comparable in many other ways beyond word count – this will become apparent across this series of blogs), then we would anticipate a similar level of attainment at EXS for both these cohorts. Indeed, the national level data shows that 72% of pupils met the EXS in 2017 compared to 73% in 2019 – showing a small but notable 1% increase during this time.
We could therefore be bold and argue that, according to this data, children have actually improved their reading ability during this time period. If we take into account the fact that the pass mark has increased by 2 marks during this period, we may feel even bolder in our claim.
Pass mark madness!
Bearing in mind that the 2017 and 2019 tests presented a very similar level of challenge, you would expect the pass marks to have been the same. Unfortunately, they weren’t.
Imagine if the pass mark for 2019 had remained as it was in 2017 e.g. 26 rather than 28. If this had been the case, I believe we would be celebrating a rather healthy rise in standards, rather than lamenting a dip from 2018. With this in mind, a would urge schools to reflect on their data in the following ways:
- Calculate the data for pupils reaching the EXS in 2019 if the pass mark had been set at 26 marks.
- Compare this figure with attainment at the EXS from 2017.
If this comparison looks favourable, I would argue that whatever you are doing to improve reading is working.
To clarify, I would argue that the 2017 test is more comparable to the 2019 test than it is to the 2018 test: in 2018, children had to read considerably less in order to have covered the information needed to answer the questions relating to texts 1 and 2, thus giving them a better chance of reaching the EXS. With this in mind, it might be fairer to look at the comparable data from 2017 to 2019 when making judgements about reading decline or improvement. When comparing these two tests, we note that there was an increase in national reading attainment at the EXS by 1%, despite children having to reach a higher pass mark. This means that despite the challenge of the test remaining high, and despite a higher pass mark, more pupils than ever met the grade this year.
Let me therefore be amongst the first to say Bravo to all the teachers out there. Let’s be sure that we are all armed with this information when we are told repeatedly – as no doubt we will be – that reading standards are declining.
I hope that this blog provides enough to get teachers thinking about some of the ways in which they might interpret the findings of this year’s reading SATs paper outcomes, and how their findings might be translated into effective classroom pedagogy.
Penny Slater
