Quantcast
Channel: Kent Independent Education Advice
Viewing all articles
Browse latest Browse all 516

Comment on Report claiming 'The 11-plus is a loaded dice'

$
0
0

A story on the BBC website features a Report that offers misleading and irrelevant data relating the Kent grammar school selection process, issued by Education DataLab (EDL). EDL has built this on information collected by the nebulous Kent Education Network (KEN), the link underlining the misuse of statistics by KEN in its passionate opposition to the existence of grammar schools in the county, so hardly an objective source of data. The title of the Report, ‘The 11-plus is a loaded dice - Analysis of Kent 11-plus data’, is itself highly pejorative based on the false claim in the document that there is an arbitrariness in who passes the Kent Test, although no doubt designed to capture headlines.

Education Datalab describes itself as a research organisation that produces independent, cutting-edge analysis of education policy and practice. Employing Joanne Bartley from Kent Education Network as one of the authors of the Report completely destroys any claim to independence or objectivity in this case.  

The Report purports to make proposals to increase the proportion of children on Free School Meals being identified as suitable for grammar school, although this should surely have been indicated in the title, and some of its ideas would have precisely the opposite result. The central proposal of change to achieve this, by changing the assessment process for suitability to grammar school, would actually introduce an additional and unnecessary unfairness for all children into the process, and remove the current requirement for children to reach an appropriate standard in English and maths.

Kent County Council produced a highly regarded Report last year prepared by a Select Committee of KCC on 'Social Mobility and Grammar Schools' (SMGS) published last year, which contains a range of excellent Recommendations that have been adopted in full by the Council and are currently being worked through. These Recommendations in my view offer a far more rational way forward than the mixed bag contained here, although oddly the EDL Report contains no reference to them. However, as I can see no chance of the EDL Report being adopted for the reasons below, I am disappointed it has gained such publicity.

The BBC article highlights the statement that 12% of Free School Meals children passed the Kent Test in 2015 when compared with 30% of their better off classmates. Cleverly, the Report omits the important qualification 'of those children who took the test', rather than using data for all children in the age group, which exaggerates the differential, for in the 2015 Test, the one being 'analysed', just 20.0% of all Kent children in the age group passed the test as my analysis at the time shows. It is a false, misleading and irrelevant comparison as it takes no account of the relative abilities of the two groups of children, nor of the make up and proportions of the sub-groups who chose to apply for the Test. Instead, I look at an appropriate measure of success at gaining grammar school places for children on Free School Meals (FSM) below, taken from SMGS, and comparing children with similar abilities, which is important OBJECTIVE evidence and so should surely have been quoted in the Report but was not.

The central recommendations of the Report are:
  • Kent should consider implementing an aggregate pass mark in its 11 plus test to reduce the proportion of pupils, just one mark away from passing/failing;
  • Kent should consider removing the headteacher assessment panel and correspondingly lowering the pass mark on the test.
Replacement of the Kent Test by a once only aggregate score assessment
The EDL analysis proposes the unexamined and flawed assumption that selection for grammar school should be based on a single test performance of children with the highest aggregate scores, without considering the disadvantages of this scheme for children. Both these recommendations remove some of the key strengths of the Kent process without serious analysis of the latter. The Kent Test currently requires children to reach a standard in each of the three elements of the test, Reasoning, English and Maths, together with an aggregate score, currently 320. This ensures that grammar school entrants have a basic standard in English and mathematics. According to the EDL Paper this is apparently a BAD THING, although I can see no rationale for this provided.

The Report makes great play about the unfairness of some children with an aggregate score of 320 not getting a grammar school place as they have failed to show grammar school ability in one or more elements. Quite correct; that is the point of setting a minimum standard in each subject to ensure grammar school entrants have an appropriate background across the board and so able to meet the wide academic demands of the education offered in grammar schools. The conclusion about the Kent Test that: “The test is identifying children who are (highly able) all-rounders, then, rather than children with particular aptitude in only one or two areas” is clearly meant as a criticism, but is indeed at least partially correct, although I have seen many children in the second category with basic skills across the board awarded grammar school places, so is false in that respect.  

Headteacher Assessment
What the two proposals achieve together is to turn the Test into a ‘one chance only on the day’ Test, a concept already often erroneously alleged by KEN to be a criticism of the  current set-up. This is to be achieved by abolishing the Headteacher Assessment (HTA) aspect of the Kent Assessment system, which selects a further 4% of children on top of the 21% found of grammar school ability by the Kent Test.

HTA takes into account: children’s work; a piece of written English completed at the same time as the Kent Test; a reference from the Primary Headteacher; and the marks in the Kent Test. Most importantly it can also be used for the school’s headteacher to explain personal circumstances that have played a part in underperformance, which can feature any element of disadvantage. It therefore allows a second chance for those who may have underperformed in the Kent Test and for disadvantaged children. Instead the Report proposes replacing this by lowering the pass mark, which of course simply has the effect of re-introducing this same rightly hated ‘one chance only’ effect for a different group of children.  

The EDL Report acknowledges that HTA is biased in favour of children on Free School Meals showing that it does work for disadvantaged children, but then criticises it because it delivers insufficient numbers, so it is unclear why it wishes to abolish it. Surely, on this basis, the way forward would be to extend HTA. The Report quite rightly criticises some primary school headteachers for failing to offer appropriate support to children who ought to be put forward for HTAs, perhaps like KEN because they do not believe in the selective system. However, it is clear that selection in Kent is here to stay for many years and headteachers have a responsibility for ensuring the best for their pupils. I have worked with too many families where primary headteachers have failed their pupils in this way to be blasé about this problem. The solution is surely to challenge such schools by those in authority to ensure they make the system works effectively, not undermine it. Indeed, the SMGS identifies work with primary schools as a priority. This includes amongst the 15 sensible recommendations: “Urge all Primary Headteachers to utilise Headteacher Assessment Panels within the Kent Test process to advocate for those most academically able children supported by the Pupil Premium”, along with powers to do so. You will find my own comments here 

Some children will be being passed or failed incorrectly’
This is the heading of one section of the Report, and the article appears to suggest there is precision and accuracy in defining children’s ability, by placing them in a line according to their suitability for grammar school, which is totally opposed to so many statements by KEN! Having proposed this false precision, the Report then criticises the Test for failing to deliver it: ‘In fact, no 11-plus test will ever sort children perfectly, even if we were to ask 10-year-olds to sit a test every day for a whole month’.  It then goes on to analyse the problems of the incorrect assessments it has artificially created, by exploring what it claims are the probabilities of pass/fail in individual tests. Yes, there is a band in which borderzone children can be passed or failed by the Test, but grading them solely on a single mark is grossly unfair and has no element of correctness. With the current system, it is the function of the HTA or grammar school appeals to look more closely at these situations. Both these processes rightly depend on judgments by independent panels, but there should be no correct or incorrectness in judgments. 

The section concludes with a proposed surrealistic outcome for parents, whereby instead of the straightforward pass/fail decision they are provided with at present, they will receive a report from KCC with: ‘alongside the letter stating whether the child had passed the 11-plus, parents are given an additional piece of information – the probability that they have been misclassified by the test’, still obsessing about correctness. I just don’t get this, although the Report talks about a concept called Classification accuracy. The information might include, as per the examples given: ‘one parent might be told their child had passed, and yet the probability she should, in fact, have failed was 39%. Another would be told their child has failed, but the probability he should have passed was 47%’. Quite what the point of this is I cannot see, I don’t believe in it in this context, and what parents are supposed to make of it is completely beyond me. Subsequent school appeals take many important factors into account, and I really cannot visualise this one being of any conceivable interest to Panels.

The Reasoning Element of the Kent Test and Tutoring
Back in 2014, in an analysis of Kent Test scores, I identified that unsurprisingly the Reasoning element of the Kent Test was producing the highest scores although results are nationally standardised, presumably as a consequence of tutoring which is easier for this Test. This Report identifies that children on FSM are likely to perform slightly better in Reasoning than in Mathematics, using comparative performance against non FSM children. This gives the headline figures that FSM pupils taking the Kent Test underperform other children by 7.7 points in the Reasoning Test as against 6.8 points in the maths (the figure for English is 3.9) which the Report considers highly significant. Another two tricks of statistics; firstly, these are figures comparing numbers greater than one hundred, so the difference between the two subjects is less than 1%. Secondly, for some reason, again presumably for headline purposes, it emphasises once again that FSM pupils taking the Kent Test achieve less well than non FSM pupils overall, which still proves nothing either way, as there is no comparison of relative abilities or of the way these groups self-select to take the Test.
 
 
Performance of Children on Free School Meals
As noted above, the statement that 12% of Free School Meals children passed the Kent Test in 2015 when compared with 30% of their better off classmates may well be true but it is a false, misleading and irrelevant comparison. The best comparison I can find is that is that 57.4 % of children on Free School Meals Ever (my preferred measure) who have achieved at least two Level 5’s in their KS2 SATs begin grammar school, against 78.7% of similar ability children not in this category. This of course comes from the SMGS Report, also looking at 2015 entry, when the Level 5 concept still applied. Yes, there is an unacceptable gap between these children of similar abilities, but KCC is working hard to close it in conjunction with many primary and grammar schools, as shown by examples already happening in the Report. Why no credit for these, or practical advice on how to do so?
 
Test Preparation in Primary Schools
There is one good recommendation in the Report that I would like to see KCC adopt, which is to allow all primary schools to provide up to 10 hours Test preparation for all children who wish it. This is also the level of practice recommended in the only piece of research I have ever seen into the effect of coaching, now quite dated. I don’t believe it can take place in class, as it is only relevant for a proportion of children, who are self-selected, but could easily be offered after school as a voluntary activity and I can see no legal reason to stop this. This would help to reduce the disparity with those who are privately tutored or attend those private schools geared to secure success in the Kent Test.
 
The Gambling Analogy
Even a superficial reading of this report confirms that most children are appropriately identified by a combination of the two main routes of selection. The lead claim of the report is that ‘We say that getting into a grammar school in Kent is akin to rolling a dice because of the arbitrariness of who passes the test’. The use of the word arbitrary, claiming that the process is valueless to get a good headline, is completely wrong, foolish, and offensive to those children selected. The introduction of the concept of rolling a dice to emphasise that devaluation may well come form the the gambling industry, but it has no place in what claims to be an objective analysis. What it does do is to generate false headlines such as the one in The Times that reads: ‘Biased 11-plus is no reflection of ability’, which could of course have been its intent. Even a superficial analysis of this headline shows its error. Most definitions of bias refer to prejudice and a deliberate action to support a point of view. The Report itself contains no suggestion of intent in any unfairness in the system, so bias is an inappropriate word. ‘No reflection of ability’ is self-evidently just plain wrong!
 
Grammar School Provision
The Report claims that Kent has 11 super selective schools out of the 32 in the county, defined those ‘which make use of 11-plus test scores to prioritise applicants for admission, either ranking all applicants by score, or prioritising those who have scored above a given level’. By this definition, I can count just 8: Judd, Skinners and Tonbridge; Dartford Boys and Girls; Cranbrook; and Maidstone and Simon Langton Boys Grammars. Perhaps just a careless error, but again, one that becomes replicated in media that have not carried out an elementary check on accuracy.
 
Half of these - Dartford Girls, Judd, Skinners and Tonbridge Grammars - have introduced priority for a proportion of Pupil Premium or Free School Meals children, as have a number of other grammar schools, and I anticipate this will extend further for admission in 2019. 

It’s also worth noting that passing the 11-plus is not enough alone to gain entry to any grammar school of choice’. Of courseit is not! The outcome with regard to places is no different for the rules for any oversubscribed school of any type, so why suggest it is different and of interest unless one is showing bias, but succeeding in further discrediting the Report.  

Meopham Grammar School?
Amongst the plethora of data (along with a multitude of follow-ups) that Joanne Bartley collects from Local Authorities and schools across the country regarding selection to grammar schools, there is an important question to which I also would like to know the answer although it is irrelevant to this article. Back last September, the Swale Academy Trust proposed changing the status of Meopham School in Gravesham to become a grammar school when and if the rules changed and instituted a Consultation. She has vigorously  pursued the results of this Consultation and indeed was promised a Report back on January 18th. Since then the Trust has illegally ignored her further requests for the information. What are they trying to hide? Perhaps the ridiculous nature of the initial proposal as demonstrated in my article written at the time. 
 
Conclusion
I have only reported on parts of a very lengthy Report that attempts to blind readers by the complexity of analysis. Many of the illustrations I have given above make no sense, illustrate false conclusions and appear counter-productive in terms of narrowing the gap for FSM children looking for places at grammar schools. If you have read this far, please consult the SMGS Report that contains many excellent and realistic proposals for closing the gap, from  a Local Authority committed to do so, some of which are coming to fruition. Sadly, it is fair to note that there are a number of institutions both at primary and secondary level, who have no interest in promoting these, and for whom there appears no sanction to force them to do so.

Given that the New Grammar School policy now looks likely to come to fruition with a Conservative victory in the forthcoming election (I have grave reservations about the policy), I presume this analysis is ostensibly designed to advise the new institutions on the way forward. However, it is clear they will be of a different character to our present grammar schools with a built-in commitment to supporting disadvantaged children, so little here will apply. Kent is a county with grammar schools operating a single base system of admission, although there are a variety of add-on differences. This is not transferable to individual institutions with individual selection processes operating to the new rules within a comprehensive school set-up. However, it may serve to warn those institutions of the perils of its recommendations. Finally, I consider the analysis is far too flawed, with too many false conclusions and errors to be useful. In practice it is clear there is a different agenda. 

I can see that it has whipped up considerable anti-grammar school feeling in parts of the media which may well have been its aim. Typical is the report in The Times, which claims amongst other misinterpretations that: ‘Only children doing exceptionally well in all three papers will be given a place at grammar schools’, which is clearly a nonsense.  I make no judgment about the rights or wrongs of the selective system here, I have simply looked at the facts.

I operate alone and part-time, producing this in short time, in order to respond to the issues raised, but I accept to late to influence them. As a result, I also accept that I may well have made errors myself in this analysis, and if so am happy to correct them.


Viewing all articles
Browse latest Browse all 516

Trending Articles