Official Report: Minutes of Evidence
Committee for Education, meeting on Friday, 14 August 2020
Members present for all or part of the proceedings:
Mr Chris Lyttle (Chairperson)
Ms Karen Mullan (Deputy Chairperson)
Mr Maurice Bradley
Mr Robbie Butler
Ms Catherine Kelly
Mr Daniel McCrossan
Mr Justin McNulty
Mr Robin Newton
Witnesses:
Mr Weir, Minister of Education
Mrs Faustina Graham, Department of Education
Mrs Karen McCullough, Department of Education
Mr Bill Stevenson, Department of Education
Summer 2020 A-level Examinations: Minister of Education
The Chairperson (Mr Lyttle): Members and witnesses, you are very welcome to this meeting of the Education Committee. We are delighted to have the Education Minister with us this morning. Clerk, are there any apologies to record?
The Committee Clerk: Just William Humphrey.
The Chairperson (Mr Lyttle): OK. Minister and members, may I apologise for the delay in starting the meeting, which was due to quite a significant technical difficulty? We will now proceed promptly.
Minister, before you begin your oral briefing, may I ask, in these exceptional circumstances and due to the delay, that we make a slight procedural adjustment? Can I ask you to focus your comments on the grade awards 2020 and that you give a commitment to return to the Committee early next week to address the equally urgent matter of school restart in more detail? Is that possible?
Mr Weir (The Minister of Education): Yes, I can do that. In my opening remarks, I had intended to address both issues. Given that this meeting was principally set up on school restart, most of my remarks were focused on that. However, I am happy to tic-tac with the Committee. I do not know whether the meeting can necessarily be scheduled for early next week because of diary commitments, but I will see what I can do.
The Chairperson (Mr Lyttle): I appreciate that. Obviously, if you want to make some key comments about school restart, we will endeavour to facilitate that. Understandably, there will be a number of questions from members about grade awards 2020.
May I ask Committee members and witnesses to bear in mind the well-being of children and young people across Northern Ireland and for our actions to be guided with them in mind? They have had significant challenges throughout COVID-19, which have been compounded by this difficult week.
On that note, Minister, I invite you to make your opening remarks.
Mr Weir: OK. As I say, I have extensive remarks on school restart, but I will leave those for another occasion.
Yesterday, young people in our schools received their AS and A-level results. It is important to put on record our congratulations to all those young people on their achievements. While we focus on the various issues arising, it is important that we take time to congratulate those young people on their achievements. That reflects not only on them but on their parents and teachers for supporting them in what are challenging times.
There has been an increase in those achieving grades A* to C across all school types, which reflects an upward trend in improvement across our education system. Specifically, in A-level results, we have seen an overall increase of 1·6% in those figures, up to 86·4%, and, at AS-level, there has been a 2·2% increase, up to 77·3%. It is also the case that, when you analyse the specific aspects of those results, it is effectively an improvement across each of the grade sectors. There has been an improvement in the number of A*s and in the number of A*s to C, and a similar pattern can be shown for AS levels. Perhaps significantly, when we are looking at the issue of attainment, there has been a significant decrease in the number of U grades awarded in A level and AS level. Those entries that have received a U grade at A level have gone down to 0·9%. In practical terms, a little under 200 papers out of 26,000 have been awarded a U grade. By comparison, that was at 1·7% last year, so that has nearly halved in the space of one year. For AS level, where there is a higher percentage of U grades — and there always has been — it has gone down from 4·5% to 3·6%.
That is also reflected across all school types. For example, between the sectors at A level, from A* to C, we have seen a better performance from non-selective schools, where the gap has closed by 1·9% in the achievements, as compared to that of selective schools. The position is more stark at AS level, with a 1·7% improvement at grades A to C for grammar schools, while non-selective schools have seen their grades improve by 7·3%, which means that the gap has closed from around 17·5% to around 12%, which is a significant reduction.
It is important that, in this debate and discussion, we do not undermine or fail to recognise those who are celebrating their achievements. There are elements within this story that need to be told as well. I am confident that the vast majority of our A-level students now have outcomes that will allow them to progress their plans for their future. However, I appreciate that concerns have been raised that the system has not worked for everyone. There will be some disappointed young people who feel that we have not achieved the stated aim of ensuring that everyone receives a fair outcome that reflects their hard work.
Everyone who was engaged in this process was focused on doing their best for our young people in what are extraordinary terms. That includes taking a long-term view that not only do these grades need to be fair this year but that they need to be fair to those who got an outcome last year, and we must ensure that these qualifications will stand up to scrutiny in future years so that there is no long-term detriment to those young people who got results yesterday. We also need to bear in mind, particularly owing to the COVID-19 disruption, the position of A-level and AS-level students who will complete their studies in 2021.
As far as was practical, we wanted to be in a position to demonstrate that a grade A that was awarded this year, without exams, was every bit as robust and valid as a grade A that was awarded last year and that the same grade in different schools was of a consistent standard. That is why I had decided that there needed to be a standardisation process for examinations, as happens every year in some form.
My Department and the Council for the Curriculum, Examinations and Assessment (CCEA) were in regular contact with their counterparts in England and Wales as GCSEs and A levels operate in a three-country model. It was therefore agreed that our young people would be best served if we aligned our processes as far as possible to take account of policy differences. It is a little bit disappointing that Wales departed from that approach to some extent at the last minute and without consultation.
As part of our process to develop a solution for Northern Ireland students, representatives of head teachers, teaching unions, students, managing authorities and the Education and Training Inspectorate (ETI) were all consulted before we reached the point at which there were final decisions on the various models. Different process models have been put in place for A levels, AS levels and GCSEs. Everyone recognised that there was no perfect solution. The best solution would have been examinations taking place but that was not practical this year. Arguably, therefore, there was no best solution and we were looking at what we could create as the least worst solution. No one suggested a different workable process for delivering fair, robust grades in the space or time that was available.
CCEA did what was asked of it, and I believe that, overall, the outcomes are valid. I accept that not everybody feels that way so I want to give an assurance that the appeals process will allow schools and young people to explore every avenue that needs to be explored to understand the outcome that they have been awarded. In particular, this year, there will be a widening of the appeals process. Normally, the appeals process deals purely with process and procedural issues. This will take into account what evidence can be submitted through schools for either groups or individuals, where prior attainment can be used as evidence to suggest that someone should have received a different grade. Where that can be shown within the appeals process, the appeals process will reflect that, leading to alterations in the A level. I have asked CCEA to publish additional information on how the process worked and to include exemplar material. It is important that as much information is given as possible.
I appreciate that the Committee will have a wide range of questions. I will be happy to respond to those. You will be having the chief executive of CCEA as well. There may be some technical issues on which he is better placed to go into direct detail than the Department but we will try to answer questions as best we can, Chair.
The Chairperson (Mr Lyttle): Thank you, Minister. May I start by asking if you wish to apologise to the pupils who have been so distressed by the downgraded results that they received yesterday?
Mr Weir: I feel very sorry for and understand the disappointment that people have. At the heart of this must be an understanding that no one has been downgraded. There has been a difference between what teachers' assessment felt pupils should be getting and the actual results. Albeit that we are in very unusual circumstances, that has been the same for a number of years. Uniquely, Northern Ireland, for the last number of years, has got teachers' assessment of what they think people should be getting at A level, AS level and GCSE. The only difference this year is that it is slightly more comprehensive; normally, that accounts for roughly 60% but it is now 100%. So, no one has been downgraded.
Now, within that, there are clearly individual cases where the system and modelling will not have produced the right result. We need an open, robust appeals service to be able to take account of and look at those cases on an individual basis, rather than simply applying a system. Every year, there will be expectations for pupils and disappointment from pupils. I can understand that it is more difficult for anyone in that position to take it this year because, rather than someone having the black and white result of, "Here is the pure result of an examination", they have, "Here is the result of a system". It is undoubtedly a lot easier for people to accept when it is an examination.
Minister, since 16 April, when you announced the cancellation of exams, you have received robust questions about the alternative model that you have chosen, particularly questions relating to the use of rank ordering and past school performance. I have minutes from the meeting of the Education Committee on 3 June; the Committee noted significant concern in respect of the statistical model used to inform the process of awarding grades, that the model had yet to be fully developed and subject to any testing, and that its characteristics and method of application had yet to be explained fully and communicated clearly. The Committee also felt that the professional teacher assessment was a reliable basis on which to proceed. Since then — we had our results yesterday — MLAs have been inundated with principals and pupils who have been shocked, confused and distressed by the outcome of that model. I have one example — I am sure that members have many — of a subject area in a school in which the grades for A* to C at A level this year are 30% reduced on last year; they went from 90% to 60%. Of 120 kids, 60 do not yet have a university place. There are examples of pupils at the top of their rank order receiving as many as two grades below that grade and yet others lower than them in that rank order still receiving the grade that they were originally rank-ordered in. This is a serious and significant problem, Minister. I put it to you that, however good-intentioned and however much hard work and endeavour went in to that particular model, you and CCEA have failed to explain in a clear and transparent manner the way in which the model has operated, and that that model has failed many pupils across Northern Ireland, and, indeed, has put university places at risk.
The extent of the failure is such that it cannot be adequately addressed by an appeals process in the timescale available. It seems that it is a failure that is likely to be replicated for GCSEs next Thursday, so I put it to you — I will ask the Education Committee to support this proposal — that, in these exceptional and unprecedented circumstances, during which young people have made unprecedented sacrifice, you act decisively and move to ensure that pupils are awarded whatever grade is highest from their AS-level grade, their teacher-assessed grade and their CCEA-awarded grade and that that same approach is applied to GCSEs for next Thursday.
Mr Weir: There was a lot in there. I will deal with each of those points, where possible, which may take a little bit of time.
Schools were given information in relation to the process, which is quite technical. However, on the ability to explain and communicate clearly, if, in a public sphere, the information is not clear enough and has not been clear enough, in part because of the complexities of that technical information, I accept that maybe we could have done more to explain.
I will deal with some of the issues that you raised. Past school performance plays no part whatsoever in the A-level modeling. There is not a single percentage point that relates to that. That is because there are other elements of reliable data that can be put in place.
The Chairperson (Mr Lyttle): That is well-established; it will apply to GCSEs, so I do not think that you need to labour that particular point, Minister.
Mr Weir: No, but you made reference to concerns, so I just want to put that on the record to make that clear.
Mr Weir: You said that professional teacher judgement is sound and accurate. There is no doubt about where we were at in terms of the statistics. I think that about 57·6% of the grades that were suggested by teachers accurately reflected what the final position was. Around about 96% or 97% of overall teacher bits were either that or one grade of difference. Around about 3% could be described as outliers, where there are two or more grade of difference. That suggests, however, that there is a considerable gap between what teachers predicted and what pupils received. In many ways that is not unexpected. Teachers, for very valid reasons, will tend to look more optimistically on the grade results than what actually happens in practice. We know that because, in the last number of years, CCEA in Northern Ireland, prior to results being issued, has been getting teacher predictions of what their students should receive. There may have been extra care taken this year, but, looking at previous years, the level of accuracy last year was 46%, the year before it was 44%, and on each of those occasions — this is a factual point — teachers were overestimating what they believed that their students should get, compared with the actual exam result. In each case, more than 40% of grades were at least one grade over. Last year, in 10% of cases, teachers estimated that grades would be at least two grades higher than what the student received.
This is not just a Northern Ireland phenomenon. It has happened in pretty much every jurisdiction where this has taken place. Whereas the gap between what teachers predicted and what students got was about 38% optimistic in Northern Ireland, I think that it was between 36% to 38% in England, and it was probably a similar figure, maybe higher —.
Mrs Faustina Graham (Department of Education): Broadly similar.
Mr Weir: It was broadly similar in Scotland, and even with, for example, the Baccalaureate in France, there has been a fairly significant indication. Therefore, it is clear that the professional teacher assessment is something that statistically and in practice, in different sets of circumstances, has been shown to be quite different from actual results, and there is going to be —.
The Chairperson (Mr Lyttle): Minister, I do not want to intervene too much — you will forgive me for doing so on this occasion — and I am keen to bring other members in as well, but how can the endeavour and contribution of teachers to predict and assess achievements on this occasion be compared generally with every other type of prediction to which you refer, given the exceptional circumstances that are at play this year?
Mr Weir: You are comparing, as much as possible, like with like. What we are saying is that teacher predictions this year have been quite different from what the actual results are. The gap has closed a bit, on the basis of their having been seen to be more accurate. The point is that if you are testing — using the A-level, AS-levels or other examples — there is a very similar picture. Consistently, year-on-year, what teachers predict for their pupils will show a large difference between what they are predicting and what the final result is. There is no doubt about that and that has happened not just here but elsewhere. That is the particular relevance.
In relation to testing, yes, there was a limited amount of time. For example, and Justin can probably go onto this in more detail, CCEA, in testing the model, worked not only with statisticians but with all those who are experts in the field. If you take, for instance, the AS-level results, where a particular model was put in place, effectively a dry run was done on what happened in 2019, where the modelling involved a mixture of past performance by the school and mean GCSE test results. Effectively, if we had abandoned exams in 2019 and applied all the data that was there in 2019, using this methodology would produce something that was exactly of the same nature as the actual results in 2019. From that point of view, it is not untested.
Forgive me, I know that you said quite a few things. There may be a few —.
The Chairperson (Mr Lyttle): The key point for me, Minister, and you used the phrase yourself in your opening remarks, is incorrect results. Do you recognise that the scale of the incorrect results that have been awarded is such that it cannot be dealt with adequately via an appeals process —
The Chairperson (Mr Lyttle): — and that you need to intervene to allow pupils to receive a grade based on their individual ability and the highest out of their AS-level grade, teacher-assessed grade or CCEA-awarded grade?
Mr Weir: Here is the point. In that regard, Chair, no, I do not. On the level of incorrect grades, if we put it that way, probably less —.
Mr Weir: Well, OK. Look, OK, we will maybe use it in relation to that. The point is that the differences, particularly between teacher performance and the final grade, were less than in previous years. On that basis, there will always be some grades that are incorrect. You mentioned the need for individuals to be looked at. That is why, actually, a robust appeals system that allows individuals and individual prior attainment to be taken into account is the correct way of doing it because that tailors it to the individual.
By contrast, if we took, for instance —. Let me make two points about what you said about, shall we say, pupils effectively being guaranteed the highest grade from three particular aspects — the AS level, the teacher assessment and the actual grade awarded. That would put us in a position of putting in each of the elements from Wales and Scotland and combining them. Within Northern Ireland, if we used, for instance, the teacher assessment, we would have a situation with A levels where the change of pass rate from A* to C would move up from 84·8% to 95·1%. That is even without taking into account the fact that there will be some beyond that who will be brought up by AS levels. We would not have any credibility. You would have a situation where, for AS levels, grades A* to C would be brought up from about 75% to about 91% or 92%. A leap such as that would mean that there would be no credibility whatsoever for the people receiving those grades, and that would count against those people in the long run when it comes to employment and when it comes to places.
It would not only be unfair between pupils in different years, but it would be unfair within years, because you would get some teachers who would be quite tough in their assessment and others who would be very, very generous. There would be no equality whatsoever. It would depend, from teacher to teacher, from centre to centre, on what attitude and approach that they took. That would mean that some pupils would get a large boost beyond what they have got at present. Sometimes that might be fair; sometimes it would not be fair. Others would actually be held back and their grade would not be changed. It would not create a level playing field between students. Yes, you can take the individual bits, but if you simply apply that blanket approach, you will lack any credibility in the examination qualifications. It would take us, for example, out of sync with England, which is about 85% of the UK market. As a small region, more so than anywhere else, if we damage our credibility and it is felt that the grades from Northern Ireland examinations are not really worth the paper that they are written on, that will be of massive detriment: "Oh, that grade is from Northern Ireland. That is not as good as a grade from somewhere else." It would also create an inequality for the students who, for instance, take examinations from examination boards outside Northern Ireland, so we would not even have equality within Northern Ireland.
The Chairperson (Mr Lyttle): I think you are increasingly isolated in that particular view. These are unprecedented times. We have asked for an unprecedented sacrifice from this cohort of pupils.
The Chairperson (Mr Lyttle): Sorry, let me finish. We have significantly changed our approach to many other aspects of life, and I think, for this particular exceptional circumstance, you are increasingly isolated in dismissing support for reliance on an extremely robust teacher-assessed approach —
Mr Weir: Chair, any objective analysis of what has happened this year, last year and in other jurisdictions would suggest that it is clearly not objectively robust, because there is very wide divergence. You rightly indicate the unprecedented bit, but we also need to look at what happens in 2021, when pupils will do examinations. The cohort that will do A levels in 2021 will clearly, arguably, have been more impacted by COVID than those left school in 2020. Those who left in 2020 may have missed a few weeks of academic teaching at the very end of term, given that most of them would have finished school either at Easter or shortly afterwards.
Those who will do exams in 2021 will have missed, effectively, a full term. They will have had levels of disruption. To put them back, without the same safeguards, to a situation in which there is a different approach in 2021 that relies purely on examination means that examination marks would go down from the 2020 figures for a group that is much more impacted and will face a more detrimental situation generally and, particularly, as regards university places, because, with the number of students who, for very valid reasons, want to defer this year, there will be a large cohort entering university in 2021 who are effectively this year's pupils. That means that the number of university places in 2021 for the 2021 group will be reduced. So, we create a double level of unfairness for those in 2021.
We have to think ahead. It is not simply about the decisions that we reach to deal with the very understandable disappointment that exists this year. We have to look at the long-term position and safeguard fairness for everyone.
The Chairperson (Mr Lyttle): I agree. Indeed, the Committee has consistently raised the need for an innovative solution in response to what the curriculum and examinations look like in the coming year. That solution also needs to be found. It should in no way prejudice the solution that you find for pupils on this occasion —
Mr Weir: It is fair, Chair, it is fair —
The Chairperson (Mr Lyttle): — in response to what you referred to as "incorrect results".
I need to move on. We have a technical difficulty with the Deputy Chairperson, Karen Mullan MLA, and Robin is not with us. We will move to Daniel McCrossan MLA, and I will come back to Karen as soon as we have her back.
Mr McCrossan: Thank you, Minister, for being here today. This discussion is very important. Whilst we talk about models, statistics and anomalies as a result of what has taken place this week, we have to be mindful at all times that we are talking about the children and young people of Northern Ireland.
Mr McCrossan: For the last day, I have seen children and young people in tears. Their dreams have been shattered. Their confidence battered. Their families are very worried and concerned about their well-being, health and mental health. I have seen teachers in tears and in shock. They are numb, angry, frustrated and feeling patronised. I have seen principals, very stressed, trying to explain the situation to parents and pupils and to the public.
I have also heard Justin Edwards continually, over the last day, spit out the same, regurgitated excuses that have been used throughout the last number of months, when this Committee and when I, as the SDLP member of the Committee, in the Assembly, flagged the issues with you, Minister, directly. I raised the concerns with Justin Edwards continually and have also spoken about them in Committee. Others have done the same. Leading academics have also raised very serious and detailed concerns that were ignored. If you have seen those concerns, Minister, I would be even more concerned about the situation that we are in today.
Children have been failed by this system. They have been failed by the Department of Education. They have been failed by the processes of CCEA. We need to be very clear, Minister, because I do not think that you or Justin Edwards from CCEA seem to get this. There is a significant level of frustration and anger in the public about how this entire process has been handled. It has not been explained. In answer to all the questions that I asked from the earliest stages, we were told, "We are still working on it. We are still working on it". Even on 'The Nolan Show' this week, Justin Edwards could not answer some of the questions that were put to him about how these grades were determined. The spotlight is very firmly on CCEA today, as it will be after today, and on the Department as to how this is handled. There is no transparency whatsoever. In fact, there is a veil of secrecy around the CCEA processes and how the grades have been determined and, indeed, around the processes in previous years.
Yes, we are in a pandemic. Yes, it is an unprecedented time and things are very serious. However, Minister, the situation that this has inflicted on the young people of Northern Ireland is even more serious, grave even. I have seen the tears of the Children's Commissioner and have heard her on the radio. As I said, I have heard teachers crying. I have heard parents crying. Last night, I spoke with a GP who rang me three times yesterday and was very stressed to get hold of me. She said that she has a deep concern about the well-being of young people after three young students, who were refused university places and did not get the grades that they needed, presented with suicidal thoughts.
Minister, this is a very grave situation because, whilst we talk today about statistics, anomalies and mistakes, tomorrow we will talk about the future of these young people, who will be left behind and who have missed very important opportunities because of a system that has not been transparent and still has not presented at all the algorithm that has overridden teachers' judgement. For the last few days, we have heard teachers being criticised and teachers' judgement being called into question. What does that do for the morale of our teaching workforce, which has supported our young people throughout the last few months and has worked in very stressful situations to ensure that people are educated?
Minister, Justin Edwards said on the radio that teacher judgement is not acceptable because it is inflated, without putting any evidence whatsoever into the public domain or providing any evidence whatsoever that suggests that CCEA or the Department of Education had this information for quite a while. He revealed that days before this, when I would argue, Minister, that there has been a revelation or you have realised in the Department and in CCEA that this formula could not work, was not going to work and could not reach whatever predictions you were hoping to attain in terms of the grades. The situation that we are in today, Minister, presents huge questions as to what we do. We can get our heels dug in and say, "We are not moving on this", and say that you will not consider rethinking this model. This model failed. It failed our children and young people.
Minister, CCEA sent out grades to schools the day prior to our children receiving them knowing that those grades were wrong — awarding U grades where they should have been Cs, awarding Bs where they were clearly As — and teachers are absolutely infuriated and were told then that it was an anomaly. An anomaly. We are talking about children's futures. We are talking about people who have missed out on university places and you are telling us that it is an anomaly. It is not acceptable, and what infuriates me, Minister, more so than all of this is that the Chair of the Committee and members of the Committee warned you and other people in CCEA, including Justin Edwards, that exactly what we are experiencing today would become the reality.
I hope, Minister, that any of the contributions that you, your Department or CCEA make will take into consideration that we are talking about young people. Whilst, yes, you are concerned that grades might be inflated — without any evidence, might I add, that suggests such — the options on the table today and the reality is much worse, because it has downgraded results in schools right across Northern Ireland, including in Holy Cross College in my town of Strabane, where grades were reduced by 45%.
The Chairperson (Mr Lyttle): No, no, I will be very brief, Daniel. I am keen to give members leeway today, but do come to your questions. Thank you.
Mr McCrossan: Minister, can you explain and outline to the Committee what the formula looks like? Can you produce it to the Committee? Can you show us the algorithm that outweighed the judgement of teachers? Minister, could you produce your homework to be marked? The reality is that we are hearing all these statistics from CCEA and from the Department and they are not being checked by others. I can tell you that the report that I have in front of me today is absolutely damning, and it warned Justin Edwards and warned you, Minister. I hope you have seen it, because Justin Edwards has replied to some of the questions and did not reply to the report. If that is the case, Minister, what we are facing today was told to you. Let me tell you this, Minister, in very blunt terms: our young people will not forgive —
Mr McCrossan: — you, will not forgive CCEA and will not forgive this system if we do not act and stand up and show some leadership and correct the wrongs that have been done to these people. They should not pay the price for the pandemic. Can you explain the formula?
This is the final question, Chair. Very bluntly, Minister, is it possible that you will support an independent academic review of the processes rolled out by CCEA? After what we have learned this week, there are major and serious questions about the veil of secrecy that surrounds the processes and the lack of regulation of this body.
Mr Weir: There was a mixture of questions and comments and I will try to deal with them all. With regard the position of children, yes, this has been incredibly stressful and very disappointing for some individual children, as actually also happens in other years. On the position of children, there are more children today who have got a top grade. There are more children who have got an A* to C grade than there were last year or the year before, and that is at A level and at AS level.
You are right to say that behind the statistics are individuals. That is why, there is no merit in simply throwing in some formula that raises grades for many and does not leave others touched. The way to deal with this is to ensure that where prior attainment can be shown, it will be used for evidence in a successful appeal. You rightly raised the issue of university places. The appeals process will be completed in time, particularly for the A-level students, for the university decisions that will ultimately be taken. UCAS has already said that the final award of places will await that outcome. It has postponed that to 7 September, which is well within the time frame. In any consideration of appeals, particular emphasis will be placed on ensuring that those that are related to university places will be tackled first. From that point of view, everyone who appeals, who is hoping to get a university place, will find that that change can be made for them, if it is merited, well before the closing date on which university places are offered.
The Chairperson (Mr Lyttle): Minister, are some university places, such as the Oxbridge places, not under 24- or 48-hour pressure to receive grades?
Mr Weir: UCAS has declared that the appeals process — effectively everything related to opportunities — will be held until 7 September. That is across the board, and I think that Queen's and Ulster University have signed up to it.
Mrs Graham: That is my understanding. Even this morning, someone from Ulster University said precisely that on the radio. They said that although the university is working towards 7 September, it can actually admit students right up to the beginning of October, which actually —.
The Chairperson (Mr Lyttle): Students will obviously have applied to a range of universities so there will need to be assurances across the board on whether that is the case.
Mrs Graham: In England, there is —.
Mr Weir: It is a UCAS assurance so it should be across the board. This year, we are also seeing that although there will clearly be pressure on individual courses, the concern across the universities is that they will not have enough students to fill their places.
Mr Weir: OK, we can get the Committee the paperwork, but let me briefly explain how things worked for A level and AS level. For A level, the results included a school's AS-level results, which count normally for 40% of the overall total. Then, an adjustment was made if resits had been taken and that was factored into each subject. Traditionally, where resits occur, there is a variation between subjects as to what the general uplift is, and that is applied to those figures as well. We then applied a formula, which is called the z-scores. Effectively, for each student, z-scores show the components of a course that they have missed by way of examination. That is the norm with z-scores. Z-scores are used every year. To give you an example, z-scores would normally be used for a more limited number of pupils. For example, Daniel, if you were an A-level student who suddenly ended up in hospital and missed some exams, the z-score would effectively calculate, based on the data from your AS-level grades, what you would have got in the exams that you missed. The other thing is that one individual exam will not uniformly be as hard or as easy as another.
Following those steps gave a statistical data breakdown of the number of A-level A*s, As, Bs etc for each school and subject. Statistical data will be produced for that bit. In some cases, questions were asked and concern expressed about how a ranked order list had been put together. However, CCEA's position was that information that was supplied on rank order lists was not altered at all; it was regarded as being sacrosanct from the school.
So, if you have a situation where, for the sake of argument, there were 100 pupils in a subject area ranked from 1 to 100, and if the result of that model, which does not take into account any previous result, produced an overall position, say, for the sake of argument, where there were 10 A*s, 10 As and whatever, that was then worked down so, irrespective of where the individual appeared in that ranking list, that was what they got when it was applied.
Mr McCrossan: Minister, there is substantial academic literature that confirms that teachers simply cannot predict rank orders within grades with any degree of accuracy whatsoever. Can you identify a single peer-reviewed study that confirms that they can? Can you? This undermines —.
The Chairperson (Mr Lyttle): Just to intervene briefly for accuracy, I think that the continuation of that argument is, "In the absence of forecasted papers", OK. I add that modifier because I realise where you are about to take this, Minister, and that is not going to be the view of this Committee. I do not want you to misrepresent the view of Daniel or of this Committee.
Mr Weir: Daniel has just said that there is no peer-reviewed research that indicates that teacher assessment or, indeed, rank order can be used for accuracy in results, yet — [Interruption.]
The Chairperson (Mr Lyttle): I do not want to speak on behalf of Daniel, but I think that Daniel would probably add to that that the actual studies suggest that there is a need for papers that are going to be set to be included in that prediction. However, at the start, Minister, I gave an example of rank order that does seem to be particularly problematic. I have been advised that a teacher had a ranked order for the B grade, where a pupil who ranked second in that B grade list received a D and a pupil who ranked twenty-first in that B grade list received a B. How does the algorithm lead to that result?
Mr Weir: Somebody who was higher got a worse result.
Mr Weir: Look, clearly, I do not know whether what the teacher put in ended up being the centre assessment grade. There could be some level of alteration. However, if there has been something in the communication of data that has been put into place that has then been in some way different —. Obviously, I cannot comment on an individual case. That is the whole —.
Mr Weir: From that point of view, that is why schools can appeal on behalf of an individual pupil. Indeed schools also have the option, if they take the view that a mistake has been made with a batch of results relating to a certain subject, of challenging that batch of results —
Mr Weir: — if it is what way it has worked out in geography in a particular school or whatever.
Ms Mullan: Thank you, Chair. Apologies, I am having internet issues here this morning. It cut out, so I missed a bit. I will try not to come in on issues that others have raised.
Ms Mullan: Thank you, Minister, for coming to the Committee this morning. First, I want to congratulate and commend our young people, the students, for all the hard work and effort that they put in over the recent period. It has been very difficult for everyone.
As the Chair and others have said, this is not like any other year, Minister, and you cannot ignore the reality of that. Therefore, the usual model of standardisation should not have been applied, and there are particular issues coming out of that, as has been evident over the past 24 hours. There has been much debate and discussion of this over the past 24 hours. Do you, Minister, trust our teachers and their professional judgement?
Mr Weir: It is not a question of trusting professional judgement. Our teachers do an outstanding job. The issue as regards teacher assessment, whether in Northern Ireland, England, Scotland, Wales or other jurisdictions, is that, when teacher assessments have been done in 2019, 2020 or 2018, there has been wide divergence between what the teacher assessments have produced and the actual results of the examinations. It is clear that, as a methodology, it does not produce an accurate result. It also does not produce a fair result. Particularly, without any level of standardisation, you are entirely dependent on how a teacher views it and the difference between teachers. One teacher will be harder than another and one will be easier. If a student is entirely dependent on what their teacher produces, some pupils will be very advantaged and some will be disadvantaged. In addition to wanting to ensure that the 2020 cohort is treated on a fair and equitable basis with the 2019 cohort, and the same for cohorts in 2021 and beyond, if it is purely reliant on teachers' professional judgement, we would not be treating people like for like with regard to the current system and current students. That would be deeply unfair.
Ms Mullan: Minister, I contacted all the post-primary schools in my city yesterday, selective and non-selective. Every principal, across each of those schools, said that it is a shambles. They are very angry and frustrated. They do know of any other profession whose judgement would be disregarded in this manner. They have put in a lot of time and work and have the evidence to stand over the grading. The question they are asking is "Why do it? What was the point when it was not accepted?" They should have just ranked the grade if that was the way it was going to go.
I will give an example of a school in the city, a high-performing school for many years and its record shows it. Over 80% of its students previously got A* to C at A level and, yesterday, it was down to under 60%. CCEA did not contact the school beforehand to say that this was an anomaly. It does not understand how that can have happened. In one subject, for the last two years, it received 100% A* to C grade. This year, that has dropped to 40%, and, as outlined by other members, all those young people who the school had graded as a C were downgraded to a U. That is the worst case, from the schools that I have contacted, but every school has a similar story for a number of students.
You gave an answer about the statistical model and how the grades can go down, but there are far too many cases for this to be left to individual appeals. You have said that you will widen the appeals and that a batch of results can be appealed. Schools did not know that yesterday. When I was speaking to principals, I asked them to contact CCEA and ask them to review the school, as a batch. As we know, the guidance went out yesterday morning. They were under so much pressure yesterday around how they were going to get through all these appeals, worried about what is coming down the line next week with the GCSEs, receiving new guidance and having to update their plans for children starting back in less than a week.
Minister, we really need you to step in, intervene and look at this. This is not like any other year. We are dealing with getting children back to school after being off for a long time. The pandemic is still out there so we need to do that safely and confidently. We need to reduce anxiety, and this situation is not helping.
Mr Weir: With regard to batches, we have said that we will look at a school, for instance, that is unhappy with the way that its cohort has been dealt with in a particular subject. That will be from a systems point of view to see whether there was a mistake there. There is also an opportunity for individual appeals. For instance, if there was an outlier result where there was a large drop or gap between the assessments, and there is clear evidence of the prior attainment of that pupil, their grade will be upgraded at appeal. There is no doubt around that.
What I indicated about the position between teacher performance assessment and the end result was that, in 96% or 97% of cases, it was either what the teacher gave or a difference of one grade. In about 3% of cases, there was a gap of more than two grades. The comparative figures from last year show that there was a 10% gap of two grades or more between what teachers predicted and what people got, so there has been a reduction.
The Chairperson (Mr Lyttle): Minister, in making that comparison, is the manner in which the grades were predicted this year the same as the manner in which they were predicted previously?
Mr Weir: Pretty much so, because the prediction is on the basis of a teacher assessment of what, if you like, Chris Lyttle, A-level student, would have got in history or whatever it was. The methodology for that was left open to schools so that they could draw on whatever evidence that they felt that there was. You know, there was not [Inaudible.]
Faustina, do you want to —?
Mrs Graham: Guidance was provided by CCEA about the type of evidence that could be used. You are quite right: it was a more robust process this year.
Mrs Graham: It was a much more robust process than last year.
Mrs Graham: That is absolutely true. I have spoken to principals who have explained their process to me so I know that it was more robust. Ultimately, no matter what happens from school to school and across schools, you cannot stand over that what happens in one school is the same as what happens in another school. Any teacher or any principal will accept that that is the case. Given that situation, in an ideal world, we would have looked at the external moderation of grades, but that was impossible given our situation and the circumstances that we found ourselves in.
A standardisation process is exactly the same thing. The standardisation process is designed to ensure that young people get a fair outcome. That does not take away, in any shape or form, from everything that has happened to the young people whom we are hearing about today. We have to be careful not to forget about all the young people who got the grade that their teacher said that they would get, which was the case for at least 60% of them. What we are trying to do at the moment is to find a way that is fair. No young person and no teacher wants a child to have a grade that they should not have got or to get something that means that, two years' down the line, somebody will say, "Oh yeah; well, you got that because of the circumstances rather than because of the hard work that you put in".
An appeals process has been put in place. Just to assure you, Karen, I take very seriously the issue of a school that has seen a significant drop, and CCEA, alongside the school, need to look at that. Yesterday, I also spoke to principals who said that CCEA, when they approached it, was very willing to listen to their concerns. Certainly, from the Department's perspective and the Minister's perspective, it is very much the case that the appeals process will be exhausted, to whatever degree it needs to be, in order to ensure that those young people get the results that they deserve.
Mrs Graham: Any teacher understands that you have to have moderation, because we will always have schools that say, "Our school did this precisely right. That school didn't". Even when people were carrying out those robust processes, they were probably thinking about another school or another department that they thought was marking or assessing more generously than they were. The standardisation process is designed to address that.
The Chairperson (Mr Lyttle): Under normal circumstances, yes. I am hearing from teachers who would prefer it, given the context that we find ourselves in, if we reverted to teacher prediction, rather than the moderation the like of which has happened.
Very briefly, the Minister referenced "incorrect results". You have referenced pupils receiving grades that they should not have got, but you still think that the appeals process is adequate to address both circumstances.
Mr Weir: Yes, Chair, because on that basis —.
Mrs Graham: Absolutely. We
Mr Weir: That is, in a sense, what will happen every single year.
Mr Weir: In every year, there will be some people who get incorrect results, and they will be —.
Ms Mullan: Yes, Chair. To finish off, I disagree. The appeals process will not cut it. Minister, you need to recognise that there is an issue, and there have been failings here. Take the example that I gave you of what happened at that school that has a long track record over many years. There are failings here. We need you to intervene. As I have explained, before the education restart and the GCSE results next week, we really need you to step in and address this.
Mr Newton: Thank you, Chair. I apologise, I was at a school transfer
for the second time [Inaudible.]
Minister, the questions I will address to you are fairly specific. You referred to teachers taking a common approach to assessment and said that some teachers may have been, perhaps, more generous and others more severe in their marking. Can those differences be quantified where the assessment showed that there were pupils who were, perhaps, judged by the school more severely and actually got an increased mark?
Mr Weir: It is difficult to quantify as such. However, if we are talking about an individual student who is showing evidence — for example, on the ranked list — it may be that somebody else looks at the ranked list and says that that student should have been higher or lower on the list, but it was not CCEA's job to interfere on the list. The point that I made about generosity and severity — it is the same every year, whether at individual teacher level or at school level — was that some teachers will treat things more generously or severely than others. It does not necessarily mean that they are coming at it from a very different position however much guidance is given.
An analogy from a different sphere is the training for making appointments on a panel, which I am sure we have all gone through. You will hear a candidate, interview them and score them. It may well be that there is a common acceptance amongst a panel doing the appointment about who the best candidate is. It may be that everybody on that panel will have given that person the highest marks. However, it may be that person A gives the candidate 75% because of the way that they see the scoring, and the next person might give them 65% because one is more generous across the board and the other is more severe. Therefore, the problem is that without any moderation, you are not comparing like with like. If there is an attempt to simply use a blanket solution that says, "We will go with the judgements of all the individual teachers and centres", some students will be treated over generously and others will be disadvantaged and discriminated against, and that is within sectors, within schools and between schools etc.
Mr Newton: Thank you. We hear about the differences between boys and girls during the results period every year, and it has not come out in this year's discussion. How have girls and boys performed against each other in the assessment process?
Mr Weir: We have some figures here. I know I have figures here somewhere about this.
Mrs Graham: We do have figures for that. I can send them to you. Karen has the figures.
Mrs Karen McCullough (Department of Education): I can send them through. The patterns are the same as usual.
Mrs Graham: They are the same patterns.
Mr Weir: The patterns are very similar: Girls tend to outperform boys at examinations of this level. Broadly speaking, the position was the same and there was no particular radical adjustment or change between the gap that was there in 2019 to the position in 2020. The same pattern, largely, emerged.
Mrs Graham: It was 84·1% for the boys at A* to C and 89% for the girls. It is the same type of difference that there usually is.
Mr Weir: It is about a 5% gap between the two, which tends to be quite often the case.
Mrs Graham: Yes. We will give it to you.
Mr Weir: I am sure that we can get you comparative figures from other years. However, the gender side of it suggests that the normal position, gaps, trends were there this year as compared to previous years. There is a wider discussion about gender in education and how particular help can be given for boys' underachievement, which is quite a specific topic. This year's results reflected what, largely speaking, has been there before with the gender gap.
Mr Newton: How do the overall Northern Ireland results under this system of moderation compare to previous years?
Mr Weir: At A level, there has been an increase of —.
Mrs Graham: Just under 2%.
Mr Weir: Yes, just under 2%.
As I said, it depends on what way you want to judge things. Taking maybe a different view of it, at the highest possible point, this year, 9·8 % of entries got an A* and, last year, it was 8·8%. You can take any gradation. The principal thing that is quite often looked at is A* to C, and there was an increase of just under 2% in those grades. At the very lowest end, roughly speaking, the percentage getting a U at A level went down from 1·7% to 0·9%. We had 193 U awards in the A-level grades, which is down from roughly 350 in the previous year. As some students will leave school or drop out etc after AS level, you will always get a higher percentage of U grades at that level and, for AS level, U grades went down from about 4·5 % to 3·6%. Across the board at AS level, there was an overall increase in A to C grades of 2·2% from, basically, 75% to 77%. Again, at each grade, there was that progression of improvement and an upwards trend throughout.
As I indicated earlier, to some extent at A level, but more markedly at AS level, the non-selective schools improved their results at a significantly greater rate than grammar schools. At AS level, grammar schools went up by an average of 1·7% overall. In non-selective schools, it went up by 7·3%, which is a very large increase. While the gap between selective and non-selective schools is still there, we have, roughly speaking, reduced it by about 30%.
Mr Newton: Northern Ireland is always particularly proud of the results of our students compared to those in other parts of the United Kingdom. I do not know how you could do the comparison this year, but how do our results this year compare with England, Scotland and Wales?
Mr Weir: I think that we were ahead of them all and that was for selective and non-selective schools. There have obviously been interventions, particularly in Scotland, which has altered the figures significantly. Concerns have been raised in Scotland for many years that their educational performance is not as good as it should be. I think that —. Maybe that is the reason why —. I do not know if Faustina has any comments on that.
Mrs Graham: We are still ahead of England and Wales by about the same number of percentage points as we would have been in past years. It would be around eight or nine —
Mrs Graham: — percentage points at A* to C. We have stayed ahead and that is something to congratulate our students on, as well: that we continue to outperform —.
Mr Weir: Well, we are certainly not comparing like with like, because, apart from anything else, they do not have A levels in Scotland; they have highers. When the comparisons are done, there are some variations, but, broadly speaking, you can compare England, Wales and Northern Ireland. Scotland has had its own education system from pre-union times —
Mr Weir: — so it is very difficult for it to compare its figures with those from England in any year.
Mrs Graham: Overall, almost all our students pass their A levels. We have practically a 100% pass rate at A level and that is really impressive for our students.
Mr Butler: I thank the Minister and his team from the Department for attending. Like you and the rest of the Committee, I want to congratulate the students who have got the grades that they are entitled to and worked towards, and I wish them every success. I make a commitment and a promise to those students who are suffering that the Committee and I will do our best, hopefully working with you, Minister, to bring about a speedy and fair resolution.
I spent all day yesterday, as most of us will have done, speaking to school principals. One of the things that shocked me was the fact that not one of those principals gave a uniform response as to what they believed to be the standardisation model and how it was used to grade students. There were definitely three distinct suggestions as to what it is. I am sure that you would agree with me, Minister, that principals are some of our cleverest people, and we charge them with responsibility for our students. That being the case, coupled with listening to interviews and taking part in debates this week, there seems to have been a shift in the model. It was put to us, certainly at the start of this week, that it was a data-rich model of teachers' assessments and AS grades. An emphasis was put on that this week, with less emphasis on rank order in that statistical modelling. The question was repeatedly asked, and no answer was given: what weighting is given to each component? I would suggest that, if the weighting was given to rank order, that is the fundamental flaw in this system.
I will not give individual examples, because we have been inundated and examples of schools and individuals have already been given, and the appeals process could remedy those. I suggest to you, Minister — I think that you have already made a reasonable commitment — that, if the appeals process shows that the AS result indicates that a grade should have been better, that would be the outcome.
Mr Weir: I have indicated that it is one of the —
The Chairperson (Mr Lyttle): Minister, may I cut across you briefly? Robbie, there is a really good question in there that I hope that you are asking about what percentage weighting was given to each component part. Hopefully, you are asking about that.
Mr Butler: That is the first question: there is a question mark over weightings. We have asked about that repeatedly. We really need to see answers because the question has been asked enough times.
The piece on — sorry, Chair. I just have to learn —
Mr Butler: There is data on AS results. Research from Durham University shows how statistically accurate AS results are. It indicates that between 50% and 60% of results are accurate to the grade, with a 20% to 25% discrepancy, both upwards and downwards. Students accept that. Our students are not asking for something to which they are not entitled. We know that there are winners and losers in exams. This system needs to be fair and just.
I do not mean to be funny and am not being facetious when I say this, but the most interesting phone call that I had came from a farmer last night. He had been listening to reports because he has grandchildren who are involved in the process. He said that the system that seems to have been selected sounds like the Signet system that is used for grading sheep and beef. What happens with that is that it is not based on the value of the ram; it is about where the ram comes from and how valuable the stock was before that ram appeared. It is not measured in the index of that ram — a student, in this context. Where is the fairness?
This is my very last point. When I look back — we picked it up in Committee, Minister — I see that, with the appeals process, CCEA rejected some of the suggested changes — it is in the consultation — and robustly stuck to it. However, CCEA has already moved. I thank it for that. Does that not indicate that the system was flawed from the start, and we really need to exercise haste to rectify the situation? I am asking you for this, because you have said that we use a three-country model. Each year, about 5,000 of our students go across to England, Scotland and Wales. They will be competing here and there with their Scottish, Welsh and English counterparts. Fair is fair, and I do not want our students to be disadvantaged because of what a Welsh, Scottish or English Minister does.
Mr Weir: I will pick up on a couple of those points. You have more of an agricultural background, Robbie, than I have and will have been in touch with others. Standardisation has to happen, in terms of past results, the performance of a school, although GCSEs will be a factor in AS levels.
It is also indicated that, with the formula that we are using, as children move on, there would be the opportunities in 2021 based purely on their A-level results. Unlike previous years, at best, this will not be as good or as robust an examination result. Therefore, in 2021, for A levels, the AS results from 2020 will not feed into that, but there will be the opportunity for them to be back-calculated, which will be the norm, or for an AS resit. On the ram or the beef analogy, where the ram came from or what the ram was in the past has no bearing on A levels, no past performance from the point of view of the school. I appreciate that.
On the issue that you mentioned for ASs, yes. ASs will have a bearing on the A-level result. They can go up, and they can go down. That is why ASs is one of the key pieces of evidence that can be used.
You made the point of rightly not wanting to see Northern Ireland students disadvantaged compared to their compatriots. That is why, in many ways, what we are doing is of a similar nature to England, but we are actually going slightly further than England. They have said that mock examination results can be used as evidence in an appeal. An official from England said that it is not an automatic assumption that that changes the A-level result but it is one of the factors. We are going slightly wider than that in saying that is any evidence of prior attainment. As part of the process by which schools gathered data, the school should have some information on each student. That is something to be forwarded, and other prior attainments can be used. In that sense, we are taking a more expansive and more opportunistic bit.
Mr Weir: It is not as straightforward as that because the rank is not a percentage bit. Depending on which particular formula or the bit that is used on the A-level side, that is what produces a statistical distribution. The distribution is then applied to the rank order. The rank order is vital, but it is not a percentage side of it. It is a different concept in that regard.
Mr Weir: Yes, it is absolutely vital. To take an obvious example: if there are 100 people in a class, irrespective of the overall generation of grades, if you are ranked one or 100, it makes a night-and-day difference in that regard. It is not a percentage in the detail of the algorithm. If there is any information that we have that would be of value, we will certainly —.
The Chairperson (Mr Lyttle): Daniel asked for the algorithm to be provided to us so that we could do that. Robbie, I will bring you back in for a supplementary. I also specifically ask you this, Minister, in a supplement to what Robbie asked: what engagement have you had with Scottish, Welsh and English Education Ministers with regards to a more standardised UK response to the challenges that have been faced with grade awards?
Mr Weir: Fairly consistent discussion has taken place in the run-up to this, over a number of months, on this and a range of other COVID challenges for education, at four nations conferences. There are two layers to it. It happens at the administerial level and at the officials level. As regards qualifications, particularly with the overarching body of Ofqual, all our qualifications bodies are in fairly constant discussions. That is particularly in what might be described as a three-nation model, but it also sometimes involves the Scottish qualifications model.
The Chairperson (Mr Lyttle): Have you recently spoken to the Scottish, Welsh and English Education Ministers with regards to standardising the UK response to this challenge?
Mr Weir: I have certainly spoken to the UK Minister Gavin Williamson on that. Prior to the results, I had spoken to John Swinney and Kirsty Williams. Dialogue is going on.
Mr Weir: There is also fairly constant dialogue between the regulations and the exam boards. While the Scottish are in a slightly different position, they are plugged into that conversation.
Mr Butler: Yes. Minister, I know that you recently established a task force to look at educational underachievement, and I believe that it is one of your priorities. The most recent academic document that has been produced in Stranmillis indicates that the scale of our educational underachievement is masked by the cohort of high achievers. Year on year, we hear about our high achievers, which is welcome. However, the industry experts tell us that that masks the number of underachievers. This is another example of that. When you came in, you rightly told us about the success — and you should do that — but that masks the hidden problems. This is why I want to push you more on the appeals. The people who are less likely to appeal are those who are less likely to have the information and, I suggest, who come from more socially deprived backgrounds. Your standard and benchmark must be AS-level results as a minimum to ensure that our pupils are not disadvantaged. I will press you on this even outside of here, Minister, because people will fall through the cracks. The value of a society is in how it treats the vulnerable and those who do not have the information. We need to aim for those who are likely to fail.
I also suggest that 193 getting U grades is 193 too many. I do not mean that to be glorious in any way. Every student who took part in this did not have a chance, through no fault of their own, to lift themselves up in the last six months. People are getting furlough payments, and businesses are getting rates breaks. We have got lots of help from the Government. We are asking this: what help are the Government going to give each and every student? This is a different cohort, Minister. I know that you have a heart for students, so I am talking to your heart.
The Secondary Students' Union has established a petition. There were over 1,100 signatures on it within 15 hours. That will be landing on the desk. That is the group that is going to really drive this. It is also interested in GCSEs. Will schools focus primarily or exclusively on a three-year performance ranking for those? The GCSE results come out next week and that will be really important.
Mr Weir: There are two issues there. No, I do not know the GCSE results because they are not out yet. We get a briefing on what the A-level, AS-level and GCSE results will be, but those briefings tend to be relatively close to the time at which those results are revealed. Those figures are done on a local and national basis, and the results are confidential; it is embargoed. So, I do not know the results, but, even if I did, I could not tell you. That is the same each year.
Robbie, I will pick up your point on underachievement in secondary schools.
Mr Weir: Yes, I will deal with GCSEs too. I will deal with the three aspects.
Obviously, we cannot second-guess the GCSE results, but the point that is being made about both the A-level and AS-level results is that, compared with last year, secondary schools and non-selective schools did comparatively better than grammar schools. The gap in both A level and, particularly significantly, AS level narrowed, which shows an improving picture.
You talked about success at the top end masking failure at the bottom end. There is always a danger of that. What I would say is that, this year, fewer pupils got the lowest grades; 0·9% got a U grade, with 3·6% of those at AS level. I do not have the comparative figures over time, but I suspect that that is the lowest figure that we have ever had. It is certainly the lowest to my knowledge.
Mr Weir: Look, by the same token, you cannot have an exam system that guarantees that everybody will pass.
Mr Weir: OK. You cannot have a qualifications system that says, "We will guarantee that everybody will pass". Indeed, that would devalue a qualifications system. We have to constantly drive down those numbers, and there has been a level of success in that.
The GCSE formula takes account of the three-year performance of the school. It is weighted, within particular subjects, so that greater emphasis is put on more recent years. The obvious argument of quite a lot of schools, particularly secondary schools, is that they are seeing improvements. It is right that those are reflected; what happened in 2019 is more relevant than what happened in 2017. There will also be a component of what the national result is. That means that some schools that have, in the past, not performed as well will actually have their performance slightly raised. That will produce statistical data for that school's individual cohort in, say, history or geography, to which the ranked list would be applied. The problem is that, the further down the school age you go, the less robust the data is. At GCSE level, for individual performance, there is no robust individual data that can apply across the system because testing earlier than GCSEs tends not to be as robust between schools as —.
Mrs Graham: We have not had statutory assessment. That is what we use traditionally, but, obviously, we do not have any statutory assessment at Key Stage 3 at the moment.
I have here a handout from a webinar that CCEA did earlier this week for school principals. We will send it to you because it gives a straightforward and quite user-friendly description of how each of the models worked. Obviously, you want the more detailed algorithm, but this explains each of the three areas. That was attended by, I think, about 150 principals, who listened to CCEA explaining the process and exactly how it worked, on Tuesday of this week. We can forward those handouts and slides to the Committee.
Ms C Kelly: Minister, yesterday morning at 8.00 am, many of our young people's hopes were dashed and their futures devastated by an algorithm. I apologise if I am repeating myself or what other people have said, but that algorithm has generated inconsistent and erratic results, which, in turn, has discriminated against many high-achieving pupils from disadvantaged areas. Consequently, I have sought input from the Human Rights Commission and the Equality Commission for them to provide clarity and guidance around that. My office, like those of my Committee colleagues, was inundated yesterday with angry and frustrated young people, principals and parents. How, after the devastation that it has caused, can you defend the system that was used? It was flawed from the outset and is seriously detrimental to the mental health and well-being of our citizens. The Scottish Minister did not hesitate to right that wrong; why can you not show the same leadership and ensure that our young people are at the absolute heart of decisions made? At this stage, do you have an understanding of the extent of the downgrading that took place in non-selective schools compared with selective?
Mr Weir: Catherine, there is an inaccuracy: nobody has been downgraded because they did not have that grade. It is not a question of there being an award and then it being reduced. That is not the case. It is the same as what happened in other years: teachers have given their assessments, and there has been a difference in what teachers have said.
No one is suggesting that the system is perfect. The better solution would have been for exams to take place, but that could not happen anywhere, so anything else was always going to be a substitute for the examinations system itself. Pupils at non-selective schools did better compared with their counterparts in grammar schools than they did last year or the year before. The attainment gap in A level and AS level closed this year to a greater extent than in previous years, so, from that point of view, there is no particular evidence of any level of discrimination. Where there are, clearly, individual cases to be looked at, that is where the robust appeals system should be used.
You said that the Scottish Minister stepped up to the mark and did the right thing.
I think that the Scottish decision was wrong, because it affects the credibility of the system. Suddenly, at a stroke, it creates no level playing field for individual results and shifts them up automatically by something between 10% and 14%, and those are percentage points. In fact, you are actually improving in a slightly different way. You are adding 15% to 20% to the numbers that are in place. That is not correct. It also does not give fairness between students, because, as I indicated, it is human nature that, with different teachers, some teachers and some groups will have produced in their assessment a more optimistic or more lenient view on things than other teachers or other groups. You would then be in a situation where, effectively, it would be determined depending on which teacher was looking at your particular situation and which school you were at. A judgement call that differs from teacher to teacher and from centre to centre would not create any level of equality for people.
The Chairperson (Mr Lyttle): Minister, Catherine, we are running slightly over time. Catherine, I will ask you for a brief supplementary, and then I will try to get a question from Justin and Maurice for the Minister to respond to.
Ms C Kelly: May I come back for just one second, Chair? At the heart of all of this are young people. We tend to forget about that when we talk about numbers, grades, systems and all the rest of it. Those young people are absolutely devastated today. Their grades were predicted by their teachers, who are the professionals in all of this, and we need to remember that, and not by a mystery algorithm that we still do not have any clarity on.
Minister, do you have any plans to actually speak to young people? Are you going to listen to them and take on board why they are so angry and confused and have zero faith or confidence now in our education system? Are you going to explain to our AS students that, even though they worked harder than they ever have done before, because their school was not high-achieving, they have now been disadvantaged? It is your responsibility to restore that faith and reverse this shambles.
Mr Weir: OK, Catherine, I will say two things. The past performance of a school in terms of its culture, academic reputation or results has no bearing whatsoever on A levels. Let us nail that bit as well.
I talk to A-level and various other students and teachers every day of my life. We are in a situation of explaining what is termed "downgrading". If we applied the same position last year, we would see that 40% of grades were lower than what teachers predicted. Effectively, that means that, if we used the same language, we had a 40% downgrade last year. The same applies in other jurisdictions as well. With the position that the children are left in, it is always incredibly disappointing when someone does not get the grades that they are hoping for or expecting. It is not the end of the story, however.
You are right — it is one of the points that I agree with you on — that we are talking here about young people and about individuals. That is why, where there is a case to be answered and where there is evidence of prior attainment, it can be looked at and appealed in every single individual case. Dealing with individual cases and with individual children is a better solution than trying to put in some system fix that will actually discriminate against some and benefit others.
The Chairperson (Mr Lyttle): OK. Justin and Maurice, I will take two really short questions. Minister, I realise that we need to let you get away.
The Chairperson (Mr Lyttle): Justin, a question? I will also take a question from Maurice and then let the Minister respond. Thanks, Justin.
Mr McNulty: Minister, you are remarkably calm given the crisis that we are in and the trauma that pupils, parents, teachers and principals are experiencing. The future of those pupils is in jeopardy here and their dreams are at stake. You are remarkably calm given those circumstances.
I am being swamped, like my colleagues in the Committee, by parents, pupils, teachers and principals who are really at a crisis point and at the stage of not understanding what the hell is going on here. Pupils are contacting me, saying, "What is the point of going to school and studying for years, when no allowances are given to your efforts?". A principal has been in contact this morning from Armagh. You said earlier that professional teacher assessment has been demonstrated to not be in line with what the actual results are. I may be paraphrasing there, Minister, but that is the gist of what you said. Why do we bother asking the teachers to go through a rigorous process of producing centre assessment grades when, in essence, they now feel like they have been totally ignored?
We have seen the data from Armagh CBS, which is a non-selective school, and it feels discriminated against. You talked about the gap being reduced; well, I would like to see the evidence that their scores have not been reduced more than those of selective schools. In year 14, 82 students studied. One student had improved grades, and 11 students had grades that remained the same. That is 13%. The other 70 students had grades decreased; that is 85%. There were 24 subjects in total in year 14. One subject had improved grades and four subjects had grades that remained the same. That is 16%. The other 19 subjects had grades that decreased — 79%. Exam entries: we had 236 entries in total in year 14. Seven centre assessment grades were improved grades — 3%. Another 108 centre assessment grades remained the same — 46%. Another 121 centre assessment grades decreased — 51%. There is a massive, massive decrease there in numbers.
Just a quick question. How can you say, Minister, that professional teacher assessment has been demonstrated to not be in line with actual results? If that is the case, why did you ask them for their professional opinion on the results of the students, whom they know very well? Why did you ask them?
Mr Weir: First of all, in terms of awards, I have indicated that if you compare what teachers assessed to be the grades with what the grades were, there is a gap. That is the same as last year; it is the same as 2018. Teachers are asked for their professional judgement of grades every year. It is not unique to this year. In Northern Ireland, we are probably unique in that that has been done for a number of years. Where it has played a particular role is in the basis of the ranking. That is where the teacher assessment has come into play.
In terms of calmness, I am very aware of the emotions around this. As does everybody, I care deeply about this issue. It is important that we try to deal with this with cool heads as much as possible. If I was getting into a shouting match or ill temper, that would not be helping anybody either.
In terms of improvement, the improvement for students across the board, but particularly for those in secondary, is comparing their actual results in 2020 with their actual results in 2019. That has increased, and that, I think, is from that point of view the relevant statistic that is in place.
Mr McNulty: Before you move off, Minister, to what extent does this process discriminate against boys?
Mr Weir: There is no evidence that it does discriminate against boys. What we have said about the results —.
Mr McNulty: You have recently launched an initiative, Minister, to address educational underachievement in boys. To what extent does this process discriminate against the boys who are last-lap sprinters? Very often, in relation to exams, boys do not [Inaudible.]
To what extent does this process discriminate against boys?
Mr Weir: If there is a wider issue of how boys perform in examinations or in the examination system, there is probably a wider consideration that needs to be taken, and that may be a debate for another day. What we are suggesting is that, in this particular system, there is a gap between what girls achieved and what boys achieved, but the patterns of that gap are pretty much the same as they have been in any other year. That is a wider systemic problem than this particular formula and these actions. Look, I think, objectively, it is clearly the case. There may well be a good argument that there is a particular problem there, but I think that it is fair to say that this neither ameliorates that problem nor worsens it.
The Chairperson (Mr Lyttle): Sorry, Justin, I am going to bring Maurice in, just to get a final question. The Minister has been generous with his time.
Mr M Bradley: Minister, thank you very much for coming to the Committee today. I thank all the other Committee members for their contributions. It has been very helpful to get
issues aired. However, a lot of young people had disappointing results yesterday — I think, more than normal. Young children right across Northern Ireland have been awarded grades that certainly could not have been predicted, based on previous results at school.
An example is a pupil who had a high level of attainment throughout their time at school and received a grade well below the expected level. I have also been contacted by parents whose children have changed school and seem to have had no regard taken of their level of attainment at their previous school. Reviews of As and Bs have been awarded, in some cases, as Us.
Will the Minister urgently review the current appeals process to widen the scope of criteria to allow more appeals and to reflect all aspects of previous school attainment so that unexpected low grades can be recognised and adjusted where necessary?
Mr Weir: I did. The sound quality was not great, but I think that I picked up the gist of it.
I reiterate that, on levels of achievement, fewer people got lower grades this year. Grades, overall, are up. Specifically, as regards somebody who has changed school, as indicated for the A-level position in particular, that will not have made any difference, because prior achievement in that school has no role at all in that.
The member made a specific point about appeals. The appeals are being widened. This is the first time in which it is not simply a process-issue challenge that can be taken. CCEA has indicated — I am very supportive of this — that the change that will be made this year is specifically to allow appeals also on the grounds of a challenge that prior achievement or attainment has not been properly reflected in the result that has been produced. As I said, that is probably in a similar line to what has happened in England, but it is a little bit wider than that because the scope of evidence that can be produced here will be wider, I think, than in England. It may be wrong to prejudge their final position because I think that they have involved Ofqual and have specifically tied it in with mock results. We are wider than that.
Mr Weir: I do not know absolutely. Certainly, our attention was drawn to the website. Obviously, a statement was made by CCEA. Certainly, as of late afternoon yesterday, it was pointed out that it was not explicitly on the website. There is a reference, I think, to where an incorrect grade has been given, but it is not explicit. We have indicated to CCEA that it needs to make a change on the website and on any information to make that abundantly clear. Having come straight here this morning, I do not know. That conversation took place at about 6.00 pm yesterday.
The Chairperson (Mr Lyttle): There are obviously a number of urgent issues: the appeals process and admissions to further education, higher education and apprenticeships. I am willing to stand corrected, but it is my understanding that some universities, such as Cambridge, are advising pupils that they will accept whatever grade they have as of the next 24 or 48 hours. Can you guarantee, then, that no pupil will lose an offer that they receive from any university prior to the 7 September deadline?
Mr Weir: That is certainly the UCAS position. Higher education directly is not within my departmental remit. We liaise with it. Can I give a guarantee about absolutely every circumstance in every university? No, I cannot, but it would go against the UCAS position.
The Chairperson (Mr Lyttle): English students can resit in November. Northern Irish students cannot. Do you have any intention to change that?
Mr Weir: Not at this stage. One of the other things that we brought in and said before anybody else is that, in any appeals process, particularly in mind of the issue of socio-economic background, we will also make sure that there will not be any charge for any appeal.
I think that it was Robbie who raised a concern about appeals, and I forgot to deal with his point. Appeals are being routed through the centres of assessment, and the schools themselves, which will also have that level of information. It is not a question of a pupil or a parent effectively being left to float on their own. They will receive support from the school. I can understand that it could be quite daunting for a family to think about how to pull together the evidence and information. That is not the way that it will work.
Mr McCrossan: Yes. Minister, for the sake of clarity, are you going to publish the algorithm? Yes?
Mr Weir: We will certainly give you all the information.
Mr McCrossan: It needs to be understandable and explainable to the public, so how you reached those results needs to be published.
Mr Weir: Anything to do with algorithms, generally speaking, will not be 100% understandable. It will carry a level of technical detail. There is no problem doing that from our side.
Mrs Graham: We can give you this for a start.
Mr McCrossan: Will you support, as I asked you earlier, an independent review of the CCEA processes?
Mr Weir: I think that the CCEA processes were sound. We will see, as the appeal mechanism progresses, if a specific problem has arisen with the processes, and that will need to be judged, but I do not intend, at this stage, to have some sort of independent review.
Mr McCrossan: In one word, Minister, given everything that you have heard and been told over the last few months and what you have learned since yesterday, do you believe that you got this right?
Mr Weir: Yes, but it is not always watertight when somebody judges their own actions. People might disagree.
Mr Weir: On the broader restart, we will liaise to see if we can find a time.
The Chairperson (Mr Lyttle): I emphasise that the school restart is as urgent as these issues, and the Education Committee will make itself available to you in whatever slot you can find early next week.
Mr Weir: I am very happy to do that; I am just highlighting the fact that, in terms of timing, a meeting next week was not anticipated and a range of other business has been put in the diary.
Mr Weir: The Department and the Committee will liaise and tic-tac to find a suitable time. We are very happy to come back.