International study shows Ireland in top performing countries in literacy at primary school reading in spite of pandemic
Published on
Last updated on
Published on
Last updated on
Minister for Education Norma Foley has today (Tuesday 16 May) launched the Progress in International Reading Literacy Study (PIRLS) 2021, The National Assessments of Mathematics and English Reading (NAMER) 2021 and the NAMER DEIS reports.
The PIRLS study found:
PIRLS is a project of the International Association for the Evaluation of Educational Achievement (IEA). The study is managed on behalf of the Department of Education by the Educational Research Centre (ERC). PIRLS is designed to assess the reading achievement of fourth class pupils (or the equivalent grade level internationally) and to support the measurement of trends, enabling each country to track their own performance over time, as well as facilitating comparisons with 56 other countries.
NAMER are national surveys of reading and mathematics at primary school level. Also conducted on behalf of the department by the ERC, they provide a snapshot of the mathematical and reading skills of second and sixth class pupils in Ireland, and can help to identify areas of strength and weakness, and to inform educational policy and practice. These assessments also provide important information on a range of factors that may affect attainment.
Minister Foley said:
“Launching both PIRLS and NAMER today gives us a unique snapshot into the learning achievements of our young people in both the spring and autumn of 2021.
“These results are extremely positive news for Ireland. We have retained our place among a small set of high achieving countries for achievement in fundamental skills in our schools at a time where particular strain was put on school communities globally due to COVID-19.
“I would like to pay tribute to and thank teachers, principals, parents and children themselves, for their sterling work to support learning and each other during this period. On the return to in-school learning, schools focused hugely on wellbeing, literacy and numeracy, and clearly this focus has allowed for fundamental skills to remain stable in our schools.
“It is a policy of the Department of Education that the curriculum is developed for all learners, from all backgrounds, and I think that these reports are an important tool in informing the decisions that ensure this.
“Work in terms of narrowing the gap between DEIS and non- DEIS schools is also underway, and utilising what we have learned from both the PIRLS and NAMER reports, we will be able to continue with what is working well within the system and work towards improving other processes.
“These assessments are a snapshot in time and I am confident that changes in our primary curriculum will have a positive impact in the very near future.”
In total, 320,542 pupils in 57 countries participated in PIRLS 2021, along with their parents/guardians, teachers, and principals. A further 47,033 pupils took part on behalf of eight benchmarking participants.[1]
In Ireland, pupils from 148 schools took part in Autumn 21 and at the start of fifth class (Start G5). The decision to test in Autumn was due to the impact of COVID-19 and school closures.
The sample of schools was balanced by language of instruction, DEIS status and gender mix. A cohort of 4,663 pupils participated in the paper-based PIRLS written test. In addition, 4,322 parents, 4520 teachers, and 4,610 school leaders responded to questionnaires. This represented a very high participation rate from Irish schools.
The 2021 score must be interpreted in the context of the relevant caveat: the average performance of the Start G5 pupils in autumn 2021 (with average age 11.0) was 11 points higher than that of the End G4 pupils in spring 2016 (with average age 10.5). It is reasonable to expect that pupils in Ireland performed somewhat better in autumn 2021 than they would have done in spring.
PIRLS 2021 records the following about performance in Ireland:
Dr Emer Delaney, Educational Research Centre, one of the report’s authors said:
"Overall, it is positive that reading proficiency in Ireland appears to have at least remained stable between 2016 and 2021, given the disruption caused by the pandemic and the fact that performance in many other countries declined.”
Overall, performance on mean scores and proficiency levels in the National Assessments of English reading and Mathematics (NAMER) appears stable; pupil attainment remained broadly similar between 2014 and 2021. In early May 2021 in 188 primary schools, 5,201 pupils in second class undertook the English reading test only and 5,516 pupils in sixth class undertook the mathematics test only. The scope of the 2021 NAMER was reduced in the context of COVID-19 to minimise the administrative load on school staff and the testing burden on pupils, while still gathering sufficient data for the study. Parent/guardian questionnaires were not administered in 2021.
The participation and completion rates for NAMER 21 are similar to 2009 but slightly below the 2014 levels. However, they are still high given that the assessments were conducted in schools when COVID-19 was still very prevalent. This is a positive outcome for the assessment.
Joanne Kiniry, Educational Research Centre, one of the authors, said:
"It is reassuring that despite the disruption to schooling of the COVID-19 pandemic, the results seen in NAMER 2021 point towards stability of performance in English reading and mathematics since 2014."
In NAMER 2021, there was a larger number of pupils from urban DEIS schools than in earlier cycles of the National Assessments. The purpose of this larger sample was to provide more accurate estimates of achievement levels in DEIS schools than had previously been possible.
There was no significant change in average English reading performance between 2014 and 2021 in Urban Non-DEIS, Urban Band 1 or Urban Band 2 schools.
In NAMER 2021, 25% of second-class pupils in Urban Band 1 schools had reading scores at or above Level 3. The target for high achievers in Urban Band 1 schools specified in the DEIS Plan was 25% so the target has been met.
Second class pupils in Urban Non-DEIS schools significantly outperformed pupils in Urban Band 1 and Urban Band 2 schools in English reading.
Similar percentages of second-class pupils in Urban Band 1 schools were classified as low achievers in reading in 2014 and 2021.
There was no significant change in average sixth class mathematics scores between 2014 and 2021 in Urban Non-DEIS, Urban Band 1 or Urban Band 2 schools.
In sixth class Mathematics, pupils in Urban Non-DEIS schools significantly outperformed their counterparts in Urban Band 1 schools. The gap in average achievement between Urban Non-DEIS and Urban Band 1 schools was very similar in 2014 and in 2021 (about 30 points in both cycles).
In 2021, there was no significant difference in the average mathematics scores of pupils in Urban Band 2 schools and Urban Non-DEIS schools.
Dr Lorraine Gilleece, Educational Research Centre, one of the authors, said:
"The achievement gap between DEIS and Non-DEIS schools has not widened between 2014 and 2021. These findings are to be welcomed in the context of international evidence of learning loss and the particular impact of COVID-19 on pupils from disadvantaged backgrounds."
Contextual reports on PIRLS, NAMER, and Urban DEIS schools in NAMER will be published later. These will examine factors associated with performance, based on data gathered from the questionnaires completed by school principals, teachers, pupils, and (for PIRLS) parents.
Delaney, E., McAteer, S., Delaney, M., McHugh, G. & O’Neill, B. (2023). PIRLS 2021: Reading results for Ireland. Educational Research Centre. www.erc.ie/PIRLS2021
Kiniry, J., Duggan, A., Karakolidis, A., Cunningham, R. & Millar, D. (2023). The National Assessments of Mathematics and English Reading 2021: Performance Report. Dublin: Educational Research Centre. https://www.erc.ie/NAMER2021
Nelis, S.M. & Gilleece, L. (2023). Ireland’s National Assessments of Mathematics and English Reading 2021: A focus on achievement in urban DEIS schools. Dublin: Educational Research Centre. www.erc.ie/NAMER2021DEIS
Sample texts and questions from previous cycles of PIRLS can be viewed at: Informational Materials.
Sample texts and questions from previous national assessments can be viewed at: Sample Test Items.
Significance in the statistical sense does not imply importance in the colloquial sense. Statistical significance is an indicator that an observed effect, difference between two estimates (for example, mean scores) or correlation between two variables, is likely to be found in the population and is not due to sampling error. Statistical significance is determined using a test which measures the probability of each statistic being observed by chance. If this probability is less than five per cent, an effect is considered statistically significant. Statistical significance means that the magnitude of the statistic is unlikely to occur if the relationship were not real.
It is important to note that the 2021 score must be interpreted in the context of the relevant caveats: the average performance of the Start G5 pupils in autumn 2021 (with average age 11.0) was 11 points higher than that of the End G4 pupils in spring 2016 (with average age 10.5).
That said, there is positive news for Ireland regarding the performance of pupils in PIRLS 2021:
International Benchmarks describe the literacy skills and strategies demonstrated by pupils at various levels of achievement. These are Low, Intermediate, High, and Advanced.
The PIRLS report provides an insight into home-based learning that pupils in Ireland experienced during the 2021 COVID-19 school closures:
The National Assessments of English reading and mathematics (NAMER) are surveys of reading and mathematics at primary school level. Conducted on behalf of the department by the Educational Research Centre, they provide a snapshot of the mathematical and reading skills of second and sixth class pupils in Ireland. NAMER can help to identify areas of strength and weakness, and to inform educational policy and practice.
The National Assessments have been conducted periodically in Ireland since 1972. Previous cycles of NAMER at second and sixth class were completed in 2009 and 2014. The 2020 NAMER were postponed until May 2021 given the COVID-19 context and the prolonged school closures from March-September 2020.
A summary of the results in NAMER are as follows.
Overall second class pupils’ performance on English reading in NA 2021 was similar to 2014:
There were no statistically significant differences between 2014 and 2021 in scores on the Vocabulary or Comprehension subscales.
As in 2014, second class girls’ overall English reading mean score (265) was significantly higher than boys (257) in NAMER 2021. Girls (264) also outperformed boys (255) on the Comprehension subscale, but boys (261) and girls (265) performed similarly to each other on the Vocabulary subscale. In terms of the process skill subscales, the only significant difference in performance between boys and girls in 2021 is on the Infer subscale; girls and boys performed similarly to each other on the Retrieve and Interpret & Integrate subscales.
The revised targets set in the Interim Review of the National Strategy for Literacy and Numeracy (Department of Education and Skills, 2017) were not met at either end of the achievement distribution although the initial targets set in 2011 were still surpassed.
Overall sixth class pupil’s performance in 2021 was broadly similar to performance in 2014:
Overall performance on the content and process was broadly similar 2014 and 2021.
In terms of process skills, pupils’ scores at the 25th and 90th percentiles on the Integrate & Connect subscale in 2021 are statistically significantly higher than the corresponding scores in 2014. Pupils’ scores at the 50th and 75th percentiles on the Shape & Space subscale and their score at the 75th percentile on the Integrate & Connect subscale, are statistically significantly lower than the corresponding 2014 scores. Only one subscale (Shape & Space) showed a statistically significant reduction across cycles.
The revised targets set in the Interim Review of the National Strategy for Literacy and Numeracy (Department of Education and Skills, 2017) were not met at either end of the achievement distribution although the initial targets set in 2011 were still surpassed.
As in NAMER 2014, second class girls (265) outperformed boys (257) on the reading test overall, but not statistically significantly so.
Sixth class boys performed statistically significantly higher than girls on the mathematics test overall and on the Number & Algebra and Measures content subscales.
Boys’ performance overall (265 mean score) is similar to 2014 (264).
Boys statistically significantly outperformed girls on the Understand & Recall, Reason and Apply & Problem Solve process subscales. This represents a shift since NAMER 2014, when boys scored significantly higher on only two subscales: Measures and Apply & Problem Solve.
In NAMER 2021, the percentage of boys (16%) and girls (13%) performing at the highest level (Level 4) in Mathematics was similar to 2014, when 17% of boys and 13% of girls achieved at this level. At level 3, the second highest level of performance, there was a small increase in the proportion of boys performing at this level between 2014 (27%) and 2021 (30%). There was a corresponding decrease in the proportion of girls achieving level 3, with 28% achieving level 3 in 2014 and 25% in 2021. These changes were very small and not statistically significantly different.
In terms of the proficiency levels: boys have improved performance in the lower levels (1 and below) whereas girls’ performance has dis-improved especially at proficiency level 1.
There was no evidence of a statistically significant narrowing of the gap between Urban Non-DEIS and DEIS schools. Internationally there is considerable evidence of learning loss in reading and mathematics amongst primary school pupils during the period of the pandemic. Despite this, findings outlined in this report show no evidence of a decline in average reading or mathematics scores between 2014 and 2021 in Urban Band 1 or Urban Band 2 schools. These findings are to be welcomed in the context of international evidence of learning loss and the particular impact of COVID-19 on pupils from disadvantaged backgrounds.
Key summary of results in NAMER DEIS
Although not statistically significant, there was a small increase in the mean reading score of pupils in Urban Band 1 schools between 2014 and 2021. One consequence of this is a slight (non-statistically significant) narrowing of the achievement gap between Urban Band 1 and Urban Non-DEIS schools between 2014 and 2021.
The percentage of very low achievers in reading (defined as Below Level 1 on the reading test) was significantly higher in Urban Band 1 schools (12.9%) than in Urban Non-DEIS (5.0%) or in Urban Band 2 schools (7.1%).
Nearly one-in-three Urban Band 1 pupils (30.3%) performed At Level 1 on the reading test, compared to about one-in-four in Urban Band 2 schools (22.1%) and one-in-six in Urban Non-DEIS schools (16.7%).
43.2% of second class pupils in Urban Band 1 schools had reading scores At or Below Level 1, a value which is very similar to the corresponding percentage in 2014 (43.9%). This shows limited evidence of progress towards the target set out in the DEIS plan (DES, 2017), which aimed to reduce the percentage of low achievers in reading in Urban Band 1 schools to 40% by 2020.
25.0% of second class pupils in Urban Band 1 schools had reading scores At or Above Level 3. The target for high achievers in Urban Band 1 schools specified in the DEIS plan was 25.0% (DES, 2017). Although the percentage of Urban Band 1 pupils At or Above Level 3 (25.0%) in NAMER ’21 does not represent a statistically significant increase over the corresponding percentage in 2014 (17.7%), it nonetheless suggests that the target for high achievers in Urban Band 1 schools set out in the DEIS plan has been met.
In 2014, there was a 31% gap between the percentage of high achievers (that is, those At or Above Level 3) in Urban Non-DEIS schools and the corresponding percentage in Urban Band 1 schools. By 2021, this gap had narrowed significantly to 23% as a consequence of the increase in high achievers (At or Above Level 3) in Urban Band 1 schools from 18% to 25%.
Just 4.1% of pupils in Urban Band 1 schools, 7.4% in Urban Band 2 schools, and 14% in Urban Non-DEIS schools had reading scores at Level 4 – the highest level of reading proficiency.
In Urban Band 1 schools and in Urban Non-DEIS schools, girls achieved a significantly higher mean reading score than boys. In Urban Band 2 schools, there were no significant differences in the mean reading scores of boys and girls.
The gap in average achievement between Urban Non-DEIS and Urban Band 1 schools was very similar in 2014 and in 2021. In contrast, there was some evidence of a slight narrowing of the gap between Urban Non-DEIS and Urban Band 2 schools in the period. Although not statistically significant, a 2-point drop in the mean mathematics score of pupils in Urban Non-DEIS schools combined with an 11-point increase in the mean score of pupils in Urban Band 2 schools resulted in a small narrowing of the achievement gap.
The percentage of pupils in Urban Band 1 schools (16.7%) with mathematics scores Below Level 1 was considerably higher than the corresponding percentages in Urban Non-DEIS schools (5.9%) or Urban Band 2 schools (8.2%). The skills of pupils at this level are not fully assessed by NAMER as pupils Below Level 1 have very low skills in mathematics relative to other Sixth class pupils.
Almost one-third of pupils in Urban Band 1 schools (31.9%) had mathematics scores At Level 1. The corresponding percentages were somewhat lower in Urban Band 2 schools (26.4%) and Urban Non-DEIS schools (20.2%).
In NAMER ’21, 48.6% of sixth class pupils in Urban Band 1 schools had mathematics scores At or Below Level 1, just slightly lower than the corresponding percentage in 2014 (49.9%). On the basis of the available data, there is limited evidence of progress towards the target set out in the DEIS plan (DES, 2017), which aimed to reduce the percentage of low achievers in mathematics in Urban Band 1 schools to 42% by 2020.
In NAMER ’21, 22.4% of pupils in Urban Band 1 schools had mathematics scores At or Above Level 3 – a value which remains below the target of 27%.
Between 2014 and 2021, there has been some limited narrowing in the gap between Urban Non-DEIS schools and Urban Band 1 schools in the percentages of high achievers (that is, those At or Above Level 3). Although not statistically significant, the 25.1% gap in 2014 dropped to 20.3% in 2021 as a result of a slight reduction in the percentage of high achievers in Urban Non-DEIS schools (from 42.7% to 42.6%) and a small increase in the percentage of high achievers in Urban Band 1 schools (from 18.6% to 22.4%).
Level 4 represents the highest proficiency level on the mathematics test. Just 5.3% of pupils in Urban Band 1 schools, compared to 11.1% in Urban Band 2 schools and 15.1% in Urban Non-DEIS schools, had mathematics scores at this level in 2021.
PIRLS | Emer Delaney | Educational Research Centre | 01 8373789 | emer.delaney@erc.ie |
NAMER | Joanne Kiniry | Educational Research Centre | 01 8373789 | joanne.kiniry@erc.ie |
DEIS | Lorraine Gilleece | Educational Research Centre | 01 8373789 | lorraine.gilleece@erc.ie |