October 20, 2021
Data
Data Analysis by Robert Vagi
The COVID-19 pandemic caused unprecedented disruptions in K-12 education. Data showing its effect on student learning are now becoming available, and some initial studies at the national level tell a similar story: the pandemic negatively impacted student learning in most grades and subjects, particularly for younger students and students of color.
While national studies shed light on this issue, understanding the local impact of the pandemic will be key for Arizona to effectively mitigate these effects and keep our students advancing.
To this end, Read on Arizona and its partners examined school literacy plan data to determine the pandemic’s impact on childhood literacy in the state. Data sharing and analysis is a guiding principle of Read On Arizona, our state’s early literacy collective impact initiative. Comprehensive data and effective data analysis inform strategies and investments, identify gaps and trends, and drive instructional decisions at all levels, from state to student.
Arizona was making solid progress toward improved language and literacy outcomes prior to the pandemic. Working in collaboration and alignment, partners in Read On Arizona are implementing effective, evidence-based strategies to accelerate progress in third-grade reading. These approaches, and several recent, large-scale policies and investments now in place, will help Arizona respond and recover from the impact of the pandemic. More on those later.
Our Methodology for Analyzing Arizona’s K-3 Data
Every year, schools serving students in grades K-3 submit literacy plans to the state. Among other things, these plans include the results of formative literacy assessments given in the fall, winter and spring which indicate how many students are on track to read at grade level by the end of the year.
In the analysis presented here, we look at differences in the percent of students who were “at benchmark” and “at risk” for a subset of schools[1] in the year[2] prior to and in the year following the statewide shift to virtual learning (FY20 and FY21, respectively).
Our final sample included roughly 200 elementary schools (i.e. 17% of all schools that report literacy plans). Although this sample represents a small proportion of all schools, differences between these schools and those not in the sample were small with a few exceptions. Compared to other schools, our sample had:
- A smaller proportion of charter schools (3% vs. 25%)
- A greater proportion located in cities (64% vs. 49%)
- Higher average enrollment (583 vs. 493)
- Higher average rates of students receiving free- and reduced-price lunch (65% vs. 55%)
- Slightly lower academic achievement (42% vs. 48% passing ELA and 40% vs. 47% passing math)
- A greater proportion located in Maricopa County and a lower proportion in Pima County (69% vs. 56% and 5% vs. 16%, respectively)
Despite these differences, our sample includes schools from a wide range of communities. Because of this diversity, the analyses presented here should reflect COVID’s impact on communities throughout Arizona.
What the Data Show
There were notable declines in the number of students scoring “at benchmark” after the move to virtual learning. This varied significantly across grades, with younger grades seeing the worst impact[3].
Looking at kindergarteners, specifically, there was a 29-percentage-point decrease in the number of students scoring “at benchmark.” Put another way, the percentage of students on track to read at grade-level dropped by more than half compared to the previous year.
Correspondingly, significantly more students scored “at risk” (the lowest achievement level) in that year. While first grade students had the largest percentage of at-risk students in FY21, the increase was, again, greatest in kindergarten with a 28-percentage point increase from FY20 to FY21.
Implications
Such a large negative impact in younger grades should come as no surprise. We’ve all heard anecdotal stories about the difficulties of teaching young children virtually. Further, students in younger grades have fewer literacy skills to build on compared to older students who have had multiple years of formal instruction.
The fact that these early years are foundational to literacy development points to the most important implication of these findings: large numbers of students are leaving kindergarten and first grade without the foundational literacy skills to be successful later on.
This is especially troubling considering that literacy is foundational to all academic subjects. As schools return to in-person learning, providing adequate supports for these students is critical for their long-term academic success.
What We’re Doing About It
Prior to the pandemic, Arizona was making steady progress in third-grade reading, a strong predictor of high school graduation and college attendance. Still, less than half (46%) of our third graders passed (“proficient” or “highly proficient”) our state’s English language arts assessment in 2019. And results of the most recent assessment in 2021 confirm the implications of the data presented here: Arizona youngest students experienced significant unfinished learning due to the pandemic. The percentage of our third graders passing the state assessment dropped to 35%, and long-standing achievement gaps among economically-disadvantaged students and other key subgroups grew even wider.
That’s why partners in Read On Arizona have deepened our commitment to taking a comprehensive, collective impact approach to accelerating progress toward school readiness and third-grade reading proficiency.
In September 2020, Arizona was awarded a federal Comprehensive Literacy State Development (CLSD) grant, totaling $20 million over five years, to support high-need populations with wide-scale professional development in the science of reading, implementation of evidence-based literacy practices, and more. Arizona’s CLSD application was developed by Read On Arizona, the Arizona Department of Education and First Things First.
Partners in Read On Arizona also adopted an Early Literacy Policy Framework to help mitigate the impact of the pandemic by scaling up proven literacy strategies for high-needs schools serving the most struggling readers. The policy framework outlines three priorities for strategic investments to improve early literacy outcomes in Arizona:
- Strengthening educator capacity and new teacher preparation to dramatically increase the knowledge of educators and practitioners in the science of reading.
- Intensive literacy coaching to support and sustain effective teaching and literacy practices for our most struggling schools and students.
- Expanding high-quality early learning opportunities in high-need areas and a statewide kindergarten entry evaluation tool to help identify supports and services a child may need as they begin school.
The collective support of these shared priorities among partners in Read On Arizona informed significant new early literacy legislation (SB1572) and guided recommendations resulting in over $100 million in new investments for early literacy in Arizona.
These investments come at a critical time, as teachers face unprecedented challenges in helping our young readers move ahead. Every Arizona student deserves equitable access to high-quality reading instruction, and building educator capacity to ensure teachers have the tools and training they need to support every child’s learning is critical to reaching our collective goals for third-grade reading proficiency.
______________________
[1] To ensure that measurements were comparable across years, we only included schools that used the most common benchmark assessment in the state: Acadience Reading. We did not examine schools using other assessments because many schools change assessments from year-to-year or are in the process of selecting a new assessment. Additionally, given the large number of assessments that schools can choose from, there were not enough schools consistently using a non-Acadience assessment to create a large enough sample for this analysis. Our final sample included roughly 200 schools in each grade. Although this sample represents a fairly small proportion of all schools in Arizona, it includes some of the largest, most diverse school districts in the state.
[2] These analyses include only the winter assessments of the FY20 and FY21 school years. We chose the winter assessments for two reasons: 1) it was the last assessment given before schools went to virtual learning in FY20; and 2) schools did not administer a spring assessment in FY20.
[3] When interpreting these results, it is important to remember that they do not reflect the achievement of individual students over time. Rather, they represent schools’ ability to help separate cohorts of students acquire literacy skills. Finally, while it is true the in many instances “correlation does not equal causation,” this is not the case in this study. Because of its abrupt and far-reaching impact, the pandemic created a natural experiment with cohorts testing prior to school closures serving as “control” groups and those testing after as “treatment” groups. Therefore, we assume that comparisons of student achievement before and after schools went to virtual learning reflect, at least in part, the impact of the pandemic.
______________________
