by Elana Karopkin-Gold & Michele Phillips

Stronger data practices can help leaders better utilize data as a way to deeply understand the students they serve.

“Data-based decision-making.” “Data-driven instruction.” These are now-familiar terms in the world of education. An emerging understanding of the power of data across various sectors continues to excite savvy and creative educators who see value in leveraging data-driven design. In the past decade or so, many schools have invested in practices such as interim assessments to help teachers and leaders monitor and respond to trends in data. Some—mostly larger—districts have even invested in processes such as school report cards and ways of measuring “value-add.” While there is not yet full-sector clarity and alignment regarding where an investment in data-based practices yields the strongest results, there is no question that schools better serve students when they understand the students they serve.

We take this maxim to heart at Springpoint, a national nonprofit that supports districts, networks, and charters to design innovative new high school models. We encourage our partners to adopt a curiosity mindset; to constantly ask “why” and seek deep insights about their students as a first step in the school design journey.

Using a wide variety of quantitative and qualitative data helps leaders gain a true understanding of the young people they serve, which in turn allows them to articulate students’ most critical needs and assets, and devise important programmatic responses in the design of school—especially for the most marginalized students. Often, this requires leaders to both define data broadly and set up strategic systems and inflection points to collect and apply data in their iterative school design work. Strong school models are developed when leaders use both quantitative data—such as assessments, test scores, and demographic information—and qualitative data, such as community and student surveys, focus groups, in-depth interviews.

There is a perception that qualitative data is useful, but extremely hard to gather and analyze; in our work with schools and districts, we have found that gathering and manipulating quantitative data can be just as challenging. This article focuses on the barriers to collecting, analyzing, and using quantitative data in service of school design.  We also discuss how districts and schools might start to address some of the technical challenges inherent in doing so.

Understanding Root Causes of Student Performance

Using data strategically and effectively allows leaders to leverage authentic opportunities that address students’ interests, strengths, and needs—and lets them continuously ensure that resource allocation and learning experiences are responsive to those specific needs and interests, as opposed to an assumed need or a fleeting education fad. For example, in our work supporting school designers to develop school models for students who are off track to graduation as part of Barr Foundation’s Engage New England initiative, we have seen that when data is not readily available and easy to manipulate and digest, it can be hard to assess whether students are off track because of isolated skill gaps, challenges with English language acquisition, chronic absenteeism, or something else. Without access to raw data and the ability to manipulate the data in service of testing hypotheses, there is little to do other than to make an assumption that all of these challenges are present in equal parts—or that they are so highly interrelated that they cannot be teased apart. This level of analysis is crude and does not allow school designers to understand key root causes, define the most critical solutions, and create learning environments that set students up for success.

Often a deeper understanding of students comes from cycles of inquiry in which leaders cut raw data in different ways, which allows educators to reject erroneous hypotheses and come to increasingly more nuanced and accurate conclusions. Without access to or aptitude with raw data, leaders are left relying on anecdotal or surface level trends that can often misdiagnose the root causes of poor student performance. Such a misdiagnosis can point leaders toward solutions that demand time and energy to devise and operationalize without actually addressing the most critical needs.

There also exist limitations on how districts represent data when they focus narrowly on state reporting functions, rendering data cuts tailored to this purpose far less useful than they have the potential to be. Finally, we have seen leaders struggle with unruly data systems for reasons that run the gamut but often stem from the sheer number of platforms and tools, compounded by a lack of compatibility between them.

The following are some of the specific areas we have seen as obstacles in our work with practitioners at the school and district level:

Challenge #1: Access to and Aptitude with Raw Data

Quite often, student-level data files are only fully accessible at the district level and not by individual school leaders and staff. Even when accessible, data files may be unwieldy and available only through elaborate data portals. District data teams often understand their role as primarily providing information necessary for local, state, and federal accountability reporting. District data personnel are therefore more frequently focused on ensuring data is clean and ready for state reporting purposes than analyzing and packaging data for school-based personnel. Many school leaders would like to use data as a way to make decisions, but they do not always have the skills to use specific data tools or know how to draw conclusions, refine a hypothesis, and articulate meaningful research questions. District data teams’ roles rarely allow them to provide sufficient school-based training or prepare school-facing data reports. Therefore, even when school-based personnel have access to raw files, few have enough training in data practices to extract what they need.

Data reporting and collection systems often evolve and change and states or districts sometimes decide to switch vendors. Modifications such as these mean that data specialists at the district-level must spend a bulk of their time getting retrained in new systems and reporting functions. Another demand on their time may include needing to turnkey new information to school-based staff, but only for reporting purposes. There then remains little time for inquiry cycles that extract meaningful, informative data on student performance. As a result, school-based staff often become frustrated and left without usable school-level data to inform their practices or shape instructional and programmatic shifts.

Challenge #2: State Reporting Limitations, Especially for Vulnerable Populations

When data is collected exclusively for use by the state, it is cut and analyzed according to state requirements, which may leave important gaps especially pertaining to the most vulnerable sub-populations of students. For example, state-reported data might not include subgroups of demographic data when the number of students falls below a required sample size. Leaders, therefore, cannot identify or address significant trends. Say a school experiences an uptick in enrollment of English Language Learners but does not exceed the minimum threshold in a given year (e.g., 10 or 20 students per cohort), then year-to-year data will not be available regarding this group’s attendance, graduation, and state assessment outcomes. It may also be that important subgroups at a school are not defined in a traditional demographic sense. Natural groupings of students with similar characteristics and similar support needs may not be immediately apparent, such as previously incarcerated students, recent immigrants, students who are the breadwinners for their family, etc. Leaders need to ask the right questions and go beyond quantitative data to better understand who they are designing for. And finally, new, merged, or re-named schools may not have consistent reporting year-to-year because of changing school codes. Challenges such as these can render state-reported data inaccessible or inaccurate, making it harder to access data trends for students who most need visibility and support. 

Challenge #3: Systems Overload

An increasingly pertinent issue in the education world is the lack of data interoperability—wherein systems are not set up to speak to one another. Even when systems are designed to make data gathering and analysis easier, inconsistent use can compound challenges. For example, schools often use multiple systems—perhaps there is one system for overall student enrollment information (SIS), one for benchmark or interim assessments, one for participation in state assessments, possibly a separate system for grading, attendance, or behavioral data tracking, and, increasingly, learning management systems that may or may not connect directly to the school’s SIS. Lack of data interoperability renders data-based decision-making arduous and unrealistic. While there are both local bright spots and nationally-focused companies and initiatives—like Project Unicorn, Ed-fi Alliance, and Illuminate Education—that continue to work toward solutions, data interoperability challenges still cause confusion and frustration, all while taking up leaders’ precious time.

These are not insignificant challenges but solutions exist that could provide leaders with a baseline set of conditions for a student-centered, data-driven process.

Solution #1: Re-Defining the Work of District Data Teams

Districts must expand the vision of what their data teams can and should do beyond state reporting requirements. When districts cultivate a shared belief in the importance of getting data into the hands of school leaders, this data can be used as the basis of making and evaluating important decisions. In order to do this, districts need to build capacity for this work. That can include training programs as well as the direct development of key data cuts and analysis for leaders to leverage and employ. Through trainings and other engagements, district data experts can help schools understand that data-based practices are a not once-a-year occurrence. They can demonstrate why data-based cycles must be embedded in school practices throughout the year with a comprehensive understanding of what data is, how to collect and analyze it, and when to use it to inform decisions and assess progress toward goals.

Solution #2: Systems Fixes

Because the very systems that are designed to support data use often make it challenging, leaders can develop standard operating requirements for data systems and school-based data entry. District data experts can play an important liaison role between systems providers and schools, ensuring that key systems speak to one another and school leaders can extract the data they most need. Before implementing or purchasing a new system, districts must consider how the new product will interconnect with other systems. When possible, new products should reduce the overall number of unique systems to avoid overload, minimize training needs, and increase ease of access and usability of data.

Solution #3: Finding the Right Data

Along with access to and aptitude with data, leaders need to be able to ask the right questions that can surface useful answers, think creatively about how to source information, approach analyses from different angles to yield actionable insights, and develop the muscle that will allow them to apply the valuable information surfaced during inquiry cycles. At Springpoint, we conduct research visits alongside leaders, demonstrating how to execute robust inquiry cycles as a way to gather qualitative and quantitative data, which can inform nuanced conclusions that lead to community-specific solutions. We encourage and support leaders to be strategic about the grain size of their research questions within each inquiry cycle since that will help them find patterns, variation, and bright spots. We help build leaders’ capacity to iterate on school designs. Districts interested in supporting data-based practices need to ensure they cultivate the mindsets of curiosity and the skills of data analysis, and they need to allocate sufficient time for this in trainings and other professional development experiences. School leaders and teams can then track student experiences and school structures that they believe will lead to strong student outcomes, see the results, refine, and repeat. These practices allow leaders to understand the real impact of their choices and gives them the necessary information to course correct when needed.

*    *    *

Effective data practices have the power to help districts and schools be more aware of trends in student performance. Once school leaders can see important patterns, they can be more strategic, intentional, and responsive. The existence of foundational data systems and practices is a crucial first step but schools also need training and support from the district level, easy access to the most important data sets and data cuts, and effective and efficient systems that speak to each other. Critical data practices, when done well, can elucidate a true understanding of students and their needs, which in turn fuels effective school design and continuous model improvement. 

Author bios

Elana Karopkin-Gold is Springpoint’s Senior Strategist and draws on her experience as a charter network superintendent, founding school leader, and classroom teacher to help expand and codify the organization’s impact and shape design support for partners. Prior to joining the team, she spent a decade as the regional superintendent at Achievement First, where she directly managed the development and launch of Achievement First’s high schools and managed several middle and high school superintendents. During her tenure, Elana also oversaw the development of the network’s high school and college access and success programming, including as manager of the college success team. Under her tenure, 98% of Achievement First students took at least one AP class by their senior year, with more than half of them passing at least one AP test. High schools in the network saw a 94% graduation rate and a projected college graduation rate of 52%. Previously, Elana spent 12 years in the New York City Department of Education, as a high school English teacher, Assistant Principal, Regional Instructional Specialist, and founding principal of the Urban Assembly School for Law and Justice, a public high school focused on academic rigor, supportive relationships, and integrated partnerships. She designed the school and led its growth and iteration as principal for four years. Under Elana’s leadership, the school received A’s on the school’s Progress Report and “Well-Developed with Outstanding Features” on the School Quality Review. Most critically, 93% of the founding class graduated on time and 100% of them were accepted to college. She holds a Bachelor of Arts in English from Bryn Mawr College and a Master of Arts in English Education from New York University.

Michele Philips is a consultant with Springpoint who incorporates her experiences in school scheduling, data analysis, and strategic planning to help Springpoint staff and partners successfully engage in the design process. Michele conducts observation and research site visits alongside Springpoint staff, provides one to one coaching with partners engaging in student-centered school design, and works collaboratively to develop tools, systems, and protocols for data-driven design and iteration. She has expertise in using data to inform school design priorities and strategic plans, systems for monitoring and analyzing school/district level data, and student and teacher scheduling models. Michele spent nine years at the Urban Assembly School for Law, Government and Justice first as a Spanish teacher, and ultimately taking on such roles as data specialist, assistant program chair, and assistant principal. In her various roles she oversaw all school data systems for collection and analysis of student exam, credit accumulation, attendance, and scholarship data for all grades. She also managed professional development, led school-based data inquiry teams, and analyzed and disaggregated all data for subgroups and target populations using formative and summative assessment data. She has worked for New York City’s Department of Education, leading work on flexible scheduling, data system management and training, student sub-group monitoring, and more. She holds a Bachelor of Arts degree as well as a Master of Arts from Middlebury College.