.png?width=552&height=414&name=How%20to%20Choose%20a%20Math%20Curriculum%20(2).png)
How to Reach Diverse Learners with Differentiated Math Instruction
In a math class, no two students are the same. Teachers see a wide range of skills and abilities in their classrooms,...
When a school or district considers a new curriculum, instructional strategy, or education program, one question matters most:
Will it make a meaningful difference for students?
And then maybe a whole slew of questions and data follow. One report says students improve with the program in question. The next one says results were significant. And a third says results weren’t significant but there was a meaningful educational effect.
What does this all mean? How can you tell whether the program improves outcomes for students like the students in your district? What is a big improvement? Is this program any better than what you already have?
Research can help answer these questions, but the numbers aren't always easy to interpret.
So, researchers use a tool called an effect size to help answer these questions. What is effect size? Despite what can sometimes look like an intimidating math equation, an effect size helps answer a straightforward question: How big is the difference or relationship?
In education research, having a clear understanding of the actual impact of the outcomes is critical for teachers and schools to interpret how meaningful the research results are to their classrooms.
Ahead, we’ll explain what exactly an effect size in education is and explore 5 key questions every educator should ask when interpreting education research.
Effect sizes in education can help us understand how much student outcomes differ between groups or change over time.
For example, a study might report that students who used a new math curriculum scored 5 points higher on a test than students who didn’t. Is 5 points a lot? It depends.
Without knowing how much student scores typically vary, we can’t tell whether that difference is substantial or relatively common.
An effect size helps put a difference in context. Just as a percentage puts one quantity in relation to a larger whole, an effect size can put the difference between groups, or the amount of change, in relation to other information about the results.
"In education, even a seemingly modest effect may matter if it leads to meaningful improvements for students across a school or district."
Imagine you’re comparing two runners in a race. One finishes in 10 minutes, and the other in 9 minutes— a 1-minute difference. Is that big? It depends on the timing of the other runners in the race.
If most runners finish within a few seconds of each other (variation), a minute is a huge gap. But if runners finish anywhere between 5 and 20 minutes, then that same 1-minute gap doesn’t stand out.
Effect sizes work the same way. They tell us not just that one group outperformed another, but how big the impact is, in context.
Now, let's explore how to interpret education research when effect size comes into play.
Effect sizes can help educators understand how much difference an instructional strategy, curriculum, or intervention made in a study. But what counts as a small, medium, or large effect depends on the measure researchers use and the context of the study.
Researchers use different types of effect-size measures depending on what they're examining. Some describe the difference between two groups, while others describe the strength of a relationship between variables.
You don’t need to memorize the formulas to be a thoughtful consumer of education research. What matters is knowing which measure a study reports, what it represents, and how it should be interpreted.
Researchers have proposed guidelines for describing effect sizes as small, medium, or large. However, interpretations vary by measure, field, and study context, and there is no single set of cutoffs that works for every education study.
Some education research uses ranges such as the ones below as reference points. Treat these as contextual guidelines—not universal thresholds—and check how the study defines and interprets its measure.
Here is a guide for how effect sizes in education research are currently interpreted?2,3,4,5,6:
|
Effect Size Type |
Effect Size Number |
Key Considerations |
|
Small education effect size |
0.01 ~0.24 |
Keep in mind that even a small effect size can be trustworthy and represents a meaningful change. |
|
Medium education effect size |
~0.25 - ~0.45 |
A medium effect is a good place to aim for in education but represents a lot of developmental time and experience. |
|
Large education effect size |
>0.50 |
This is equivalent to a landslide victory. However, as we discuss below, big numbers on their own are not enough. We need to know the context of the victory to evaluate it properly. |
Key point: A bigger number doesn’t automatically mean better evidence or a more effective program. Context matters.
An effect size is one piece of evidence. Before deciding what a study’s findings mean for your school or district, consider these five questions.
Not all races are the same. A fun run isn’t the same as the Olympic trials.
The type of study can make a big difference in how we interpret an effect size and in what effect-size measure researchers report.
A common type of study is a correlational study, which observes if two things are related – like, “is height related to achievement?”
These studies describe patterns when we are just starting to learn “is it important.” They do NOT tell us whether one thing causes the other.
In this example, taller students might have higher achievement because they are older and have had more educational experiences—not because their height makes them smarter. A relationship between two things does not, by itself, establish cause and effect.
Quasi-experimental studies and randomized controlled trials (RCTs) can provide stronger evidence about whether an intervention caused observed results, although the strength of the evidence still depends on how each study was designed and carried out.
|
What to ask: Does this study show that the program caused improvement, or only that the program and improvement were associated? What kind of study was conducted, and what measure of magnitude does it report? Key points: If a study does not report an effect size, look for another statistic or information that helps you understand the magnitude of the results. If that information is missing, consider asking the research team for more detail rather than judging the study on the headline result alone. |
A sprinter might crush a 100m race—but struggle in a mile effort. Similarly, studies that test students immediately after a lesson may show gains that look different from outcomes measured months later. Results can also vary depending on what outcome is measured.
Effect sizes can look bigger or smaller depending on what is measured and when. They tend to be larger when:
For example, imagine a math program teaches a specific way to solve problems, and students take a short quiz right after the lesson. If the quiz uses the same style of problems, scores may improve because students have just practiced those skills. That is useful information, but it is different from seeing improvement months later on an assessment that measures learning in other ways.
|
What to ask: When was the outcome measured? What was measured, and how? Was the assessment developed independently, and does it reflect the learning you care about? Are there longer-term results? Key points: Independent assessments and longer-term follow-ups can provide useful additional evidence, but independence alone does not make an assessment more trustworthy. Consider whether the assessment was created by a reputable source relevant to the learning goal, and appropriate for the students and context. |
A 1-min. lead in a race of elite runners has more impact than the same lead in a local 5k.
Likewise, studies focused on specific student populations may produce different effect sizes from studies involving broader populations.
These differences can reflect students’ starting points, the relevance of the intervention to their needs, or conditions under which the program was implemented.
That’s not a bad thing— it is good to study student sub-groups—but it does mean we should be careful when comparing effect sizes across studies and all types of students.
|
What to ask: Were the students and schools in the study similar to those in your setting? Did the study focus on a particular group of students? Were results reported for individual students, schools, or districts? Key points: Effect sizes are easier to interpret when you consider whether studies involve similar populations and measure outcomes at a similar level. Findings from one group may not transfer in exactly the same way to another. |
A runner who wins once on a perfect track might falter in a real race. A runner who has only completed small distances may struggle with longer distances.
Similarly, programs tested in highly controlled settings or over short periods may show different results when used across real classrooms and schools.
When more schools and districts use a program, there is a wider variety of students, teachers, and contexts. Some schools may use the program as designed, while others may adapt it or use it for less time.
These differences can affect the size of the results, even when a program is still helping students.
A program designed for a full school year, for example, may be studied for only 12 weeks. That study can still provide useful information, but it may not tell the whole story about longer-term outcomes.
Smaller studies, especially those where researchers closely support implementation, may operate under more controlled conditions than everyday classroom use. That is one reason it is useful to look for evidence from settings that resemble your own.
We care about how a program performs at scale, in real-world conditions.
|
What to ask: Where did the study take place? How long was the program used? Were teachers and students using it under typical classroom conditions, or did the research team provide additional support? Key points: A program might show a medium effect size in a tightly controlled study but a smaller effect when implemented across real classrooms. That doesn't automatically mean the program is less effective. Real-world implementation involves diverse students, teachers, and conditions, so it's not always fair to compare effect sizes across these different settings and assume the bigger number is better. A smaller effect in everyday classrooms can still represent meaningful progress and may offer a more realistic picture of what to expect when a program is used at scale. |
Consistency matters. There’s a difference between a runner who won one big race many years ago and one who finishes strong in different types of races year after year.
In education research, there’s a difference between a program that shows positive results once and one that has been studied multiple times in different settings.
A single randomized trial can provide valuable evidence. Multiple rigorous studies that find similar results can add confidence that the findings are not unique to one study or setting.
Studies will report a range of effect sizes. Research conducted by program creators often report larger effects than research conducted by independent teams, so it is useful to look for studies from more than one research team.
|
What to ask: Has the program been studied more than once? Have independent research teams examined it? Are findings positive across rigorous studies, and do the studies reflect settings and students like yours? Key points: Effect sizes may vary across studies because participants, settings, measures, and implementation vary. Repeated findings can strengthen confidence in the evidence, but they do not guarantee that the same results will occur in every setting. |
Next time you are faced with purchasing a program that shows a test score difference or some sort of student improvement, ask how the study was conducted, what was measured and when, who participated, whether the results hold up in real-world settings, and whether other studies have found similar results.
Effect sizes can help you understand the magnitude of a research finding, but they are only one part of the evidence. You don’t need to be a research expert to ask informed questions.
Use our 5 Questions to Ask When Reviewing Effect Sizes checklist the next time you’re evaluating a study.
Explore the Accelerate Learning Research Library: https://acceleratelearning.com/research/
---
In a math class, no two students are the same. Teachers see a wide range of skills and abilities in their classrooms,...
Identifying the right math interventions for a classroom isn’t an exact science....
Keeping students engaged in science takes more than a great lesson plan.
Students learn more when they take an active...
