School districts spend billions of dollars on educational software, yet many of these programs lack rigorous proof that they help students learn. A recent analysis of independent research reveals a divide between marketing claims and scientific reality, prompting calls for more scrutiny from school boards and parents. As school leaders face tightening budgets, they need to know which digital tools actually work.
What Happened
In a recent industry analysis, Sal Khan, founder of Khan Academy, reviewed independent, randomized controlled trials (RCTs) across the most popular K-12 math platforms. As we previously reported, many edtech marketing claims rely on correlational studies. These studies merely compare heavy users of an app with non-users. This method suffers from selection bias because highly motivated students are naturally more likely to improve. By contrast, randomized controlled trials assign students by chance, which establishes whether a program actually causes academic gains.
The review showed that only a few math programs have independent trial support. Khan Academy has six independent RCTs with gains between 0.06 and 0.70 standard deviations (SD), depending on staff support. Meanwhile, ASSISTments has two independent RCTs showing positive math gains of up to 0.18 SD, and the lottery-assigned math tool Mindspark showed a 0.37 SD increase.
Other popular tools have far less support. Programs like Carnegie Learning and Zearn have only one or two independent RCTs, often with small or statistically insignificant results. For other widely used programs, like i-Ready, researchers found no independent RCTs. This means school districts are relying on studies sponsored by the developers themselves.
The Bigger Picture
To see if these numbers matter, educators look at how researchers measure academic improvement. According to a study by educational researcher Matthew Kraft, the median effect size across nearly 2,000 real-world education trials is 0.10 standard deviations. Kraft suggests that for actual classroom interventions, any effect size between 0.05 and 0.20 is medium, and anything above 0.20 is large.
Comparing edtech software to non-digital interventions provides context. According to a systematic review by the Stanford SCALE Initiative, high-dosage tutoring produces academic gains of 0.29 to 0.37 standard deviations. Similarly, reducing class sizes in early grades, as seen in the Project STAR study in Tennessee, leads to long-term academic gains generally ranging between 0.15 and 0.35 standard deviations.
While intensive tutoring works well, it is expensive and hard to scale. An inexpensive software program that achieves a reliable effect of 0.10 to 0.18 standard deviations offers schools a cheap, scalable way to support students.
What This Means for Families
For parents and educators, this research shows that not all classroom technology is equal. Some programs have strong scientific backing and help children learn, while others are unproven marketing plays.
Even the best software only works when implemented correctly. When schools buy software without training teachers or providing enough devices, the benefits disappear. Technology cannot replace a teacher, but a proven tool used consistently is a helpful safety net for struggling learners.
What You Can Do
First, demand independent evidence from school boards. When your district evaluates or renews software contracts, ask if the programs have undergone independent, randomized controlled trials. Do not let them rely on evaluations funded by the software vendors.
Second, focus on how children use these programs at home. If your child uses educational apps, ensure they use them consistently and on their own devices. Software works best when students have dedicated, individual practice sessions rather than sharing screens.
Finally, advocate for balanced budgets. Urge school administrators to combine validated digital tools with high-impact human interventions, such as small-group tutoring, to get the best results.