Educational software often promises to act like a personal tutor, but results in the real world rarely match the sales pitch. A new field experiment involving Khan Academy suggests the missing ingredient isn't better code, but better human supervision.
What Happened
Researchers conducted a large-scale randomized trial in Uttar Pradesh, India, to test why educational technology often fails to deliver results. In 2022, Khan Academy partnered with government schools to launch a math improvement program. Initially, the program followed standard procedures: teachers received training and technical support lines. However, usage remained low because teachers, juggling competing demands, could not consistently schedule lab time.
To solve this, the study introduced "lab-in-charges"—staff members whose sole job was to manage the computer lab and ensure students logged in. The results were dramatic. Students in schools with dedicated staff practiced math for 47.4 minutes per week, compared to just 7.2 minutes in schools relying on regular teachers.
This increase in "dosage" led to significant academic growth. Students in the supervised group saw math scores improve by 0.44 to 0.47 standard deviations. In the context of low-and-middle-income countries, this improvement is roughly equivalent to two to three years of typical schooling.
The Bigger Picture
This study addresses a massive global inequality: the widening gap between a student's enrolled grade and their actual ability. As we previously reported, tools like Khan Academy are evolving rapidly to offer personalized support. However, access alone does not create learning.
While high-dosage human tutoring is often considered the gold standard, it costs thousands of dollars per student annually, making it difficult for districts to scale. Computer-assisted learning offers a cheaper alternative, but previous research has been inconsistent. Some studies found negative or null effects when technology was implemented with low fidelity.
The new findings confirm that the "binding constraint"—the main thing holding students back—is often organizational capacity. When schools rely on overburdened teachers to manage IT logistics, the software sits unused. When dedicated staff remove that burden, the technology works as intended.
What This Means for Families
For parents and educators, this research highlights that buying the right app is only the first step. The "magic" of personalized learning only happens when there is a structured, human-led routine ensuring the work gets done.
- Technology is not a replacement for structure: The software adapted to the students' levels, but the humans ensured the students showed up.
- Teacher bandwidth matters: The study showed that regular classroom teachers often struggle to fit tech management into their schedules.
- Supervision drives usage: The massive jump in practice time came simply from having a person dedicated to troubleshooting logins and keeping students on task.
What You Can Do
- Ask about implementation: If your school adopts a new platform, ask who is responsible for managing it. Is it an add-on for a busy teacher, or is there dedicated support?
- Create "lab" time at home: If you use apps like Khan Academy at home, mimic the study's success by setting a fixed schedule and supervising the session, rather than expecting your child to self-manage.
- Focus on consistency: The study found that steady, weekly practice yielded results. Aim for regular, moderate sessions rather than sporadic binges.