A study of over 6,000 middle school students in Tennessee shows that artificial intelligence improves math learning only when it forces students to slow down. By requiring kids to correct their mistakes and repeat the skill, this "mastery-based" approach counteracts the quick-fix shortcut that hurts long-term retention.
What Happened
According to a working paper scheduled for circulation by the National Bureau of Economic Research, researchers recently tested different approaches to practicing fractions. They randomly assigned middle school students to use either traditional computer software or software enhanced with an AI tutor called Numi. Within those groups, some students had to use a "mastery learning" approach, meaning that if they made an error, they had to answer three consecutive questions correctly before moving to the next topic.
The researchers found that AI alone was not enough. Instead, the combination of AI tutoring and strict mastery repetition led to the best results. Students who used the AI-plus-mastery system scored 3 percentage points higher on a follow-up test a week later than those using conventional computer programs. The AI tutor walked students through their mistakes step-by-step and kept them from skimming past the correct answer.
The Bigger Picture
The results show a clear trend: how students interact with AI determines whether they learn or fall behind. When students use AI to bypass effort, the academic consequences are severe. For example, a study of over 26,000 secondary students published by the Centre for Economic Policy Research revealed that students using generative AI to complete homework finished their assignments faster, but their subsequent exam scores plummeted by 20%. This "AI learning trap," analyzed in Psychology Today, occurs when students outsource their thinking to technology.
As we previously reported in our analysis of how ultrafast AI modes impact learning, instant answers short-circuit the brain's natural retrieval process. But when AI forces active engagement, it works. A study in Scientific Reports found that AI-supported personalized practice and immediate error correction improve long-term math retention. Similarly, a systematic review in Discover Education noted that the success of AI tutoring platforms depends entirely on whether the software forces students to struggle with the material.
What This Means for Families
For parents and educators, the lesson is simple: AI should be a demanding coach, not an obedient assistant. If children use AI like a high-tech cheat sheet, their test scores will suffer. But if they use tools that force them to think and correct their errors, they will build stronger skills.
As we noted in our guide on shifting from asking to doing AI, adults must teach children how to use these tools for active learning rather than outsourcing the work. When an educational tool makes learning feel too easy, it usually undermines long-term retention.
What You Can Do
First, prioritize mastery over completion. Parents should encourage children to use learning platforms that require consecutive correct answers before they can move to the next level. Second, practice error analysis. When a child gets a math problem wrong, ask them to explain their mistake out loud. This practice builds metacognitive skills, which we covered in our review of Duolingo's new error review tools. Finally, set clear boundaries for generative AI. Ensure students do not use tools like ChatGPT to write answers. Instead, guide them to ask the AI for hints or explanations of the concepts.