When a school stops using an educational software program, administrators usually assume they can simply delete student accounts. But as classrooms adopt artificial intelligence, digital deletion is breaking down. Once student voice recordings or reading habits are used to train an AI model, that data becomes baked into the software's mathematical parameters, making it nearly impossible to extract.
What Happened
Traditional databases store student information in rows and columns that are easy to delete. When an AI system trains on student work, that personal data becomes part of the model’s mathematical weights.
Regulatory bodies warn that standard data deletion requests do not work for AI. The UK Information Commissioner's Office notes that erasing personal data from trained AI models is often technically impossible without retraining or deleting the model itself. France’s data protection authority, CNIL, shared a similar stance in guidance updated in January 2026, stating that filtering out old data is a weak fallback and true erasure requires retraining.
The European Data Protection Board guidelines define data erasure as reversing what an AI has memorized. This requires removing the original data and the statistical mark it left behind. Privacy expert Georg Keferböck writes that this forces developers to use "machine unlearning" to comply with privacy laws.
The Bigger Picture
This technical challenge has legal consequences for schools and technology companies. In the United States, the Federal Trade Commission (FTC) now orders "algorithmic disgorgement," a penalty that forces companies to completely destroy AI models built with illegally gathered child data.
The FTC applied this penalty against the classroom platform Edmodo, which had to delete the machine learning models trained on student data without parent consent. The agency took a similar stand in its action against Amazon's Alexa, stating that companies cannot keep children's voice recordings indefinitely to train algorithms.
School management is complicated by overlapping federal laws. According to Promise Legal, the Family Educational Rights and Privacy Act (FERPA) applies directly to schools, protecting student records and giving parents review rights. Meanwhile, the Children’s Online Privacy Protection Act (COPPA) places obligations directly on tech vendors. While schools struggle to monitor every contract, vendors face heavy fines. Under recently amended COPPA rules, the FTC can fine operators up to $53,088 per violation, per day.
What This Means for Families
The level of risk depends on how an educational app is built and configured. Some major classroom tools take a cautious approach. For example, Instructure's AI assistant, IgniteAI, is disabled by default in the Canvas learning management system. Teachers or administrators must manually turn the tool on before student interactions begin. Instructure also publicly commits to not using student data to train external models, which gives schools greater oversight.
Other specialized tools rely on a constant loop of student data to function. AI tutoring platforms like Studeia's AI Tutor use multi-agent pipelines that pull snapshots of student data, such as quiz scores and concept mastery levels, directly into AI prompts to customize lessons. This creates a personalized tutor, but it also means a detailed learning profile constantly feeds into large language models.
What You Can Do
Parents and teachers can start by asking school administrators if classroom platforms have AI features switched on by default, or if they require active consent. Before approving new software, school IT departments should check if contracts explicitly forbid vendors from using student data to train general AI models. Finally, schools must ensure that contracts clearly define what happens to student data when an account is closed, including whether the vendor must use machine unlearning or model retraining to erase the student's digital footprint.