Major changes are reshaping how students interact with artificial intelligence in school. Tech giant Microsoft has agreed to sweeping student privacy rules, while the country’s two largest school districts have temporarily halted classroom AI use. These moves come amid a growing national debate over how to protect student data and emotional well-being from rapidly expanding classroom technology.
What Happened
Microsoft announced a legally binding agreement with the American Federation of Teachers (AFT) to establish strict AI safety and privacy standards in classrooms. Under this new policy framework, Microsoft will not use student or educator data to train its AI models, nor will it sell this data or use it for advertising. The agreement also prohibits Microsoft from creating AI features designed to cause emotional dependency or to keep students online longer than a task requires. Starting November 1, Microsoft will apply these rules to all school contracts nationwide.
This tech-industry shift occurs as school districts take matters into their own hands. As we previously reported, the nation's two largest school districts have put the brakes on student generative AI. According to The American Quorum, New York City public schools launched a one-year moratorium on student-facing generative AI for grades 3-K through 8, shutting off AI capabilities in over 38 educational products. Meanwhile, administrators in the Los Angeles Unified School District placed a temporary ban on student generative AI use on district-owned devices to comply with goals to limit overall student screen time, as reported by Education Week.
The Bigger Picture
While Microsoft has signed on to these guidelines, other tech providers are expanding classroom AI access. Google recently expanded its Gemini in Classroom tool to students of all ages, which has raised concerns about student safety. While school IT departments can use administrative controls to manage access to Gemini, Google has not signed the AFT's binding privacy agreement.
The ban on emotionally manipulative AI features points to a growing psychological concern among educators. Recent academic research shows that students are increasingly turning to chatbots for companionship rather than just schoolwork. According to a study published in Scientific Reports, a distinct group of "Attached Users" view AI as human-like companions. These users report higher rates of loneliness and lower real-world social satisfaction. Research published in ResearchGate indicates that the emotional bonds students build with AI can mimic human psychological attachment. These findings make rules against emotionally engaging AI features important for protecting student mental health.
What This Means for Families
For parents and teachers, this means the digital tools kids use every day are changing rapidly. Depending on where you live, your student may find certain AI features disabled on classroom devices. While some areas are pushing back, other states are passing legislation to protect kids. For instance, laws like California’s AB 1159 are already stopping edtech companies from training models on student data.
However, because federal laws have not caught up, privacy protections remain unequal. If your school uses Google or other platforms that have not signed the AFT agreement, your child's data might not have the same protections as those using Microsoft products.
What You Can Do
- Ask your school which AI tools are active in your child’s classroom and whether the district has signed specific privacy agreements with tech providers.
- If your school uses Google Workspace, ask administrators how they manage access to Gemini and other generative tools for different age groups.
- Talk to your children about how they interact with AI, emphasizing that chatbots are task-oriented tools rather than friends to help protect them from the social isolation and dependency risks identified by researchers.