Microsoft has signed a binding agreement to limit student data tracking and ban emotionally manipulative AI tools in schools, according to the American Federation of Teachers. This shift comes as school districts nationwide push back against classroom artificial intelligence. Both New York City and Los Angeles recently implemented major halts on student use.
What Happened
Under this new, legally binding pact, Microsoft has pledged to stop using student and teacher data to train its artificial intelligence systems. Starting November 1, 2026, these rules will apply to all of the company's school contracts. The agreement also bans any AI features designed to foster emotional dependency in children. This announcement comes as major school districts reject the rapid expansion of educational technology. The New York City public school system recently enacted a one-year moratorium on student-facing generative AI for grades 2-K through eighth grade. Meanwhile, the Los Angeles Unified School District blocked generative AI access on all district-owned devices to study the technology's long-term impacts. As we previously reported, educators increasingly question whether automated tools actually help students learn or simply act as a distraction.
The Bigger Picture
In August 2026, Google expanded Gemini in Google Classroom to users of all ages, offering custom starter prompts to guide student work. Google categorizes Gemini as a core service, which dictates how data is processed under the Google Workspace for Education Terms of Service. Unlike Microsoft, Google has not signed a broad, union-backed safety pact.
A primary target of the new Microsoft rules is "affective AI," which refers to chatbots built to mimic human feelings. Research shows that children are particularly vulnerable to these relationships. A study published in JAMA Pediatrics found that 21.1% of students surveyed used generative AI for emotional support, and this behavior was strongly linked to clinical signs of distress. researchers on arXiv warned that adolescents often develop emotional dependencies on companion chatbots, which can cause academic disruption and social isolation. A scoping review in Frontiers in Artificial Intelligence echoed these concerns. The review noted that human-like agents can trigger "dependency-like use" and displace real-world human connections.
What This Means for Families
For parents and educators, the debate goes beyond simple screen time. School leaders worry about losing the human element in classrooms. The New York City Mayor's Office emphasized the need for guardrails to protect real human connection and curiosity in early childhood. While Microsoft's agreement sets a high standard, it only applies to schools that contract directly with them. Many districts rely heavily on other platforms. As noted by EdWeek, school boards must now decide whether to follow the strict moratoriums seen in New York and Los Angeles or manage a complex web of individual vendor policies.
What You Can Do
- Ask school administrators how they manage Gemini access in Google Classroom or other integrated AI tools.
- Set firm boundaries at home to prevent children from using AI chatbots as a substitute for human friendship, given the documented psychological risks of affective AI.
- Encourage critical thinking by asking children how they think AI tools handle their personal information. This helps them understand that school tech isn't automatically private, as discussed in student privacy guidelines.