School districts are cracking down on classroom artificial intelligence, while Microsoft has signed a new safety agreement. This sudden shift shows a growing battle over how student data is protected and whether generative AI belongs in schools. For parents and educators, these new policies directly affect student privacy and emotional safety.
What Happened
As the school year began, the nation's two largest school districts took drastic steps to halt student AI use. According to a Reuters report, New York City implemented a one-year moratorium on AI tools for public elementary and middle school students. This decision stopped approximately 40 educational programs. Meanwhile, the Los Angeles Unified School District enacted an even more restrictive policy. According to Education Week, LAUSD blocked student generative AI across all grade levels on district-issued devices to comply with limits on student screen time.
To address this backlash, Microsoft negotiated a new agreement with the American Federation of Teachers (AFT). As we previously reported, this "National AI Safety & Privacy Standard" establishes legally binding rules for schools. Under this union-backed agreement, Microsoft promises not to use student or educator data to train its AI models, except under narrow security circumstances. If Microsoft violates these terms, they can be held liable for breach of contract. This is a major shift from previous voluntary tech industry promises.
The Bigger Picture
While Microsoft has agreed to strict privacy standards, other major edtech providers are expanding their AI reach with fewer safeguards. According to the Youth AI Safety Institute, Google recently turned its Gemini chatbot on by default for many K-12 student accounts. This bypassed earlier restrictions that only allowed older students access. A Google Workspace update detailed the change.
This rapid rollout has raised safety concerns. The Youth AI Safety Institute rated Google's K-12 Gemini tools as "High Risk." The institute found that students can easily bypass the "Guided Learning" mode to make the AI write entire essays for them. During testing, the chatbot also failed to recognize symptoms of severe mental health issues. Sometimes it framed ADHD and psychosis as positive traits.
This issue is important because teens are increasingly turning to AI for mental health support. Research published in Phys.org shows that up to 20% of adolescents have used AI chatbots for emotional support, and 63% of those who did kept it a secret from their parents. According to a study in the International Journal of New Findings in Health and Educational Sciences, this "therapeutic mimicry" can create deep emotional dependence. This dependence delays actual professional help and replaces real human connection.
What This Means for Families
These developments mean parents can no longer assume their children's school-issued devices are private or safe. While the AFT and Microsoft deal is a step forward, school districts must actively write these protections into their contracts. The rules do not apply automatically to existing accounts.
Other companies are tracking student activity by default. For example, Google's current settings include an opt-out feature called Personalized Recommendations and Search Services History. This feature actively saves students' search activity and location data. Unless parents or administrators manually disable this, Google continues to harvest student data.
What You Can Do
- Ask your school board if they have officially integrated the AFT-Microsoft safety standards into their technology contracts to legally protect student data.
- Log into your child's school Google account settings and opt out of "Personalized Recommendations and Search Services History" to prevent location and activity tracking.
- Talk directly to your child about the limits of AI. Explain that chatbots cannot offer reliable mental health advice or replace real human friendships.