Big Brother in Schools: AI Watching Students

Schools are using AI to monitor students' online activities on school-issued devices. This is done to keep students safe, but it raises serious questions about privacy and security. Students use these devices for more than just schoolwork; they also cope with personal struggles like depression, heartbreak, and bullying. The AI tools often help counselors reach out to students who might otherwise struggle in silence. The AI software scans what students search or write online 24 hours a day. It detects potential indicators of problems like bullying, self-harm, suicide, or school violence. If the AI finds something serious, it sends a screenshot to human reviewers. If confirmed, the school is alerted. In cases of imminent danger, the company may contact law enforcement for a welfare check. The AI tools have helped counselors reach out to students who might have otherwise struggled in silence. But the technology has also outed LGBTQ+ children and eroded trust between students and school staff. In some cases, the technology has failed to keep schools completely safe. For example, a nonbinary teenager at Owasso High School named Nex Benedict died by suicide after relentless bullying from classmates, despite the school using surveillance technology.

The long-term effects of this technology on student safety are unclear. There's no independent research showing it measurably lowers student suicide rates or reduces violence. A 2023 RAND study found only "scant evidence" of either benefits or risks from AI surveillance. The study concluded that "No research to date has comprehensively examined how these programs affect youth suicide prevention. " Parents are often unaware that their children's online activities are being monitored. Even when families learn about school surveillance, they may be unable to opt out. For example, Owasso Public Schools in Oklahoma has used Gaggle since 2016 to monitor students outside of class. When Tim Reiland, the parent of two teenagers, asked if his daughter could bring her personal laptop to school instead of being forced to use a district one because of privacy concerns, the district refused. Despite the risks, surveillance technology can help teachers intervene before a tragedy. A middle school student in the Seattle-area Highline School District who was potentially being trafficked used Gaggle to communicate with campus staff. The student knew that the staff member was reading what they were writing. It was, in essence, that student's way of asking for help.

Actions