A new privacy debate is emerging around more sensitive information that could be harvested by AI-powered tools: neural data, writes Julia Gilban-Cohen, a staff writer for the Center for Digital Education, in a GovTech article.
The debate centers on neural data, which can include direct measurements of brain activity and broader forms of information that may reveal a person’s mental state. Experts warn that AI-powered systems could increasingly infer attention levels, fatigue, stress, emotional engagement, and other cognitive characteristics through eye tracking, facial analysis, voice monitoring, wearable devices, and virtual reality headsets.
The issue has prompted legislative action in several states. California, Colorado, Connecticut, Montana, and Vermont have laws addressing neural data or related privacy protections, although definitions and regulatory approaches vary widely.
Neural data is information generated by the brain or nervous system. It can include direct measurements such as electroencephalogram (EEG) readings that capture brain activity. Neurotechnology to collect this data remains far from mainstream in most classrooms. But questions are being asked whether schools and education-technology vendors could one day collect or infer information about students’ mental states — including attention, fatigue, stress and emotional engagement.
“At a high level, neural data is very specific to an individual, so once you have neural data regarding how someone’s brain operates, it’s essentially like other types of biometric data, like a fingerprint or an iris scan,” says Lily Li, founder of Metaverse Law and an attorney focused on AI and data privacy. “There’s a lack of … appreciation by a regular consumer as to how much information may be used and collected about them with respect to neural data.”
Nita Farahany, a Duke University law professor and leading scholar on neurotechnology and privacy, argues that the focus should not only be on neural data but also on “cognitive biometrics” — information collected from a variety of technologies that can be used to infer mental states, emotions, attention, fatigue or cognitive performance.
Farahany foresees increasingly sophisticated AI systems being able to make intimate inferences about what a person is thinking, feeling or experiencing cognitively.
Some education technologies on the market claim to measure attention and engagement. One example is BrainCo, a neurotechnology company whose headbands use EEG signals to monitor student attentiveness.
For now, widespread classroom use of neural-data technologies remains limited; but as AI systems become increasingly capable of inferring attention, emotion and other cognitive states, the questions raised by neural data may soon extend beyond neurotechnology itself.
The emerging debate, experts say, is ultimately about mental privacy — and whether schools should collect, or even infer, information about what students are thinking and feeling in the first place.
“If [students] feel like they’re constantly being analyzed by this really high-powered tool, they probably aren’t going to feel comfortable learning and growing,” says one expert.
GovTech
