Data protection for AI in learning: GDPR and AI Act

Anyone using AI in learning works within two legal frameworks. The General Data Protection Regulation applies to all processing of personal data, including questions to an AI tutor, voice recordings, knowledge levels and exam results. The AI Act (Regulation (EU) 2024/1689) governs AI systems themselves: which practices are prohibited, which uses count as high-risk and which transparency obligations apply.

Both sets of rules apply side by side. An AI system can be permissible under the AI Act and still breach the GDPR if it processes data without a legal basis, and vice versa.

The first step is a clear picture of the data flows. For a typical question in a course, spoken language is converted to text, the text is sent with the course content to a language model, the answer is checked, converted into speech and, where applicable, spoken by an avatar. Each of these steps can take place with a different service and in a different location.

Article 5 of the AI Act prohibits certain practices, including AI systems that infer the emotions of natural persons in the workplace and in educational institutions, except for medical or safety reasons. These prohibitions have applied since 2 February 2025. A system that tries to infer from camera images whether learners are bored or stressed is therefore impermissible in principle in education.

Annex III classifies as high-risk, among others, AI systems that decide on access or admission to educational institutions, evaluate learning outcomes, determine the appropriate level of education or monitor prohibited behaviour during tests. Strict obligations apply to such systems, including risk management, data quality, documentation, logging, human oversight and accuracy. The exact date from which these obligations apply should be checked against the current version in the Official Journal.

The GDPR also sets limits. Under Article 22, people have the right in principle not to be subject to a decision based solely on automated processing which produces legal effects concerning them or similarly significantly affects them. An exam grade that decides pass or fail can be such a decision.

The practical consequence: AI may make suggestions, such as a pre-assessment against a rubric. The decision is made by a person who actually reviews the suggestion and can depart from it. Merely rubber-stamping is not enough.

Learners should know where AI is used, what it does and what happens to their data. That includes an understandable privacy notice, visible labelling of AI speakers and AI answers, and an answer to the question of who makes assessments.

For the organisation itself, the record of processing activities, data processing agreements with all services and, where risk is high, a data protection impact assessment are mandatory. A vendor should provide an overview of which type of service is used for what, where it operates and what it gets to see.

In Universe, language models receive course content as context and the question, not names or account data. Speech recognition runs in the EU and would rather refuse service than fall back to a provider outside the EU; avatars run on self-operated GPU servers in the EU. Each tenant decides which text models are used; one runs in the EU (Paris). AI neither grades nor decides: for written work there is a grade suggestion, a person sets the grade, and there is deliberately no AI score for cheating.