# AI in the Classroom: Better Learning or Less Privacy?

> AI tutors and learning analytics promise personal teaching, but can weaken thinking and collect sensitive data. How schools can get the benefit, not the cost.

URL: https://www.clarityailab.com/insights/algorithmic-classroom-ai-innovation-privacy-critical-thinking/

AI in the classroom can personalise learning, but the same systems can weaken the thinking they are meant to build and collect detailed data about children. Whether schools get the benefit or the cost depends less on whether they use AI than on how the tools are designed, what data they keep, and who checks them.

This article is for school leaders, teachers and parents weighing AI tutors and learning analytics. It sets out the evidence on learning, the privacy risks, and a short list of checks before adopting a tool.

## How widely do students use AI for schoolwork?

Fast-rising use is already a fact. The [Pew Research Center](https://www.pewresearch.org/short-reads/2025/01/15/about-a-quarter-of-us-teens-have-used-chatgpt-for-schoolwork-double-the-share-in-2023/) found that 26% of US teens aged 13 to 17 had used ChatGPT for schoolwork, double the 13% in 2023. Schools are deciding how to manage AI that students already use, not whether to allow it in.

## Does AI in the classroom improve learning?

It can, if it is built to teach rather than to answer. The clearest evidence comes from a field experiment with nearly 1,000 high school maths students in Turkey, [published in PNAS](https://doi.org/10.1073/pnas.2422633122). As [Knowledge at Wharton](https://knowledge.wharton.upenn.edu/article/without-guardrails-generative-ai-can-harm-education/) summarises it:

- Students with a standard chatbot did 48% better than the control group during practice, but 17% worse on a later exam taken without it.
- Students with a tutor version that had safeguards did 127% better during practice and about the same as the control group on the exam.

*Analysis:* an AI that hands out answers improves the work students hand in, not what they learn. An AI that makes them do the thinking at least does no harm, and can help them practise more.

## Can AI weaken students' critical thinking?

There are warning signs. A 2025 study by [Michael Gerlich](https://www.mdpi.com/2075-4698/15/1/6) of 666 participants found a significant negative correlation between frequent AI use and critical thinking, linked to cognitive offloading: letting the tool do mental work the person would otherwise do. Younger participants showed more dependence on AI and lower critical thinking scores. The study shows an association, not cause and effect, but it points the same way as the classroom experiment.

## What are the privacy risks of AI in schools?

AI tutors and learning analytics work by collecting data: answers, mistakes, time on task, sometimes writing, voice or video. That record can be sensitive and long-lived.

- **High-risk uses.** The EU AI Act's [Annex III](https://artificialintelligenceact.eu/annex/3/) classes AI as high-risk when it is used to decide admission, evaluate learning outcomes, assess the level of education a student will receive, or monitor students for prohibited behaviour during tests. Those uses carry extra obligations.
- **Children's data.** In the United States, the [COPPA Rule](https://www.ecfr.gov/current/title-16/chapter-I/subchapter-C/part-312) requires verifiable parental consent before collecting personal information from children under 13. It also says such data may be kept only as long as is reasonably necessary for its purpose, never indefinitely, under a written retention policy.
- **Children's rights.** UNICEF's [*Guidance on AI and children*](https://www.unicef.org/innocenti/reports/policy-guidance-ai-children) sets ten requirements for child-centred AI, including ensuring safety, protecting children's data and privacy, and providing transparency and accountability.

*Analysis:* the deeper risk is labelling. A system that predicts which students will struggle can help teachers step in early. The same prediction, kept and shared, can follow a child and shape how others see them.

## What do the optimists get right?

Personal feedback at scale is valuable. Teachers cannot sit with every student, and a well-designed tutor can give hints, explain mistakes and let a student practise at their own pace. Early-warning analytics, used carefully, can help teachers reach students who would otherwise fall behind. The goal is to keep these benefits while limiting the data and protecting the thinking.

## What should schools check before adopting an AI tool?

1. **Does it teach or tell?** Prefer tools that give hints and ask questions over tools that hand out finished answers.
2. **What data does it collect, and for how long?** Ask for the retention policy in writing; refuse indefinite storage.
3. **Who sees the predictions?** Keep risk scores with teachers, not on permanent records.
4. **Is a human in charge of decisions?** Do not let AI alone grade, admit or discipline students.
5. **Is it high-risk under law?** Check whether the use falls under the EU AI Act's education categories or child privacy rules where you are.
6. **Teach the tool, too.** Show students how it works and where it fails, so they use it to think, not to stop thinking.

For how assessment needs to change, see [How Should Schools Assess Learning in the AI Era?](/insights/beyond-the-prompt-redefining-educational-rigor-in-the-era-of-ai/).

### Does AI help students learn?

It depends on the design. In a field experiment with nearly 1,000 high school maths students, an unrestricted chatbot raised practice scores but lowered later exam scores, while a tutor that gave hints instead of answers avoided the drop.

### Does using AI weaken critical thinking?

Some studies suggest a link. A 2025 study of 666 participants found that frequent AI use was associated with lower critical thinking, through cognitive offloading, especially among younger people. It shows a correlation, not proof of cause.

### Is AI in schools regulated?

Increasingly. The EU AI Act classes AI used to admit students, evaluate learning outcomes, decide the level of education a person receives, or monitor students during tests as high-risk. In the US, COPPA governs personal information from children under 13.

### How long can an education app keep a child's data?

Under the US COPPA Rule, only as long as is reasonably necessary for the purpose it was collected for. It may not be kept indefinitely, and operators must have a written data retention policy.