Ten Questions DSLs Should Ask About AI Wellbeing Tools
Students are already using AI, and vendors are knocking on school doors. Before allowing any AI wellbeing or study tool, a designated safeguarding lead (DSL) and senior leaders need clear answers. These ten questions apply to any product, including ours.
This is a practical checklist, not legal advice. Take advice from your data protection officer and align your decisions with your own safeguarding, online safety and filtering and monitoring policies and with current Department for Education guidance, including Keeping Children Safe in Education and the DfE's guidance on generative AI in education.
1. Who is it for, and how is age enforced?
What is the minimum age? What happens if a younger child signs up? Is there an age gate, and what is it based on?
2. How does it detect and respond to risk?
Ask exactly what it looks for (self-harm, abuse, severe distress), how, and what the user sees when it triggers. Ask about false negatives: no automated check is perfect, and a good vendor will say so.
3. Who is told, and how quickly?
Does it alert anyone: the school, parents, the vendor's team? Within what time? How does that fit into your existing safeguarding pathway? If it does not alert anyone, that is acceptable only if everyone understands it clearly.
4. What does it do with the data?
Who is the data controller? Which sub-processors see conversations (including the AI provider)? Where is data processed? Is it sold or used for advertising or model training? Ask for it in writing.
5. How long is data kept, and can it be deleted?
Ask for specific retention periods for conversations, accounts and safeguarding logs, and how a student or school can have data deleted.
6. Is a DPIA needed, and what support will the vendor give?
Your DPO will decide whether a DPIA is required. Ask what information the vendor can provide to support one, and how it aligns with the ICO's Age Appropriate Design Code (the Children's Code).
7. What content rules apply?
How does the tool keep language and topics age-appropriate? What will it refuse to do? Ask to see how it handles sexual content, drugs and alcohol, violence and requests for personal information.
8. Is it honest about being AI, and about its limits?
Does it identify itself as AI? Does it avoid presenting itself as therapy or a counsellor? A tool that implies it can replace trained people is a red flag.
9. Is it designed to maximise screen time?
Look for streaks, guilt-based notifications and engagement hooks. Prefer tools with built-in limits and a clear message that closing the app is fine.
10. How do we pilot it, review it, and leave?
Start small, with a defined group and clear success criteria. Agree how you will review it with staff, students and parents. And check how easily you can stop, and delete the data, if it is not working.
Red flags
- Vague answers on data, sub-processors or retention.
- Claims to be therapy, or to replace counsellors.
- No clear age policy.
- Promises of "100% safe" or "fully compliant" with nothing to back them.
- Hooks designed to keep students chatting.
How withsonder answers these
We have written our answers in a printable school information pack, including what we do not yet do, such as school-wide accounts and automatic alerts to schools. If you want to talk it through, register your school's interest or email hello@withsonder.ai.
Frequently asked questions
What is a DPIA and does a school need one for an AI tool?
A Data Protection Impact Assessment (DPIA) is a structured assessment of data protection risks, required under UK GDPR where processing is likely to be high risk, which can include new technologies used with children's data. Schools should take advice from their data protection officer on whether one is needed.
Should an AI wellbeing tool alert the school about concerns?
It depends on how the school wants its safeguarding process to work. If a tool can alert, the school needs to know who is notified, how quickly, and how that fits with its existing safeguarding policy. If it cannot alert, the school needs to be clear that the tool does not replace its own processes.
Can an AI wellbeing tool replace counselling or pastoral care?
No. At best it can complement existing support. Schools should be wary of any tool that presents itself as therapy or as a substitute for trained people.
About withsonder
withsonder is a UK-built AI companion app for ages 13 and over, designed around the principles in this guide: clean language, teach-don't-cheat study support, daily limits and UK crisis signposting. It is not therapy or a crisis service. Read the parent's page or see how schools use it.