Because the decisions that shape a child's experience are made during design: what data is collected, where it goes, who can see it and what happens when something goes wrong. Once a tool is live, those choices are built into the architecture and the business model, and they are far harder to reverse.
For a child, these are not simply technical decisions. They can determine what a child understands, what they share, what they are exposed to and who is there when something goes wrong.
Regulators agree. The UK Information Commissioner's Children's Code tells services likely to be accessed by children to begin a data protection impact assessment "early in the design of your service, before you start your processing", and to complete it "before the service is launched". Under the GDPR, Article 35 requires that assessment "prior to the processing" wherever processing is likely to result in a high risk. Before launch is not a preference. Increasingly, it is the rule.
"We hadn't really thought about that"
Earlier this year, I was in conversation with a group of tech colleagues: brilliant people, building and launching new AI tools at a remarkable pace. Every week, sometimes every day, there is something new.
One of them invited me to test a financial forecasting app. On paper, it was impressive: a tool designed to project budgets, anticipate risk and support long-term planning. Useful, no doubt.
I was ready to try it, until I asked a simple question. Where does my data go? If I upload my financial history and personal data, where is it stored? Who has access to it? How is it used, now and later? Can you guarantee it won't be reused, shared or exposed?
Silence. A pause. A few exchanged looks. Then: "We hadn't really thought about that." That answer should concern all of us. Because this is not simply a technical oversight. It is a governance failure.
We are building AI at speed, but we are not always embedding accountability at the same pace. Transparency is still too often treated as something to address later. But it is not the user's responsibility to discover these risks. It is the developer's responsibility to understand them, address them and explain them before the product reaches the market.
That conversation was not about a children's product. But it made me think about what happens when the same questions are overlooked in technologies designed for, or increasingly used by, children. If we are uncomfortable sharing our own financial information without knowing where it goes, why would we expect children to navigate similar questions about their personal information on their own?
If your tool relies on personal, financial or sensitive data, and especially children's data, transparency is not a feature. It is a baseline obligation.
When children are involved, the threshold is higher
When children are involved, you are not just managing data. You are shaping experiences, opportunities, risks and rights, often for those who have the least control over how technology is designed and used.
The law already recognises this. The GDPR's Recital 38 states that "children merit specific protection with regard to their personal data, as they may be less aware of the risks, consequences and safeguards concerned", and Article 12 asks for "clear and plain language, in particular for any information addressed specifically to a child" (more in our guide to GDPR and children's data).
In the UK, the Age appropriate design code sets 15 standards, starting with the best interests of the child and including high privacy by default and data minimisation. Since 5 February 2026, section 81 of the Data (Use and Access) Act 2025 has also written "children's higher protection matters" into the UK GDPR's data protection by design duty.
In the EU, the AI Act has, since 2 February 2025, prohibited AI that exploits people's vulnerabilities due to their age in ways that materially distort their behaviour and cause, or are reasonably likely to cause, significant harm. Since 2 August 2026, Article 50 has required AI systems that interact directly with people to tell them they are dealing with AI.
Internationally, UNICEF's Guidance on AI and Children, Version 3.0 (December 2025) sets ten requirements, including "protect children's data and privacy" and "provide transparency, explainability and accountability for children". None of these can be meaningfully addressed the week before launch.
The questions every developer must be able to answer
The questions are not difficult.
- Where is the data stored?
- Who can access it?
- What is it used for? Not only now, but later: training future models, sharing with partners, building profiles or developing other products.
- What safeguards exist?
- What accountability mechanisms are in place if harm occurs?
When your users include children, I would add five more.
-
How do you know how old your users are? A tick box alone may not be enough. The method, from self-declaration to age assurance, should be proportionate to the risk, and a child's experience of the product should actually be different where appropriate.
-
What happens when a child tells your product they are being hurt? A child may confide in a chatbot or an app before telling any adult. Is there a tested route to a real person, a helpline or an appropriate reporting mechanism?
-
Who, by name, is accountable? Not "the team" or "the algorithm", but someone with the authority to pause a feature when necessary, with meaningful human oversight wherever decisions may significantly affect a child.
-
How did you seek children's views? If children were not involved before the key decisions were made, there is a real risk that the design reflects adult assumptions about what children need. I have written about where children are in the responsible AI conversation; a child rights impact assessment is one structured way to bring children's rights and experiences into the process.
But participation should go further than asking children what they think once a product has already been designed. Where appropriate, children should have meaningful opportunities to influence the technologies that affect their lives. Children are not only people who need protection from technology. They are rights-holders whose experiences and views should help shape it.
-
Can you explain all of this to a child, and to the adults around them?
If you cannot answer these questions clearly and confidently, then your tool may not be ready. Innovation without accountability is not progress. It simply redistributes risk to the user. And when that user is a child, the consequences can be much greater.
Our safety-by-design checklist turns many of these questions into practical design decisions, and this is the work we do with developers through our responsible AI advisory. For the wider picture, from how children meet AI to what regulators now expect, see our guide to child safety in AI systems.
Governance has to keep pace
AI systems are already shaping education, healthcare, public services and decisions that affect people's lives. The question is no longer whether governance should catch up with the technology. It is whether accountability is being built into these systems from the beginning.
Responsible AI requires governance, transparency and meaningful human oversight before harm occurs, not simply a response after it happens. This matters particularly where children and vulnerable communities are affected, because poorly governed systems can reinforce exclusion, bias and existing institutional failures while remaining largely invisible to those affected. Safeguarding cannot become an afterthought.
Transparency that parents, educators and children can actually use
In those early conversations, several of the tech professionals made a point I still agree with: parents, educators and children themselves need to understand not only the benefits of AI tools but also their potential risks. User-friendly guidance can help caregivers and teachers make informed decisions. But transparency has to work for children too.
The ICO's transparency standard points in the same direction: information must be "concise, prominent, and in clear language suited to the age of the child", with "bite-size" explanations at the moment a use of data is switched on.
There is also an important caution here. Monitoring is not automatically child-centred. A tool that allows a parent to read every word a teenager types may feel safer to the adult while undermining the child's privacy, trust and developing autonomy. Protecting children should not mean removing their rights in the name of protection.
Parental controls should therefore respect children's evolving capacities. Good design should make clear to the child, as well as to the parent, what information is being collected, what is being shared and with whom. Safety, privacy and participation are not competing principles that we have to choose between. Good design has to consider all three.
Trust has to come first
The developers I spoke to in 2025 knew that problems are difficult to fix after launch. The financial app conversation showed me how easily the fundamental questions can still be skipped.
Trust in AI will not be built through capability alone. It will be built through transparency, responsibility and a willingness to be held accountable. That means asking difficult questions while a product is still being designed, involving the people who will be affected by it and being prepared to change course when the risks are too great.
For children, that matters even more. AI has enormous potential to support children's learning, creativity and access to opportunities. But children should not have to exchange their privacy, safety or rights for those benefits.
That is why safety has to be designed in from the beginning. And that has to come first.
Frequently asked questions
What should an AI developer be able to answer before children use their product?
Where the data is stored, who can access it, what it is used for, what safeguards exist and who is accountable if harm occurs. For children, developers should also consider how age is established, what happens when a child discloses harm, whether meaningful human oversight is available where needed and how children's views shaped the design.
Is a data protection impact assessment required before launch?
In the UK, the ICO's Children's Code says services likely to be accessed by children should begin a DPIA early in design and complete it before launch. Under the GDPR, Article 35 requires one before processing where that processing is likely to result in a high risk.
Does the EU AI Act require chatbots to tell children they are talking to AI?
Article 50, applying since 2 August 2026, requires AI systems that interact directly with people to tell them so, unless this is obvious from the circumstances and context of use, at the latest at the first interaction. It applies to everyone, including children, and should be understood as a baseline transparency requirement rather than a complete child-safety safeguard.
Can safeguards be added after an AI tool has launched?
Some can. But fundamental decisions about data, privacy defaults, age, transparency and accountability can be much harder to change once a product is live. That is why safety-by-design work needs to begin before launch.
Sources
- Age appropriate design: a code of practice for online services, Information Commissioner's Office.
- Age appropriate design code, standard 2: Data protection impact assessments, Information Commissioner's Office.
- Age appropriate design code, standard 4: Transparency, Information Commissioner's Office.
- Regulation (EU) 2016/679 (General Data Protection Regulation), Recital 38 and Articles 12 and 35, Official Journal of the European Union, 2016.
- Data (Use and Access) Act 2025, section 81, legislation.gov.uk; in force from 5 February 2026.
- Regulation (EU) 2024/1689 (Artificial Intelligence Act), Articles 5, 50 and 113, Official Journal of the European Union, 2024.
- Guidance on AI and Children, Version 3.0, UNICEF Innocenti, December 2025.
Last reviewed . Legal provisions and dates checked against the sources listed. They should be checked against the primary sources before being relied on.