A Safety-by-Design Checklist for Digital and AI Products Used by Children

Most product teams now accept that a service used by children has to be safe for them. Fewer can say what that means when a feature is half-built and a launch date is fixed. The frameworks are plentiful, from Australia's eSafety Commissioner to the UK Information Commissioner's Office, Ofcom, the European Commission and UNICEF, but they are written as principles and codes rather than as a list a team can work through. This article turns them into one.

What is a safety-by-design checklist for children's digital and AI products?

A safety-by-design checklist is a stage-by-stage list of concrete, checkable decisions that a team makes before and during development so that a product is safe for children by default. It draws on regulatory codes and international guidance, and it produces evidence: every item should leave a record showing what was decided, when and why.

The frameworks behind the checklist

The main frameworks agree on far more than they disagree on. The checklist below draws on seven of them:

• eSafety Commissioner, Safety by Design. Australia's framework rests on three principles: service provider responsibility, user empowerment and autonomy, and transparency and accountability. eSafety publishes free Safety by Design assessment tools, one for start-ups and one for mid-tier and enterprise companies, which any organisation in the world can use.
• ICO Age Appropriate Design Code. Fifteen standards for online services likely to be accessed by children in the UK, led by the best interests of the child and covering defaults, data minimisation, geolocation, profiling and nudge techniques.
• Ofcom Protection of Children Codes. More than 40 practical measures under the Online Safety Act, in force since 25 July 2025, including safer feeds, age checks, reporting and a named person accountable for children's safety.
• European Commission guidelines under Article 28 of the Digital Services Act. Published on 14 July 2025, they set out what a high level of privacy, safety and security for minors looks like on online platforms.
• UNICEF Guidance on AI and Children. Version 3.0, released in December 2025, sets out ten requirements for child-centred AI and now addresses generative AI and AI-generated child sexual abuse material directly.
• IEEE 2089-2021. An age-appropriate design process standard developed with 5Rights Foundation.
• OECD Recommendation on Children in the Digital Environment. Adopted on 31 May 2021, it balances protection from risk with the opportunities the digital environment offers children.

Each item below should be answerable with yes, no or not applicable, and each "not applicable" should have a written reason.

Stage 1: Before you build

1. Decide whether children will use it. Use evidence, not your terms of service. If children are likely to access it, treat it as a children's product.
2. Complete a child rights impact assessment before design is fixed. A child rights impact assessment identifies which children could be affected, how, and which rights are engaged, including participation, information and play as well as protection.
3. Complete a data protection impact assessment that accounts for the different ages, capacities and developmental needs of the children who will use the service.
4. Map harms by feature, not by product. List every feature that lets a child be contacted, be seen, be recommended content, spend money or share location, and assess each one separately.
5. Record who signed off. Name the senior person who accepted the residual risk.

Stage 2: Defaults and privacy

• Children's accounts are private by default, so their profile, content and connections are not visible to people they have not accepted.
• Only the data needed for the feature the child is actively using is collected.
• Profiling and personalised advertising are off by default for children unless there is a demonstrable reason in the child's best interests.
• Geolocation is off by default, a clear sign shows when it is active, and any setting that makes a child's location visible to others reverts to off at the end of the session.
• No design nudges a child towards weaker privacy settings or longer use.
• If parental controls allow monitoring, the child is told, in words they understand, that they are being monitored.

Stage 3: Age assurance proportionate to risk

• The level of age assurance matches what is being gated. Pornography and content promoting self-harm, suicide or eating disorders call for strong checks; low-risk features do not.
• Self-declaration is not relied on where the risk is high. Ofcom does not accept it as highly effective age assurance.
• The age assurance method has been tested for accuracy across ages, skin tones and disabilities, and there is a route to challenge a wrong result.
• Age assurance data is deleted or minimised once the check is complete, so the check does not create a new store of children's data.

Stage 4: High-risk features

• Contact. Adults who are not connected to a child cannot message them by default. Children are not recommended as connections to unknown adults. Children can accept or decline group chat invitations, and can block, mute and disable comments.
• Feeds and recommender systems. Harmful content is filtered out of children's recommendations, not only removed after a report. Children can tell the system what they do not want to see, and explicit signals carry more weight than inferred engagement.
• Compulsive design. Streaks, visible like counts, read receipts, "is typing" indicators, infinite scroll and autoplay are off by default for children or removed.
• Livestreaming. Children cannot broadcast to strangers by default, viewers cannot send gifts or private messages to a child who is live, and moderators can end a stream quickly. Ofcom has consulted on further livestreaming protections aimed at grooming.
• Payments. No loot boxes, pressured purchases or undisclosed paid promotion aimed at children.

Stage 5: AI-specific controls

AI features raise distinct risks because they generate responses rather than host them. A responsible AI review of a child-facing product should confirm each of the following.

• Disclosure. Children are told, clearly and throughout the interaction, that they are talking to an AI and not a person, together with its limits, in language suited to their age.
• Not on by default. AI chatbots and companion features are not switched on by default or promoted to children on a platform unless an assessment supports it.
• No simulated relationships. The system does not claim feelings, discourage a child from speaking to trusted adults or use emotional pressure to keep a conversation going.
• Content filters for children. Input and output filters block sexual content, sexualised role play, self-harm and suicide instruction, eating disorder content and dangerous challenges, and filters are tuned for children rather than inherited from an adult product.
• Child-specific red-teaming. Before launch and after every significant model change, testers try to extract sexual content involving minors, grooming-style dialogue, self-harm encouragement and age-inappropriate advice, including through multi-turn and role-play attacks. Findings and fixes are logged.
• Training data. Datasets are screened against known child sexual abuse material hash lists, images of real children are not used without a lawful and documented basis, and image models are tested to confirm they cannot generate sexualised imagery of children. AI-generated abuse imagery is already a present harm.
• Escalation to humans. When a child discloses abuse, self-harm or suicidal thoughts, the system stops generating ordinary replies, gives age-appropriate help information for the child's country and, where the service's safeguarding policy provides for it, routes the case to a trained person.

Stage 6: Reporting and response

• A child can report a person, a piece of content or an AI response in no more than a few taps, from the place where the problem happened.
• Reports from children are prioritised, and the child is told what happened next.
• Moderators are trained in child protection and have a documented route to law enforcement and to national reporting bodies for child sexual abuse material.
• There is a tested crisis procedure for a live risk to a child, with named people on call.

Stage 7: Transparency to children and parents

• Privacy information, community rules and safety tools are explained in versions written for different ages, not only in legal text.
• Parents and carers can see what protections are in place and how to use them, without the product relying on parents to make it safe.
• The organisation publishes how it enforces its own rules and how well its safety features work. eSafety treats this as a core part of transparency and accountability.

Stage 8: Testing with children

• Children of the relevant ages have been consulted on the design, safely and with informed consent from them and their parents or carers.
• Usability testing confirms that children can actually find and use the safety tools and reporting routes.
• Disabled children and children from different language backgrounds are included.

Stage 9: Governance and review

1. A named senior person is accountable for children's safety, as Ofcom's codes require of in-scope services.
2. A senior body reviews risk to children at least once a year.
3. Assessments are repeated before any significant change to features, recommender systems or AI models.
4. Staff who design, build, moderate and market the product are trained in child safeguarding, and the organisation has a safeguarding policy that covers online and AI risk.
5. Third-party components, including AI models and age assurance vendors, are covered by contracts that require evidence of their own child safety controls.

What this means for organisations

For a platform or AI developer, the checklist is a working record of compliance. Ofcom, the European Commission and national regulators increasingly ask what was assessed before launch, and the answer is only credible if it was written down at the time. A dated impact assessment and a red-teaming log carry far more weight than terms of service. The UK and EU regimes differ in detail but converge on this point.

For schools, NGOs, health providers and public bodies that buy or deploy digital and AI tools, the same list works as a procurement questionnaire. Ask suppliers for evidence against each stage, especially defaults, AI filters, escalation to humans and reporting. Staff who choose and use these tools also need safeguarding training that covers digital and AI risk, because the safest product can still be deployed unsafely.

Frequently asked questions

Is safety by design a legal requirement? The term itself is not a single legal duty, but its components are. The UK Online Safety Act, the ICO Age Appropriate Design Code, and the EU Digital Services Act all require risk assessment, protective defaults and effective reporting for services used by children.

Does this checklist apply if our product is not aimed at children? Yes, if children are likely to use it. UK and EU rules turn on whether a service is likely to be accessed by children or accessible to minors, not on who it was designed for.

How is an AI chatbot different from a social media feature? A chatbot generates new content in response to a child, so harm can arise without any other user being involved. That is why disclosure, child-specific content filters, red-teaming and escalation to humans are separate items on the checklist.

Where should a small team start? With the eSafety Commissioner's free start-up assessment tool and a child rights impact assessment of the highest-risk feature. Those two steps surface most of the decisions that matter.

Regulatory details cited here are accurate as of September 2026 and should be checked against the eSafety Commissioner, the ICO, Ofcom, the European Commission and UNICEF before being relied on in a compliance decision.