Is AI safe for kids?

What data does the AI actually collect about my kid?

This is the first question almost every parent asks, and it’s the right one. AI products are software, and software stores things. So the question becomes: what gets stored, where does it live, and who can see it.

The honest answer for any AI product, ours included, is that the kid’s prompts and the model’s responses pass through a model provider’s servers in order to generate a reply. For minors that requires extra care under COPPA in the US and similar regimes elsewhere. The kind of careful product takes three positions: it minimizes what gets sent, it tells you what it sent, and it gives you the controls to delete or export the record.

The studio’s posture, which you can read in full at our privacy page, is that kids don’t have their own logins. Parents own the account. The kid’s creative output stays on disk locally and the only thing sent to the model is the scoped prompt for the next change. We don’t train models on kid work. We don’t sell data. We don’t serve ads against kid content. Those aren’t marketing claims; they’re the architecture.

If you’re evaluating any AI product for a kid, ask the same three questions of the vendor: what gets sent off-device, what is retained, and what controls do I have. If the answers are unclear, that’s the answer.

What might my kid be exposed to inside the tool?

General-purpose AI assistants will respond to almost any prompt a user types. That’s their job. A kid using a general-purpose AI is one curious search away from material a parent would not have chosen.

This is the part of the safety question that depends most heavily on the product’s scope. A scoped product, one whose AI is wired only to certain kinds of tasks, is exposing the kid to a smaller surface area. The studio is scoped. The AI’s job is to propose changes to a game, story, site, or short film the kid is building. It is not a general chat partner. There is no “ask me anything” box. The wizards have intents. The intents are kid-appropriate by design.

Common Sense Media’s long-running census of media use among US kids and teens documents how much of a child’s screen time is unstructured, with general-purpose tools and feeds doing most of the exposure work.¹ The mitigation isn’t to ban AI; it’s to choose tools whose surface area was designed for kids in the first place. Our safety page spells out the content guidelines, the rated content settings, and the parent-approval gate before anything publishes.

If a tool was designed for adults and then a “kid mode” was added, the scope question is worth asking with some skepticism. Retrofitting a chat surface doesn’t make it kid-appropriate. Scoping the AI to a small, observable set of jobs does.

Is the AI talking to my kid about age-appropriate things?

Two distinct concerns sit inside this question. One is content rating: will the AI ever produce something violent, sexual, or otherwise inappropriate. The other is emotional posture: will the AI position itself as a friend, therapist, or confidant in ways that don’t belong in a kid’s life.

On content rating, every AI product for kids needs filtering. The studio has rated content guidelines tied to project type and a parent-set content ceiling. Some tracks (Game) tolerate more “peril” than others (Story-for-younger-readers). The ceiling lives in the parent settings, not in the kid’s prompt box.

The emotional-posture question is more interesting and underdiscussed. The American Academy of Pediatrics’ 2016 policy statement on media and young minds argued that children develop best when their primary relationships are with people, not screens, and that media works best when it supplements rather than replaces human interaction.² An AI that markets itself as “your kid’s new best friend” is taking the wrong position relative to that finding.

Inkie, the studio’s AI partner, is a tool. Not a friend. The voice is warm and helpful, but it doesn’t pretend to know the kid’s life, doesn’t remember birthdays, doesn’t simulate emotional intimacy. The product positioning matters as much as the filtering does.

What about when the AI just confidently makes things up?

Large language models hallucinate. They produce confident, fluent output that is sometimes factually wrong, in ways that have been documented across the research literature.³ A kid who treats AI output as authoritative is going to be misled some of the time, and the smooth fluency of the model is the very thing that hides the error.

There are two design choices a product can make about this. Hide the failure (retry behind the scenes, fuzzy-match wrong names, present finished output) or surface it (show the kid what the model proposed, give them a beat to read it, let them keep or undo). We made the second choice for pedagogical reasons covered in When AI is wrong: the kid who learns AI through visible mistakes builds a working theory of when models fail. The kid who learns AI through smoothed-over output learns nothing about when not to trust it.

What the kid sees inside the studio is a small card the moment Inkie finishes proposing a change. Not a confirmation popup, not a notification — a structured surface showing what the model touched and why. A miniature version of it looks like this:

The kid who learns AI through visible mistakes builds a working theory of when models fail. The kid who learns AI through smoothed-over output learns nothing about when not to trust it. On why hallucination is a feature of the curriculum, not a bug to hide

The second of Duri Long and Brian Magerko’s five AI-literacy competencies, “knowing what AI can and cannot do,” is exactly the skill that surface-the-mistake design produces.⁴ A kid who has clicked Undo on a confidently wrong ChangeDisclosure card a few dozen times has internalized something most adults using ChatGPT today still haven’t: the model is fluent, useful, and routinely wrong.

Isn’t AI just more screen time?

This question often hides two different worries. One is a worry about the total minutes a kid is in front of a screen. The other is a worry about what happens during those minutes. They aren’t the same.

The AAP’s 2016 guidance acknowledged what every parent already knew: not all screen time is equivalent. Co-viewing high-quality programming is different from passive YouTube autoplay. Building something is different from scrolling.² The studio is on the “building something” end of that spectrum. The kid is making decisions, naming characters, watching their own choices propagate through their project.

We’ve written separately about why we deliberately don’t track session minutes; the short version is that what happens in a session matters more than how long it lasts.⁵ A 90-minute session where a kid is iterating on their game and showing it to a sibling is not the same animal as 90 minutes of TikTok. Reducing both to one number called “screen time” loses the thing parents actually care about.

If the studio belongs in your family, it belongs alongside other things. Not as a replacement for them. That’s in the product. The wizards run faster than a kid can watch them, so the kid usually wants to go run around between iterations. That’s by design too.

Will my kid get too reliant on it?

The deepest version of the safety question is cognitive. A kid who outsources every hard problem to AI doesn’t build the muscle that makes hard problems doable. The fear is real and the research base on productive struggle is well-developed: students who work through difficulty before being shown the answer learn more durably than students who are shown first.⁶

So the design question for an AI product becomes: does the AI hand the kid a finished thing, or does it propose a move the kid then evaluates and decides on? Those produce different learners. A homework-doing AI produces dependency. A scoped partner who proposes specific changes to a kid’s artifact produces a different kid: one who is constantly evaluating, choosing, and steering.

Mitchel Resnick’s work at the MIT Media Lab’s Lifelong Kindergarten group has spent two decades arguing that creative tools for kids should have a low floor, wide walls, and a high ceiling.⁷ The AI partner version of that is: the partner makes the floor low (kids can ship) without raising it (the kid still does the choosing). The over-reliance pattern shows up when AI is wired to do the choosing too. The studio’s Keep / Review / Undo decision row exists exactly to keep the choosing on the kid’s side.

Is AI safe for kids? The shape of the honest answer: some AI products are, some aren’t, the question depends on the product, and the questions worth asking are the six above. Walk through them with whatever tool your kid is using. If the answers hold up, the tool probably does too.