When AI is wrong — and what your kid does about it.

The moment a kid catches the model.

A nine-year-old is working on a game. They ask Inkie to add a sound when the player picks up a coin. Inkie produces a ChangeDisclosure card. The card lists a change to a file called coin.js and a call to a function called playPickupSound(). The kid clicks Keep. The preview reloads. There’s no sound.

The kid opens the file tree. There is no coin.js. There is a coins.js, plural. The function Inkie called is also slightly off. The actual sound helper in the project is audio.play(), not playPickupSound(). Inkie wrote a confident-looking change that referred to two things that don’t exist.

That moment is the most important pedagogical moment in the studio. Up to that point the kid has been treating Inkie as an authority. After that moment, the kid is treating Inkie as a collaborator who can be wrong. The relationship has changed. That is the whole game.

What kinds of wrong Inkie actually is.

Large language models fail in characteristic ways. The research community has converged on a small vocabulary for these failures, and parents who use Tell and Show pick up the same vocabulary by watching their kid debug. Four categories cover most of what your kid will see.

Confabulation, sometimes called hallucination. The model produces output that is grammatically clean and confidently phrased but factually wrong. It names a function that was never written. It cites a documentation page that doesn’t exist. The output sounds correct because language models are optimized for plausibility, not truth. This is a known and well-documented failure mode of current LLM architectures.¹

Context window limits. The model can only consider so much of the project at one time. A change that looks right in isolation can collide with a setting somewhere else in the codebase that the model never re-read. The model didn’t lie. It forgot.

Naming errors. A subspecies of confabulation specific to code. The model has a strong prior for what a file or function "should" be called and emits that name instead of the real one. coin.js instead of coins.js. playSound instead of audio.play. Catching naming errors is one of the easiest model failures for a kid to learn, because the fix is a one-character search.

Scope drift. The kid asks for a small change. The model produces a sweeping one. The kid wanted a sound on coin pickup; the model also rewrote the player-movement code "while it was in there." Scope drift is the failure mode parents notice first, because it feels like an over-eager intern who keeps reorganizing the kitchen.

Why hiding the mistakes would be the wrong design choice.

A reasonable product decision, taken by most consumer AI tools today, is to hide the failure modes. Smooth over the rough edges. If the model confabulates, retry behind the scenes. If a function name is wrong, fuzzy-match it. Present the kid with a finished thing.

This is the design choice that makes sense if your product is a productivity tool. It is the wrong design choice if your product is a learning tool. The whole pedagogical argument for visible AI is that the kid’s mental model of the system is what they walk away with.² Hide the failure modes and the kid develops a mental model in which AI is magic. Magic that can’t be wrong. Magic that doesn’t need to be checked.

The whole pedagogical argument for visible AI is that the kid’s mental model of the system is what they walk away with. On why the failure modes stay visible

Five years from now, the kid who built that mental model is going to be sitting in front of a much more capable AI and accepting whatever it tells them. They’ll have no internal alarm. The Long & Magerko AI-literacy framework names this directly. Their second competency, "understanding what AI can and cannot do," is about developing a working theory of the model’s strengths and limits.³ The kid who has never seen a model be wrong has never started building that theory.

So the studio shows the rough edges. Not as a confession. As curriculum.

How the studio surfaces the mistakes.

The structural element doing most of this work is the ChangeDisclosure card. Before any change lands in the kid’s project, the card lists what files Inkie is about to touch, what functions it’s about to change, and a one-sentence "why." The card is rendered before the change applies. The kid has to click Keep to commit it.

That sequence does three things. It forces Inkie to commit to specifics before acting. It gives the kid something concrete to look at. And it puts the kid in the decision seat: Keep, Review, or Undo. The decision is theirs. The studio doesn’t hand them a finished artifact; it hands them a proposal.²

When the proposal is wrong, the kid sees it on the card. The card says coin.js. The kid’s file tree says coins.js. The mismatch is right there in the disclosure, before any code has been touched. Click Undo. Tell Inkie the right file name. Try again.

The tool-trace pane in the chat shows the same story from a different angle. It prints what Inkie did as it did it: reading coin.js... and then nothing, because coin.js doesn’t exist. The kid sees the model fail in real time, in a vocabulary they can read. Bret Victor’s 2012 essay on learnable systems argued that learners need this: visible state, immediate feedback, the ability to scrub backward.⁴ Ben Shneiderman made the same case for direct manipulation in 1983.⁵ The studio is the same idea, run on a new medium.

The "find a mistake" quest, and the "same prompt, two answers" quest.

Surfacing the mistakes is necessary but not sufficient. Some kids will breeze past the disclosure card and click Keep without reading it. So the studio includes a short quest line that asks kids to deliberately catch Inkie wrong.

The first quest is called Find a mistake. It unlocks once the kid has made a few sessions of progress. The brief is short: "Inkie is going to do something wrong today. Find it before you click Keep." The first time a kid completes the quest, they get a transparency badge. The first time isn’t hard. Inkie is wrong often enough that the quest completes itself if the kid is paying attention.

The second quest is called Same prompt, two answers. The kid asks Inkie for the same change twice. The studio runs the second request in a fresh context. The kid sees two different ChangeDisclosure cards. Same intent, different implementation. The point isn’t that one is right and one is wrong. The point is that the model is probabilistic. Comparison teaches what a single sample can’t.

Both quests sit inside the studio’s side-quest panel. They’re part of a longer line of AI-transparency quests that surface progressively as the kid’s account levels up. Most kids finish the first two in their first month. The transparency line isn’t a curriculum the kid has to opt into. It’s the curriculum the studio is.

What the kid walks away with.

This is the working theory we want every kid in the studio to develop, in their own words, by the time they’ve shipped a few projects:

• The model is useful, often very useful.
• The model can be wrong, and is wrong often enough that I check.
• I can tell when it’s wrong by reading the disclosure before I click Keep.
• If I run the same prompt twice, I get two answers. That’s normal.
• The decision is mine. Not the model’s.

None of those are exotic claims. A working engineer who uses AI tools at a job would phrase them the same way. The studio just gets the kid to those claims early, in a low-stakes setting, with a partner that fails often enough to keep the lesson concrete. Five years from now, when your kid is using a much more capable model to do much higher-stakes work, the working theory is going to do most of the safety work for them.

We’re honest about what the studio doesn’t solve. We haven’t fixed hallucination. Nobody has. What we’ve done is build a workflow where the next generation of kids grows up assuming AI gets things wrong sometimes, checks it before committing, and treats the decision as theirs. That’s the safer relationship to have with a tool that’s about to be everywhere.