A model is only as good as the pile you hand it
Meet the Dataset Engine: a free, tap-only studio where kids label 60 drawn pictures, check their work with a second labeler, clean up the pile, and watch the same tiny model do better on their set than on the mess they started with.
November 17, 2026 · Hi, Bot
Most talk about AI is about the model. Bigger, faster, smarter. Kids hear it too, and they come away thinking the model is where all the cleverness lives.
The people who build these systems tell a different story. A lot of the real work happens before any training starts: someone decides what goes in the pile, what each thing is called, and what gets thrown out. That job has a name, data curation, and almost nobody explains it to kids.
So we built a studio for it. The Dataset Engine is free, runs in the browser, and has nothing to type. It's for ages 10 to 16 and the grown-ups next to them.
The pile arrives messy, on purpose
You start with 60 drawn pictures of bots and critters. Each one already has a label, stuck on by a pretend web scraper, and the pile has the problems real piles have:
- It's lopsided, with more of one kind than the other.
- Six pictures show up twice.
- The scraper has a bad rule: it calls anything on wheels a bot. A critter on a skateboard gets the wrong label every time.
None of that is hidden. The point is to find it.
Five steps, the way real teams do it
Guideline. Before anyone labels anything, you write the rule sheet. What is this set for? When a picture is a mix of parts, what decides it? What happens to a picture that appears twice? You pick from a few choices, and those choices follow you through every later step.
Label. You go through the pile and give every picture your own call, following your guideline.
Second labeler. Real teams have two people label the same pictures, then check how often they agree. Hand the device to someone else in the room, or use the practice labeler, which is right about 9 times in 10. The engine scores your agreement with Cohen's kappa, a number that doesn't give you credit for agreeing by luck. Pictures you disagree on stay undecided until you make a final call.
Clean and split. Remove the copies. Balance the two kinds if you want to. Split off a test set, about 80/20, and the engine warns you if the same picture ended up on both sides. That's called a leak, and it makes a model look smarter than it is.
Check. The same tiny model, with the same settings, learns from the raw pile and from your set. Then both take a test on 40 pictures nobody has seen.
The claim, and how we know it holds
Here's the lesson the engine is built around: the choices you make about data can matter more than the choices you make about the model.
We didn't want to just say that, so the engine's tests check it. With careful labels, we tried nine combinations of model settings: three learning speeds, each trained for three different lengths. The raw pile's best score never reached a carefully labeled set's worst. A set beats the raw pile even with the practice labeler as the second labeler, once you settle the pictures you disagreed on.
No amount of turning the model's knobs rescued the pile. Fixing the pile did.
Try this in 10 minutes
- Open the Dataset Engine and pick a guideline. Any one will do for a first run.
- Label all 60 pictures. Watch for the critter on the skateboard.
- Choose the practice labeler, then settle the pictures you disagree on.
- Remove the copies and split the set.
- On the last step, compare the two scores. Then ask your kid why the raw pile did worse. If they say "the wheels," they've got it.
What you can take with you
Everything you make downloads for free: a ZIP with train and test folders of PNG pictures (the layout common image trainers read), a labels.csv, and a datasheet.md that says what your set is good for, what it isn't for, and where its gaps are. Writing that datasheet is part of the job too. The Activities Pass adds one extra, a printed booklet of your datasheet, but nothing in the studio needs it.
The pictures stay on your device. Nothing you label gets uploaded.
Next time a chatbot gets something confidently wrong, there's a good question to ask at the dinner table: what do you think was in its pile?
› Keep reading
Get the next dispatch.
Occasional, never spammy.