Skip to content
Hi, Bot
Back to blog
philosophyparents

When the chatbot runs out of answers

A chatbot is very good at the first answer. Here's how kids ages 8 to 16 practice the second question, and what the research does and doesn't show.

November 18, 2026 · Hi, Bot

Kids have figured out something useful. You can ask Siri, ChatGPT, Claude or plenty of other tools almost anything and get an answer in a few seconds. To a kid, that can feel like magic. Ask, and an answer appears. Siri, ChatGPT, and Claude are trademarks of their respective owners.

It isn't magic, though. A chatbot is a very good guesser. It predicts likely next words from patterns in an enormous pile of text, and that's why it sounds so sure of itself even when it's wrong. Our five minute explainer on what people mean by AI says it plainly: AI is pattern prediction.

The question we keep coming back to is simple. What does a kid do when the first answer runs out?

The first answer is the easy part

Most everyday questions have an obvious answer, and a chatbot will hand it over quickly. That's a good thing, since nobody needs to spend twenty minutes finding the boiling point of water.

The trouble starts at the edges. Sometimes the answer is thin, and sometimes it's out of date, invented, or a fluent summary of the wrong thing. Chatbots rarely say "I don't know." When they fail, you usually get a smooth answer that ends the thinking too early, and an empty screen is rare.

Most adults learned to dig before they had a shovel this good. We learned to rephrase a question, break it into smaller ones, stick with it across several tries, and weigh what came back. Kids today are getting the shovel first, so they need deliberate practice at the parts the shovel can't do.

Here's how we'd put it. Kids are growing up in a world that is answer rich and question poor. Answers are everywhere, and what's scarce is practice at asking good questions.

What the research does and doesn't show

This is a young field, and it's easy to overclaim. Below is what we think the studies support.

A 2025 field experiment in PNAS gave nearly 1,000 high school math students in Turkey access to GPT-4 during practice. Students with an unrestricted chat did better on the practice problems, then scored 17% lower than students with no AI on the exam that followed. A version built to give hints instead of answers mostly avoided that drop. It was one school, one subject and a short time frame. Even so, we think the lesson is clear enough. How a kid uses the tool matters more than whether they use it.

A survey of 319 knowledge workers by Microsoft Research and Carnegie Mellon (CHI 2025) found that people more confident in AI reported doing less critical thinking, while people more confident in their own skills reported doing more. Those are adults describing their own habits, so the survey shows an association and can't show a cause.

You may have seen headlines about the MIT study "Your Brain on ChatGPT". It measured brain activity while people wrote essays, and the group using a chatbot showed the weakest brain connectivity. The paper is a preprint (an early draft that other scientists have not yet formally reviewed), and it had 54 adult participants. Treat it as a reason to pay attention rather than a verdict. A 2025 survey of 666 people found heavier AI use went with lower critical thinking scores, but that only shows the two moved together. It does not prove one caused the other, and it relies on questionnaires.

The other side deserves a fair hearing. People have handed work to tools for a long time, and it's often smart. Researchers call that cognitive offloading: letting a tool hold some of the thinking so your brain can do something else. Calculators were supposed to ruin arithmetic, yet a review that combined 54 studies found they didn't hold back students' math skills. Kids will offload plenty. What matters is which skills they still need to own.

The second question

A chatbot is very good at the first answer. Learning starts at the second question.

The second question sounds like "How do you know?" or "What would change your mind?" Sometimes it's "Can I break this into smaller pieces?" or "Where could I check that?" Questions like these are the habits of research, which means reframing, sticking with something, testing a guess, and keeping track of what you still don't know. A machine that answers quickly doesn't make any of them go away. It does make them rarer, so kids have to practice them on purpose.

You could own every book ever written about Ancient Greece and still have nothing to say about it. Books hold answers, and only you can bring the question.

Seven things to try at home

Each of these takes an evening or less, and most are free to print.

  1. Search it, then ask it. Run the same ten questions through a search with a source and through a chatbot with no source, then chart which answers you can actually check. The Search It or Ask It sheet is free to print.
  2. Keep three columns. Write what you found, what you think and what you still don't know, and never let them blur together. Use The Research Notebook.
  3. Show the bot's work too. Log what you asked, what came back and what you changed, so anyone can see who did the thinking. Try Show Your Working, Including the Bot's.
  4. Guess the next word. Play Next Word and see how a language model picks what comes next. Kids who know how the machine guesses are much harder to fool with a guess.
  5. Make the bot cite its card. In Open-Book Bot, kids build an index card library and catch the bot grabbing the wrong card, an old card or no card at all.
  6. Test a hunch. Cause or Coincidence? asks who's really in charge when two lines move together, which is hypothesis testing in card game form.
  7. Agree on a homework rule. Use AI to explain, quiz and review, but never to write the answer. Our family homework rule has a card you can sign together.

Understanding the machine is the best antidote

We think the best protection against a fluent wrong answer is knowing where answers come from. Once a kid sees that a chatbot is predicting likely words, they tend to stop treating it like an oracle. Some of them also want to know how the prediction works.

Those kids will find a lot waiting for them. Math for AI covers the eight ideas AI runs on, from vectors to probability. The Tokenizer Engine lets them cut a story into tokens the way a language model does. The free AI Foundations course walks through how AI works, how to check it and how to build with it. We can't promise every kid gets hooked on the math, but we can make sure the door is open.

To go deeper on the asking side, read The Best Question Beats the Right Answer. Grown ups who want to learn the tools first can start with our short sequence for parents and teachers.

Fast answers are a gift. Knowing what to do when they run out is an education.

Print Search It or Ask It tonight and see which answers your kid can actually check.

› Keep reading

Get the next dispatch.

Occasional, never spammy.

By submitting, you agree to our Terms and Privacy Policy.