Schools Are Adopting AI With No Proof It Works. Your Child Needs This One Skill.

Last month, a school district in Ohio renewed its contract for a $78,000 AI chatbot designed to help struggling students work through their emotions. When a reporter asked how they knew it was helping, the external evaluator said he could only get as far as “perception.” No hard data. No control group. Just a hopeful feeling. Stanford researchers who reviewed more than 800 academic papers on AI in classrooms described the evidence base for that kind of spending as “extremely limited.”^[1][2] That finding changed the way I think about what my kids actually need to learn.

Schools across the country are adopting AI tools at speed, spending tens of thousands of dollars on programs with no proof they work and no shared way to measure whether they are helping.

Stanford University’s Center to Support Excellence in Teaching analyzed more than 800 academic papers on AI in K-12 classrooms and found the evidence base is “extremely limited.”^[1] There is no consensus on what metrics schools should be tracking. Many existing studies rely on self-reported perceptions rather than hard learning outcomes. That has not stopped adoption. Districts are buying chatbots, AI tutors, and humanoid robots based on vendor promises and gut instinct, with no shared framework for evaluating what works.^[2]

One of the clearest data points comes from a randomized controlled trial at the University of Maryland. Researchers gave 2,379 undergraduates access to a GPT-4-powered AI tutor connected to their actual course materials. They expected to see learning gains. Instead, students who had access to the AI tutor scored about four percentage points lower on final grades. Their engagement with course materials, measured by login activity and time on the platform, dropped nearly a full standard deviation.^[3]

This was not a poorly designed product. It was a purpose-built tool reviewed by faculty, connected to real course content, and tested rigorously. When researchers looked at why it backfired, they found most students who used it were asking for answers, not guidance. And even students who never used the tutor showed reduced engagement with their coursework, possibly because knowing the shortcut existed changed how they approached studying in the first place.^[4]

If schools cannot reliably evaluate whether AI tools are helping or hurting students, the responsibility for asking that question shifts to families.

Robin Lake, director of the Center on Reinventing Public Education, told NPR the situation “feels very Wild West in terms of the randomness.”^[2] That characterization surprised me. Most parents I talk to assume schools are making careful, evidence-based decisions about what technology they introduce. The reality is that most are making judgment calls with limited data, significant vendor pressure, and no clear policy guidance from state or federal governments.

Rebecca Winthrop, a senior fellow at the Brookings Institution, said AI experimentation should not happen on vulnerable students without more evidence.^[2] She raised the risk of cognitive stunting, the possibility that AI could reduce a child’s capacity to learn by doing their thinking for them before their foundational skills are formed. The University of Maryland study added a harder number to that concern: first-generation college students showed the largest grade losses when the AI tutor was available.^[3][4]

This matters for parents because most of us assume that if a school is using something, it has been vetted. That assumption is being tested in real time. Schools are not reckless. They are navigating something genuinely new with almost no guidance. But the gap between what AI tools promise and what they have proven is still wide, and your kid will encounter that gap in every classroom, every app, and every confident claim about what AI can do for them. They need a skill for it.

The skill is evidence thinking, the habit of asking “how do we know this actually works?” before trusting any tool, claim, or new approach.

This is not cynicism. Evidence thinking is not about assuming everything is a scam or deciding AI is bad. It is a genuine, curious question: what proof exists that this does what it says it does? Scientists ask it before publishing. Doctors ask it before prescribing. Investors ask it before committing. And children can learn to ask it too. In a world flooded with confident claims about new technology, the habit of asking for evidence is one of the most transferable skills a kid can build.

My middle daughter, who is nine, does something that quietly drives her older sister a little crazy. Before she tries anything new, she wants to know why. She will hold a new snack and say, “But does it actually taste good, or does the package just say that?” She is not being difficult. She is being appropriately calibrated. That natural instinct to ask “but how do we know?” is exactly what I want to help all three of my daughters develop more deliberately, because it applies far beyond snacks.

The useful thing about evidence thinking is that it works in every domain, not just AI. It applies to the study technique a teacher recommends, the app a friend says changed their life, the supplement that promises better focus. In a world where confident claims about technology arrive faster than research can evaluate them, the ability to ask for evidence, and to accept “we don’t really know yet” as a legitimate and honest answer, is one of the most practical habits a child can take into adulthood.

Your kid is developing evidence thinking when they ask what something is supposed to do before they try it, not after it has already become a habit.

Watch for the small moments. Does your kid ask a teacher why the class is using a new tool? Do they compare how they feel before and after trying something, even casually? When a friend says “this app is amazing,” do they ask what makes it amazing, or do they just download it? These might look like stubbornness or cautiousness. They are actually a child building the habit of checking before committing, which is the starting point for real evidence thinking.

A child who does not rush to adopt what everyone else is using is not a difficult child. They are a child who has begun to trust their own judgment over social pressure. In the context of AI, where new tools appear daily and most available “evidence” is marketing, this is exactly the habit that protects them. The kids who will navigate the next decade most effectively will not necessarily be the fastest adopters. They will be the ones who built the habit of asking whether something is actually working for them personally, not just in someone else’s account.

Have a “how do we know?” conversation with your child this week about one tool they already use, AI or otherwise.

Try the “Before We Try” test together. Pick any app, tool, or study technique your kid uses regularly and ask them two questions: What is this supposed to help you do? How would you know if it stopped working? You do not need a conclusive answer. The goal is to practice the question out loud. A ten-year-old who can articulate what a tool is supposed to do, and what evidence would change their mind about it, is already ahead of most adults in this department.

Send your school one question. If your kid’s school uses any AI tools in the classroom, send a short, friendly email asking how they measure whether it is effective. You are not challenging them. You are modeling the same behavior you want your kid to develop: thoughtful adults ask for evidence before they commit to something new. The school’s answer, whether a clear framework or an honest “we’re still figuring it out,” is worth knowing.

Name the habit this week. When your kid encounters a bold claim, a new app recommendation, or a teacher’s suggestion, ask them quietly: “How do we know this actually works?” Not as a gotcha. As a genuine, curious question you are both thinking through together. Practice it briefly and often. The more they run the question, the more naturally it surfaces when the stakes are higher.

This is not about teaching your kid to distrust technology. It is about teaching them to ask the same question every good scientist, doctor, and investor asks before committing to something new. That question does not require special knowledge. It is available to any child at any age. The earlier they learn it, the more uses they find for it.

The schools buying unproven AI tools are not bad actors. They are responding to the same pressure all of us feel: the world is changing fast, and waiting for perfect evidence feels like falling behind. But the gap between what AI promises and what it has proven is real. When one university ran a rigorous controlled experiment giving students access to a purpose-built AI tutor, grades went down, not up. That result is not an argument against AI. It is an argument for asking better questions before you trust it.

What I want my daughters to carry into that uncertainty is not fear and not enthusiasm on faith. It is the habit of asking one question: how do we know? A two-second gut check that changes how you evaluate every claim, every tool, every confident statement about what AI can and cannot do for you. Teach your kid that question and you have given them something no AI tool can generate on its behalf.