AI Knowledge vs. Experience: How to Tell the Difference

Recently, when we were interviewing candidates for our open positions, I noticed something interesting. Some had knowledge right at their fingertips. Ask them anything and their answer had texture: a story, a caveat, a “well, it depends, and here’s what it depends on.” You could feel the experience behind their words. Others were just as fluent, sometimes more polished, but the moment we pushed past the surface, we found nothing underneath.

The difference wasn’t intelligence or effort; it was where their knowledge came from. One group earned fingertip knowledge through experience. The other borrowed knowledge — and, typically, they borrowed it from AI.

AI knowledge is the output of expertise without the expertise, whereas fingertip knowledge is the real thing: It comes with all the parts the manual leaves out, because you learned those parts the hard way.

Here’s why this matters well beyond hiring: The only way we used to be able to learn something was by investment. We had to seek out a book at the library, have someone teach us the knowledge, or learn through trial and error. There was a cost: You didn’t become fluent in a field without paying with time and effort. AI broke that link. Fluency is now cheap, and as AI works its way further into our day-to-day, the ability to discern earned knowledge from borrowed knowledge is essential.

Whether you’re hiring someone, buying from someone, or deciding whose advice to act on, you need a way to test what’s actually underneath the words.

The good news is that the two types of knowledge reveal themselves quickly once you know how to push. Here are 4 ways to push that have worked for me . . .

1. Move the goalposts. Take what they just told you and change a variable. Apply it to a situation that their tidy explanation didn’t anticipate. Someone with fingertip knowledge runs the new scenario through the model they built and adapts on the spot. Someone with AI knowledge freezes, because there was never a model — just the output, and the output doesn’t bend.

2. Question the source. Ask how they came to know what they’re telling you. This tests whether they generated the knowledge or absorbed it. People who earned it can tell you the story of learning it: the project, the mistake, the person who set them straight. People who absorbed it this morning can give you only the conclusion.

3. Ask about a time it went sideways. Have them walk you through a moment when things didn’t go as planned and how they adapted. This makes them demonstrate real-world experience, and it’s almost impossible to fake, because adaptation lives in specifics. The texture of a real recovery can’t be invented on the spot. Someone who has been there gives you detail. Someone who hasn’t gives you generalities.

4. Ask about the most common mistake. “What’s the most common mistake you see people make in this field, and why do you think they make it?” This tests two things at once: enough experience to have seen the pattern, and enough analytical distance to explain its cause. Anyone can list errors. Explaining why people commit them requires having watched it happen more than once, and to have thought about what they saw.

None of these are gotchas. They’re simply questions that require a real model to answer well, and they quietly reveal the absence of one when it isn’t there.

To be clear, I’m not against the tool. AI is remarkable for getting oriented in unfamiliar territory fast, which is real and useful. The mistake is confusing the foothold for the summit, and treating two minutes of access as two decades of experience.

Friction was never a bug in learning — it’s a feature. The struggle to work out an answer yourself is what builds the model and the model is what lets you adapt when reality asks a question your source never covered.

So, test for it. AI doesn’t fix ignorance. It can mask it, sometimes well enough to fool a room — and even the person who’s using it. Now, more than ever before, a leader’s job is to find out what knowledge is actually at candidates’ fingertips.


Negotiation Training and Consulting to Help You Tell the Difference Between AI Knowledge and Experience.

As AI works its way further into our day-to-day, the ability to discern earned knowledge from AI-borrowed knowledge is essential. Rely on Scotwork’s expertise to help you push for the truth.

Get in touch with one of our experts today.