When can you trust a number from an AI?
Decide how much to check based on what's at stake, build your routine, and go back to the four questions from the start.
Free course. Your Maestro, an AI, .
After four modules of figures that didn't answer the question asked, it's easy to draw the wrong conclusion: "you can't believe anything an AI says." Look at what you found. In the whole course, not one sum, count, or division was wrong. What failed were decisions: what a row is, what to divide by, what to leave out. And the only wrong figure in the course was in a sentence of a summary, not in the calculation.
So the useful question isn't "do I believe it or not?" but "how much do I need to check this number before I use it?" That depends mostly on one thing: what happens if it's wrong.
If you ask "roughly how many people answered the survey?" to decide whether the answers are worth reading, a rough number is enough, and being wrong costs you nothing. If the number is going into a meeting, a report for management, or a purchasing decision, being wrong does cost something, and that's when you ask the three questions and recompute at least the figure that matters most.
The AI gives you a number
What happens if it's wrong?
If you don't know, ask what it will be used for
Path 1 of 2: Little
Use it as a starting point
An idea, a draft, an exploration
Continues at: Use it with its base and its source
Path 2 of 2: A lot
Ask the three questions and recompute
A meeting, a report, a decision
Continues at: Use it with its base and its source
Use it with its base and its source
Signs that should raise or lower your trust
Besides what's at stake, the answer itself gives you clues.
You can trust an answer more when it states its base ("40 of 45"), comes with a receipt whose figures add up, lets you see the code, and is a reasonable size for what you know about the business. It also counts if you recomputed a key figure and got the same result.
You can trust it less when it gives a percentage with no denominator, lots of decimals that sound precise but don't say what they're a share of, a table that doesn't add up to the file's total, or a summary that doesn't say the same thing as the code.
No single sign proves anything. Decimals don't make a number exact, and matching your recomputation only proves you understood what it calculated. Together, they tell you how much work is left before you use it. I have a number from work and I don't know how much to check it.
Your routine
This is the course's complete routine, in the order you use it:
Decide how much to check
What happens if the number is wrong? If nothing does, use it as a starting point.
What is a row?
If the question is about sales and the table lists products, counting rows gives you something else.
What did it divide by?
Every average and every percentage has a number on the bottom. Find it and say it.
What did it leave out or count twice?
Empty cells, filters nobody mentioned, repeated rows. Reconcile the total against the file.
Recompute the figure that matters most
With your pandas calculator, or ask for the receipt and check it.
Report it with its base
Out of how many, what was counted, and what was left out, in one sentence.
The four questions from the start
On the first page you answered four quick questions with what you thought back then. Here they are again, unchanged. Answer them without going back to see what you picked, and then compare.
Check yourself: You ask an AI for the month's total sales. It shows you the Python code it ran, with no errors, and a total. What does that tell you about the total?
Check yourself: Your sales file has one row for each product within a ticket. The AI tells you "There were 312 sales in March," and the file has 312 rows. What did it count?
Check yourself: A survey went out to 112 customers and 45 answered. The AI reports: "89% of customers are satisfied." What do you need to know before you repeat that number?
Check yourself: In a column of ratings from 1 to 5, 67 of the 112 cells are empty. The AI computes the average and gets 4.4. What did it do with the empty cells?
If any answer changed since the first page, notice which one. It's almost always the one the course worked on most with you. If one still gives you trouble, ask your Maestro before you go. I want to go over the question I still find hard.
Think of a number an AI, or a person, gave you at work in the last few months. Which of the three questions would you have asked first, and what do you think you would have found?
Reflect: Think of a number an AI, or a person, gave you at work in the last few months. Which of the three questions would you have asked first, and what do you think you would have found?
Only you and your Maestro read this. It's kept on your account and deleted with it.
How far this course goes
Here you worked with one question and one answer at a time: you ask, the assistant calculates, you check. That's how the whole course works, and the three questions work for any number that comes out of ChatGPT, Copilot in Excel, Claude, or any other tool.
There's another way to work: handing the AI a whole analysis, with many steps, files, and decisions, and letting it do the work on its own while you direct. That's working with agents, and this course stops right before it. If that's where you want to go next, the course Agentic AI for Data: Your Judgment, at Scale starts from the same idea you learned here, that the judgment is yours, and takes it into that other kind of work.