Skip to content
Start free

How do I know if the numbers my AI gave me are right? Check them before you share

Check the numbers an AI hands you with five quick checks, and recalculate the ones that don't add up.

Free course. Your Maestro, an AI, .

Lucía sent her brief. Before the dashboard, the AI sent back the table of totals she asked for, plus this summary of the week.

Illustrative scenario: written for this course, not a real response from any AI.

Week of September 21According to the AI
Appointments completed4,464
No-show rate13.7%
Occupancy87.0%
Average wait19.4 min

And one line at the end: "I calculated everything with Python on your file."

All four look reasonable. None is absurd, and that's exactly the problem: a wrong number almost never looks wrong. Lucía has a definition sheet for each number, and she has the data. Before you keep reading, think about which one you'd check first.

Check yourself: Lucía asks the AI, "Are you sure about these numbers?" The AI answers, "Yes, I double-checked the calculations and they're correct." What does Lucía know now?

Checking means calculating, not asking

A check compares the number against something you can see for yourself: the data, the definition sheet, a total you know. For a dashboard, five checks are enough, in this order, because the first ones catch the biggest errors with the least work. The second is a : making sure the parts add up to the total. The fifth is a : redoing the math by hand for one case. If you can't get to the source data at work, ask your Maestro how to check anyway.

  1. Is it the period you asked for?

    A total that's way too big or too small usually means the wrong period. Compare it with a normal week: over the quarter, the clinic saw an average of 343 people a week.

  2. Do the totals add up?

    The sites add up to the clinic, and booked = canceled + no-shows + completed. If it doesn't add up, something got dropped or counted twice.

  3. Did it use the right denominator?

    No-shows divided by appointments still on the books, not by everything booked. If the definition sheet says one thing and the code does another, the definition sheet wins.

  4. Did it average averages?

    A clinic-wide rate or average comes from adding up the counts and dividing at the end. If the code calls .mean() on a column of rates or averages, that's your error.

  5. Recalculate one row by hand

    Pick one site and one week, do the math on a calculator, and compare it with what the dashboard says.

The first two, done for you

Period. 4,464 completed can't be one week: that's more than thirteen times a normal week. It's the total for all thirteen weeks in the file. The AI didn't filter to the latest week, even though the brief asked for it. That week, the clinic saw 331 people.

Totals. For the week of September 21, 402 booked = 28 canceled + 43 no-shows + 331 completed. It adds up. And the sites add up to the clinic: 183 + 131 + 88 = 402 booked.

That leaves three numbers. After module 2, you may already suspect where the 13.7% came from.

Try it

In the editor, the code already filters to the latest week, runs the reconciliation and calculates completed appointments and occupancy. Two lines are missing: the no-show rate and the average wait, as Lucía's definition sheet defines them. When you run it, each number appears next to the AI's, marked "matches" or "DOES NOT MATCH". If you're not sure how to write one of the two lines, ask your Maestro for a hint.

Fill in the two marked lines: noshowrate (no-shows divided by appointments still on the books, adding up before you divide) and avgwait (everyone's wait minutes divided by patients seen). You're done when only occupancy says "matches" and your numbers are 331, 11.5%, 87.0% and 19

Fill in the two marked lines: no_show_rate (no-shows divided by appointments still on the books, adding up before you divide) and avg_wait (everyone's wait minutes divided by patients seen). You're done when only occupancy says "matches" and your numbers are 331, 11.5%, 87.0% and 19.7.

Run your code to see what it prints and the value of its last expression. Every run starts fresh.

Ready to run your code.

Run your code to see its output here.

Data: citas-clinica, 117 rows

Enter saves a cell and Escape leaves it as it was. Leave a cell empty for a missing value.

The exercise's data, "citas-clinica". Your edits run with your code.
RowRemove
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Page 1 of 6

Run it yourself

If you finished it, only occupancy matches. The completed appointments are for the whole quarter. The 13.7% no-show rate is the average of the week's nine rates; the clinic's actual rate is 11.5%. And the 19.4-minute wait is the average of the nine rows' waits; from the totals, it's 19.7.

Three out of four numbers wrong, and none of them looked off. It wasn't an arithmetic mistake either: the AI added and divided correctly. What it got wrong was the period and what to average, which are exactly the decisions the definition sheet writes down. That's why the definition sheet goes in the brief, and why you still check when it was already there.

What you tell the AI

Don't say "That's wrong, fix it." Say what's wrong and what the rule is, quoting the definition sheet: "Appointments completed adds up all thirteen weeks; use only the week of September 21. The no-show rate is total no-shows divided by total appointments on the books, not the average of each row's rate. Same for the wait: total minutes divided by patients seen." Then check all four numbers again: a fix can repair one number and break another. Tell your Maestro what you'd say to your AI, and it'll tell you if anything's missing.

Check yourself: You do a spot check. Nutrition at Cholula, week of September 21: 13 appointments booked, 1 canceled with notice, 3 no-shows. The dashboard says the no-show rate is 23.1%. What happened?