When an agent did it
Look at the tables two AI agents made from the same question, and plan how you'd apply the three steps before accepting one.
Free course. Your Maestro, an AI, .
So far the AI gave you a formula and you pasted it in. An AI agent goes a step further: it opens your files, writes the code, runs it, and hands you the result. It does more on its own, so it makes more decisions you didn't ask for.
The three steps are the same. What changes is where you apply them.
Two agents, one question
This comes from my course on agentic AI for data. Two agents got the same instruction, each in a fresh session, on October 8, 2026. Agent A is Sonnet 5.5 and Agent B is Haiku 5.5, a smaller model from the same family. These are test sessions with Claude models; with another agent, or in another session, the steps may differ. The data is Stack Overflow's public 2024 survey, where each row represents one person’s survey response. That is the table’s unit; its provenance is Stack Overflow’s public file. The sessions ran in Spanish; the instruction and the agents' explanations are translated here.
The instruction:
Give me the top 10 programming languages with the percentage of developers who use each one. Column
LanguageHaveWorkedWith.
Here are the first five rows of each table.
Agent A
| Language | % of those who answered | % of all rows |
|---|---|---|
| JavaScript | 62.8% | 57.3% |
| HTML/CSS | 53.3% | 48.6% |
| Python | 51.4% | 46.9% |
| SQL | 51.4% | 46.9% |
| TypeScript | 38.7% | 35.4% |
And it explained: "Base: 59,745 respondents answered the column. The file has 65,437 rows, and the remaining 5,692 have the column empty or NA."
Agent B
| Language | % |
|---|---|
| JavaScript | 57.29% |
| HTML/CSS | 48.62% |
| Python | 46.94% |
| SQL | 46.89% |
| TypeScript | 35.38% |
And it explained: "The percentage is over all respondents. Since every row has an answer in this column, it's the same as the percentage over those who answered."
Both tables look right. Both ran without errors. And they don't say the same thing: for JavaScript, one says 62.8% and the other 57.3%. Stack Overflow publishes 62.3% for JavaScript across 60,171 responses. The published processing criteria have not been fully reconciled here with the public file; a nearby result does not resolve that difference.
Before you decide, plan
I'm not asking which one is right. I'm asking how you'd know. It's the same question as the bakery, and you have the same three steps:
- Check a few rows by hand. At the bakery you looked up a code in Precios. Here, which rows of the file would you look at, and what would you look for in them?
- Try to make it fail. At the bakery you gave it a code that didn't exist. Here, what's the hard case in a survey? Think of someone who didn't answer that question. What should happen with that row?
- Reconcile with a number you already know. At the bakery it was the cash count. Here, what numbers would you compare against, and what difference would you have to explain?
Look at Agent B's sentence: every row has an answer. At the bakery, the formula never said "not found." Does it look familiar?
Try it
Write your plan before you ask your Maestro anything:
- For each of the three steps, what you'd do with these tables and what you'd expect to see if the table is right.
- Which step do you think would find the difference between the two tables first, and why?
- What population and denominator would each number describe, and what limitation would the report need to state?
A good answer has a concrete step for each rule, with what you expect to see, and says what happens to the people who didn't answer.
Share your plan with your Maestro.
A precise phrase is "the share of participants who reported using JavaScript," followed by the chosen base. Do not assume use or non-use among those who skipped the question, or extend the result to all developers.