You already do data engineering
Put names to the three steps you already do, and see why what you know about your numbers is what lets you check AI.
Free course. Your Maestro, an AI, .
The three steps from the last module have names in software. You may have heard them before and figured they were for programmers. They're what you just did with the bakery.
Spot check
Checking a few rows by hand. You recompute a handful, spread across the sheet, and compare with what the formula gave.
Unit test
Giving it a made-up code knowing what should come out. One input, one expected answer, and you compare.
Edge case
Since the code doesn't exist, the test goes after an edge case: the value that isn't there, where formulas tend to act strange.
Reconciliation
Matching against the cash count. You compare with an independent number and explain the difference down to the last peso.
At PPIC I didn't call them that. I gathered the data, cleaned it, analyzed it, and made the charts. Years later I was a data visualization analyst, then a data engineer, and today I'm a data architect at a government agency in the San Francisco Bay Area. Looking back, at PPIC I was already doing all of that. I just didn't have those names.
If you've ever reconciled a spreadsheet against a bank statement, you've done data engineering, even if nobody called it that.
What the AI doesn't bring
Go back to the bakery and notice what each step needed:
- To check rows by hand, you needed to know where to look up each product's price.
- To try to make it fail, you needed to know what should come out for a code that doesn't exist.
- To reconcile, you needed the cash count, and to know that pan de muerto is new and costs 28.
None of that was in the formula. The AI wrote a reasonable formula without knowing there was a new product. What it took to find the error, you knew.
That's why these steps don't stop being useful as AI gets better. Checking AI's work takes the same thing doing the work would: knowing what your numbers should look like. AI is one more tool, one that helps you use the others, and with it what you know goes further. The number you sign is still yours.
Try it
Think of a check you do at work without thinking about it: something you always look at before you send a report or a spreadsheet.
- Describe it in one or two lines.
- Which of the four names fits it: spot check, unit test, edge case, or reconciliation?
- What do you know, that isn't in the sheet, that lets you do it?
A good answer names the check, gives it one of the names, and says what knowledge of yours it needs.
Tell your Maestro about a check you do without thinking.