Clean Your Data Without Breaking It
By the end, you'll be able to clean a work table in pandas or in SQL without losing rows that matter, and explain every change you made.
You go at your own pace, with your Maestro beside you, 1:1, through the whole course: ask it anything, and it asks you questions too.
13 skills · 5 modules
What will you be able to do?
Each box is a skill. The base is at the bottom, recognizing and explaining, and you work your way up to building something of your own and judging what works and why. Hover over a box to see what it builds on. Click it to see where I teach it and how your Maestro checks it.
- Synthesize
- Analyze
- Apply
- Understand
Synthesize
Complete and adapt a cleaning script you can rerun on a new file that logs every step with its row counts
Where I teach it: A script you can run again · How do I turn my cleaning into a script I can run again?
Builds on:
- Decide whether a blank means zero, unknown or not applicable, and who confirms it
- Normalize text categories in pandas with strip, lower and an explicit map
- Flag the values that break a confirmed rule instead of deleting them
- Count rows before and after every step to catch the ones that vanish without warning
- Decide which copy of a repeated record stays, and keep it by sorting before you drop
Your Maestro asks you to build something of your own with it and saves it as evidence when you do.
Analyze
Decide whether a blank means zero, unknown or not applicable, and who confirms it
Where I teach it: Look before you change anything · What does an empty cell mean? Zero, "unknown" or "not applicable"
Builds on:
- Make an inventory of a table's problems before changing it
Your Maestro asks you to break a problem down with it and saves it as evidence when you do.
Choose the columns that make two rows the same record and predict how many rows remain
Where I teach it: Duplicates: real and fake · Why doesn't drop_duplicates() remove my repeats? Define the key first
Builds on:
- Count rows before and after every step to catch the ones that vanish without warning
Your Maestro asks you to break a problem down with it and saves it as evidence when you do.
Decide which copy of a repeated record stays, and keep it by sorting before you drop
Where I teach it: Duplicates: real and fake · How do I keep the latest entry for each folio in pandas?
Builds on:
- Choose the columns that make two rows the same record and predict how many rows remain
Your Maestro asks you to break a problem down with it and saves it as evidence when you do.
Compare DISTINCT with ROW_NUMBER for removing repeats in SQL, and write the query the key calls for
Where I teach it: Now do it in SQL · Why doesn't DISTINCT remove my duplicates? ROW_NUMBER and QUALIFY in DuckDB
Builds on:
- Choose the columns that make two rows the same record and predict how many rows remain
- Clean text, numbers and blanks in SQL with TRIM, LOWER, CASE, REPLACE, TRY_CAST and COALESCE
Your Maestro asks you to break a problem down with it and saves it as evidence when you do.
Apply
Make an inventory of a table's problems before changing it
Where I teach it: Look before you change anything · What should I check in a table before cleaning it? Build a problem inventory
Builds on:
- Describe a cleaning method: look, decide with the person who knows the data, fix in code, count rows and write it down
- Explain why one value written in several ways counts as different categories
Your Maestro asks you to use it on a new case and saves it as evidence when you do.
Normalize text categories in pandas with strip, lower and an explicit map
Where I teach it: Fix it in code, not by hand · How do I clean text written many different ways in pandas? strip, lower and a map
Builds on:
- Explain why one value written in several ways counts as different categories
- Make an inventory of a table's problems before changing it
Your Maestro asks you to use it on a new case and saves it as evidence when you do.
Convert numbers stored as text and count the values that could not be converted
Where I teach it: Fix it in code, not by hand · Why won't pandas add up my amounts? Numbers stored as text
Builds on:
- Make an inventory of a table's problems before changing it
Your Maestro asks you to use it on a new case and saves it as evidence when you do.
Flag the values that break a confirmed rule instead of deleting them
Where I teach it: Fix it in code, not by hand · Should I delete or flag an impossible value? How to find it in pandas
Builds on:
- Convert numbers stored as text and count the values that could not be converted
Your Maestro asks you to use it on a new case and saves it as evidence when you do.
Count rows before and after every step to catch the ones that vanish without warning
Where I teach it: Fix it in code, not by hand · How do I know if my cleaning lost rows? Count before and after every step
Builds on:
- Describe a cleaning method: look, decide with the person who knows the data, fix in code, count rows and write it down
Your Maestro asks you to use it on a new case and saves it as evidence when you do.
Clean text, numbers and blanks in SQL with TRIM, LOWER, CASE, REPLACE, TRY_CAST and COALESCE
Where I teach it: Now do it in SQL · How do I clean text, numbers and blanks in SQL? TRIM, CASE, TRY_CAST and COALESCE
Builds on:
- Normalize text categories in pandas with strip, lower and an explicit map
- Convert numbers stored as text and count the values that could not be converted
Your Maestro asks you to use it on a new case and saves it as evidence when you do.
Understand
Explain why one value written in several ways counts as different categories
Where I teach it: Look before you change anything · Why does pandas count "CDMX" and "cdmx " as different cities?
Your Maestro asks you to explain it in your own words and saves it as evidence when you do.
Describe a cleaning method: look, decide with the person who knows the data, fix in code, count rows and write it down
Where I teach it: Look before you change anything · Why does pandas count "CDMX" and "cdmx " as different cities?
Your Maestro asks you to explain it in your own words and saves it as evidence when you do.
Who is this course for?
For anyone who receives messy tables at work, with a city written three different ways, amounts stored as text, blanks, and duplicates, and wants to clean them without losing rows that matter. You should know how to open and filter tables in pandas and write SELECT with WHERE and GROUP BY in SQL. If your file shows broken accents, that's covered in another course.
Ask your Maestro
Your Maestro is an AI that knows this course. Ask it about the first page and it answers right here, free.
You: Where do I start?
Your Maestro: From what you already know. Tell me what you've done with data, and I'll start this page from there.
Asking about: Answers from the page "Why does pandas count "CDMX" and "cdmx " as different cities?"
Only this browser keeps what you ask here. How it works
What we'll cover
Look before you change anything
Fix it in code, not by hand
Duplicates: real and fake
Now do it in SQL
A script you can run again
How long does it take?
1 to 1.8 hours
A guide: you go at your own pace, and the course waits where you left off.
Common questions
- How do I clean a table without losing rows that matter?
- First look before you change anything: take inventory and decide what each gap means. Then fix with code, not by hand, and count rows after each step to notice if you lost any. At the end you complete a script you can rerun and explain. You do it in both pandas and SQL.
- How do I find duplicates without deleting good data?
- Not everything that repeats is a duplicate: two identical rows can be two real sales. First decide what identifies a row, then flag the repeats and choose which one to keep, for example the most recent. The course does it with DISTINCT and ROW_NUMBER in SQL, and with pandas.
- What do I need to know before starting?
- For the exercises you need to open, select and filter tables in pandas, and for the SQL module, to write a SELECT with WHERE and GROUP BY. Your Maestro helps if something is unclear. If you have never used either tool, the course still helps you understand the method, which also works in Excel.
- Do I need a course, or can I learn this on my own?
- You can learn it on your own, and many people do. A course gives you the order: what to practice first, ready-to-use data, and exercises that show where you go wrong before it happens at work. This course is free, and your Maestro answers questions as you go. If you prefer going solo, the method still works.
- Is it free? Do I need an account?
- It's free. You can read every page without an account. With a free account you enroll with one tap, run the code on each page and ask your Maestro questions.
- Do I have to install anything?
- No. The Python and SQL code runs inside Maestro: you type on the page and see the result below it. You can download the project from the last module and run it on your own computer whenever you like.
- Is the data real?
- No. The coffee cooperative's sales and members are synthetic data, made up for the course, with the problems that show up in real work tables.
- My file shows broken accents or everything in one column. Will this help?
- That's a problem with how the file is read, not with cleaning, and this course doesn't cover it: it starts once the table has opened correctly. Dates aren't covered here either.
How do I check what AI does? I teach it in Your AI and Your Data: Don't Let It Make Up the Number and Check what AI gives you in three steps.
What you get
- 5 modules and 13 pages, open whenever you want them.
Open to everyone: read all of it, and your Maestro, an AI, answers your questions on every page.
With a free account: your Maestro for the whole course, always on the page you're on, and the exercises run with nothing to install.
Words you'll meet
In the lessons, tap one to see what it means, or ask your Maestro about it.
Who's teaching you
Sergio Sánchez
I'm Sergio, chekos online. I'm a data architect, and I've spent more than ten years working with data in the Bay Area, mostly at mission-driven organizations: policy research first, then education, and now early childhood services. I grew up in Tijuana. In 2018 I started tacosdedatos to share what I was learning, and since then it's been a blog, a podcast, courses, guides, and a community. These days I spend a lot of my time building with AI agents and writing about how I organize my work around them. I share what I've learned; you take what's useful and make it yours.
See you in the course
Clean Your Data Without Breaking It