Skip to content
Start free

Why is data missing from a table, and what changes in my analysis?

Look for the cause of each gap in the other columns, with a worked example and a chart, and decide what changes in your analysis.

Free course. Your Maestro, an AI, .

Counting the gaps tells you how many there are. Knowing why they're missing tells you what you can conclude from what's left. Almost every gap in a real-world table falls into one of three families:

FamilyWhat happenedExample
Doesn't applyThe data doesn't exist for that casea one-day pass with no membership expiration date
Doesn't exist yetThe event hadn't finished when the table was pulleda patient with no discharge date because they're still admitted
LostThe data existed and wasn't recordeda sensor with no signal, a field someone didn't fill in

All three leave a trace in the other columns: the gaps cluster in a group, on certain dates or in one place. That's the trick on this page. Don't stare at the empty column; look at the other columns of the rows where it's missing.

  1. A column with gaps

  2. Keep only the rows with the gap

    data[data["columna"].isna()]

  3. Do they cluster in a group, a date or a place?

    • Path 1 of 2: yes

      1. Look for the cause in that group

        compare with the group's total

      Continues at: Write down the probable cause and whom to ask

    • Path 2 of 2: no

      1. Note them as scattered

      Continues at: Write down the probable cause and whom to ask

  4. Write down the probable cause and whom to ask

First, see how it's done with the birth years. Open one step at a time and try to guess the next one before you open it.

Why are 73 birth years missing?

Worked example · 6 steps

73 of the 311 records are missing anio_nacimiento. Before summarizing recorded ages, you want to investigate why.

Work out the next step yourself, then open it to compare.

Now it's your turn with the 8 trips that have no destination. On the previous page you saw that they're the same 8 in three columns. The question is when they happened. A chart of trips per day tells you at a glance.

Your turn

The chart counts all the trips on each day. Add .transform_filter("!isValid(datum.duracion_min)") at the end to keep only the trips with no duration. You're done when the chart shows only those 8 trips and you can see which day they cluster on.

Data: viajes-rueda, 311 rows
The exercise's data, "viajes-rueda". Your edits run with your code.
RowRemove
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Page 1 of 16

What the chart shows

Bars with the number of trips on each day from March 2 to 15, 2026, between 14 and 32 per day.

Of the 8, six started on Sunday, March 15, between 10:08 p.m. and 11:51 p.m.: the last trips in the table. That's the suspicion you wrote down in module 1. These records have no closure and may have been in progress at export. The "doesn't exist yet" family is a hypothesis: we do not know the cutoff time or whether the bikes were still in use.

What changes: the mean duration for the 15th excludes these six late-night records, and the closure count may be incomplete. Explain which records are included and ask for the export time before describing the day's total.

Two are left over: 40291 (Friday, March 6, 3:03 p.m.) and 40413 (Wednesday, March 11, 7:02 p.m.), both by occasional users, in the middle of the period. Their age needs separate review; a cutoff time alone does not explain why they never closed. Perhaps docking was not recorded, or there was an operational problem. You don't know, and that's fine: it goes in the log as a question for the operations team.

Observation: 6 of the 8 trips with no destination started on March 15 after 10:00 p.m.; 2 are from earlier days. Evidence: the chart of trips with no duration per day. Question: was the table exported close to midnight on the 15th? What happened to the bikes on trips 40291 and 40413?

Notice what you just did: you didn't touch the empty column. All the evidence came from inicio and tipo_usuario. What to do next with those rows, whether to exclude them, flag them or estimate them, is a cleaning decision; another free Maestro course, on cleaning data, covers it.

If your table has gaps and you don't know which column to compare them with, ask your Maestro.

Try it

Check yourself, 1 of 2: You want to report the average trip duration for each day. What do you do with the 6 trips on March 15 that had no duration?