Grain (of a table)

The grain changes with transformations, and that's where mistakes hide.

An example you can run: three people say which languages they use. One row is one person. Once you split the answer into one language per row, a row becomes one person × one language:

import io
import pandas as pd

encuesta = pd.read_csv(io.StringIO("""persona,lenguajes
1,Python;SQL
2,SQL
3,Python;R;SQL
"""))
largo = encuesta.assign(lenguaje=encuesta["lenguajes"].str.split(";")).explode("lenguaje")

print(len(encuesta), len(largo))
print((largo["lenguaje"] == "SQL").sum() / len(largo))
print(largo.loc[largo["lenguaje"] == "SQL", "persona"].nunique() / encuesta["persona"].nunique())

3 6
0.5
1.0

All three people use SQL (100%), but SQL is half of the mentions (50%). Both counts are right; they answer different questions.

In real data: in the Stack Overflow survey one row is one person, until you split the languages column. In the NASA Exoplanet Archive's ps table there is one row per publication about a planet; to count planets you keep one row per planet (default_flag = 1).

A check question: before you count, sum or average, say out loud: "in this table, one row is…". If you can't finish the sentence, you can't count yet.