Skip to content
Start free

How do I check a column's categories with value_counts()?

Count the values in a text column, find the labels you didn't expect and chart the stations in order.

Free course. Your Maestro, an AI, .

Not every column is a number. tipo_usuario, estacion_origen and estacion_destino hold labels: each value names a group. An average or a histogram makes no sense for them. The question is a different one: which values are there, and how many rows does each one have?

That question finds problems no other check finds: a label spelled two ways, a group that shouldn't exist, a category with three rows nobody expected. And it tells you something about the business: which stations get the most trips, how big a share the occasional riders are.

Start with a bet. The description of the tipo_usuario column in the dataset says: suscriptor (with a membership) or ocasional (one-day pass).

Check yourself: According to its description, tipo_usuario is either suscriptor or ocasional. How many distinct values do you think the column has?

You count with value_counts(): it lists every distinct value with the number of rows that have it, from most common to least. It's probably the command you'll use most in a first look.

Your turn

Run the tipo_usuario count. Then change it to estacion_origen and see how many stations there are and which ones get the most trips.

Data: viajes-rueda, 311 rows
The exercise's data, "viajes-rueda". Your edits run with your code.
RowRemove
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Page 1 of 16

Three labels come out: suscriptor 228, ocasional 71 and Suscriptor, with a capital S, 12. To a computer, suscriptor and Suscriptor are as different as suscriptor and ocasional. If you group by user type without noticing, your report will have three groups, and the 12 Suscriptor trips will be left out of every subscriber figure.

Before you write down "there's a misspelled label," ask where it comes from. Twelve rows that share an odd trait usually share a cause.

Your turn

Where do the 12 trips with Suscriptor come from? Run the code. Then also print the first inicio of those trips with .min().

Data: viajes-rueda, 311 rows
The exercise's data, "viajes-rueda". Your edits run with your code.
RowRemove
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Page 1 of 16

All 12 left from the same station, Terminal de Autobuses, and the first one started on March 9 at 7:07 a.m. The last lowercase suscriptor trip from that station was on March 8 at 2:37 p.m. Something changed at that station between the 8th and the 9th: maybe a new kiosk that writes the label with a capital letter. It's a hypothesis to confirm with whoever runs Rueda.

In your log: tipo_usuario has 3 values; Suscriptor (12 rows) comes only from Terminal de Autobuses, starting March 9 at 7:07 a.m.; they're probably subscribers; group with care. Merging the labels is already cleaning, which another free Maestro course covers. Here it's enough to find it and write it down.

For stations, the count tells a different story: which ones get the most trips. A list of 12 numbers reads better as a bar chart, if you sort it.

Your turn

The bars come out in alphabetical order and the names are squeezed together at the bottom. Move the station to the y axis and the count to the x axis, and sort the bars with sort="-x" inside the alt.Y. You're done when the station with the most trips is on top and every name is readable.

Data: viajes-rueda, 311 rows
The exercise's data, "viajes-rueda". Your edits run with your code.
RowRemove
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Page 1 of 16

What the chart shows

Vertical bars with the trips from each of the 12 origin stations, in alphabetical order: Centro Histórico has the tallest bar, with 58 trips, and Estadio and Las Flores the shortest, with 10.

Sorted, the chart reads at a glance: Centro Histórico, with 58 trips, has nearly twice as many as the next one, Hospital General, with 33. Universidad and Plaza de Armas follow with 32 each. At the bottom are Las Flores and Estadio, with 10. A category with few rows isn't a problem in itself, but any figure you compute for it, like the median duration, rests on very few trips; note that if you're going to compare them.

Sorting by count is almost always what you want: alphabetical order is only good for finding a name. If you're wondering when a different order makes sense, ask your Maestro.

Try it on another table

Check yourself: A customer table has the column estado (state). Counting it, you see: Jalisco 410, JALISCO 37, Jal. 12 and jalisco 3. What do you write in your log?