How do I look at the relationship between two columns without claiming a cause?
Put two columns on the same chart, read whether they move together, and separate what the table says from what it can't say.
Free course. Your Maestro, an AI, .
So far you've looked at each column on its own. Many of the questions you'll get involve two: are the longest trips the ones that cover the most distance? What time of day does each type of user ride? On this page you put two columns on the same chart and write what you see in your log.
The chart depends on what type the two columns are:
- Two numbers, like distance and duration: a scatter plot, one point per row.
- A number and a category, like duration and user type: the one-tick-per-trip chart you saw with the histogram, or a summary by group.
- A date and a count, like the start time: bars by hour or by day.
On a scatter plot you look for three things: whether the points rise or fall together (a trend), how tightly they cluster (whether the trend is firm or loose), and whether some groups of points behave differently. That last one is the one you're most likely to miss.
Place your bet before you draw. You already know there are 32 trips of 0 km: they leave from and return to the same station.
Check yourself: You're about to plot duration against distance for every trip under 120 minutes. Where do you think the 32 round trips land?
The chart puts one point per trip under 120 minutes. Add color=alt.Color("tipousuario:N", title="User type") inside encode to see who takes each kind of trip. You're done when you can pick out the round trips and tell what color most of them are.
Your turn
The chart puts one point per trip under 120 minutes. Add color=alt.Color("tipo_usuario:N", title="User type") inside encode to see who takes each kind of trip. You're done when you can pick out the round trips and tell what color most of them are.
The chart redraws when you pause typing, or right away with Run. Ctrl+Enter runs it. Tab indents; to leave the editor, press Escape, then Tab.
Ready to run your code.
What the chart shows
Scatter plot of 300 trips under 120 minutes: duration rises with distance, from 0 to 6.53 km, and at 0 km there is a column of 32 round trips with durations from under a minute to 96.4.
Data: viajes-rueda, 311 rows
Enter saves a cell and Escape leaves it as it was. Leave a cell empty for a missing value.
| Row | Remove | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ||||||||||
| 2 | ||||||||||
| 3 | ||||||||||
| 4 | ||||||||||
| 5 | ||||||||||
| 6 | ||||||||||
| 7 | ||||||||||
| 8 | ||||||||||
| 9 | ||||||||||
| 10 | ||||||||||
| 11 | ||||||||||
| 12 | ||||||||||
| 13 | ||||||||||
| 14 | ||||||||||
| 15 | ||||||||||
| 16 | ||||||||||
| 17 | ||||||||||
| 18 | ||||||||||
| 19 | ||||||||||
| 20 |
The points with a distance climb to the right: more kilometers, more minutes, with plenty of scatter around the trend. And at 0 km there's a separate column. Color shows who makes it up: of the 32 round trips, 16 are by occasional users. That sounds like few, but occasional users take only 71 of the 311 trips; subscribers, with 240, take the other 16. For an occasional user, riding out and back is much more common.
One number sums up how close two columns come to a straight line: the correlation (pandas' corr() computes Pearson's by default), which runs from -1 to 1. Near 1, they rise together almost in a line; near 0, no line connects them. With the 300 trips under 120 minutes, the correlation between distance and duration is 0.31: weak. Without the 0 km trips, it climbs to 0.75. The round trips, with a distance of 0 and all kinds of durations, hide a relationship that does exist among trips from one place to another. Check it yourself.
Run the code to see the correlation for all trips under 120 minutes. Then compute the same correlation using only the trips with a distance greater than 0.
Run the code to see the correlation for all trips under 120 minutes. Then compute the same correlation using only the trips with a distance greater than 0.
Run your code to see what it prints and the value of its last expression. Every run starts fresh. Ctrl+Enter runs it. Tab indents; to leave the editor, press Escape, then Tab.
Ready to run your code.
Run your code to see its output here.
Data: viajes-rueda, 311 rows
Enter saves a cell and Escape leaves it as it was. Leave a cell empty for a missing value.
| Row | Remove | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ||||||||||
| 2 | ||||||||||
| 3 | ||||||||||
| 4 | ||||||||||
| 5 | ||||||||||
| 6 | ||||||||||
| 7 | ||||||||||
| 8 | ||||||||||
| 9 | ||||||||||
| 10 | ||||||||||
| 11 | ||||||||||
| 12 | ||||||||||
| 13 | ||||||||||
| 14 | ||||||||||
| 15 | ||||||||||
| 16 | ||||||||||
| 17 | ||||||||||
| 18 | ||||||||||
| 19 | ||||||||||
| 20 |
Two columns moving together doesn't tell you why. Here the explanation seems obvious (farther takes longer), but often it isn't, and the table never tells you on its own. Look at this case from Rueda: the median duration for occasional users is 27.35 minutes and for suscriptor it's 17.4. Why?
Check yourself: Rueda's occasional users have a median of 27.35 minutes per trip and subscribers 17.4. What can you conclude from this table?
The third combination is time. A date-and-time column like inicio can be split into hours, days or days of the week, and you count the trips in each part. In Altair, hours(inicio) keeps just the hour of day of each trip, from 0 to 23.
Try it: who rides when
The chart counts trips by start hour. Add color=alt.Color("tipousuario:N", title="User type") inside encode and compare the hours when each type of user rides. You're done when you can say at which hours subscribers bunch up and at which hours occasional users do.
Your turn
The chart counts trips by start hour. Add color=alt.Color("tipo_usuario:N", title="User type") inside encode and compare the hours when each type of user rides. You're done when you can say at which hours subscribers bunch up and at which hours occasional users do.
The chart redraws when you pause typing, or right away with Run. Ctrl+Enter runs it. Tab indents; to leave the editor, press Escape, then Tab.
Ready to run your code.
What the chart shows
Bars with trips by start hour, from 7 to 23 (7 a.m. to 11 p.m.): two peaks, at 8 and at 18 (8 a.m. and 6 p.m.), and a lower plateau around midday.
Data: viajes-rueda, 311 rows
Enter saves a cell and Escape leaves it as it was. Leave a cell empty for a missing value.
| Row | Remove | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ||||||||||
| 2 | ||||||||||
| 3 | ||||||||||
| 4 | ||||||||||
| 5 | ||||||||||
| 6 | ||||||||||
| 7 | ||||||||||
| 8 | ||||||||||
| 9 | ||||||||||
| 10 | ||||||||||
| 11 | ||||||||||
| 12 | ||||||||||
| 13 | ||||||||||
| 14 | ||||||||||
| 15 | ||||||||||
| 16 | ||||||||||
| 17 | ||||||||||
| 18 | ||||||||||
| 19 | ||||||||||
| 20 |
Subscribers (adding up both labels) have two clear peaks: 36 trips that start in the 8 a.m. hour and 37 at 6 p.m. (18 on the chart). That's the pattern of heading to work or school and coming back. Occasional users barely show up at those hours and spread out between 11 a.m. and 3 p.m., with 13 trips at 3 p.m. And the third color shows up again, Suscriptor: the label you found on the categories page keeps splitting every chart by user type until someone fixes it.
For your log, a relationship gets written down like any finding: what moves with what, how firm it is, and which group behaves differently. For example: duration and distance: correlation 0.75 without the round trips, 0.31 with them; occasional users take half of the round trips. If you want to try another pair, ask your Maestro for ideas. And if someone asks you for a cause, practice your answer with your Maestro.