Consider this hypothetical example. Suppose we want to study whether long commutes prevent university students from attending morning classes. We set up an online survey, and during a single week in March, receive responses from 180 students at one university. Of the 180, 60 say their one-way commute is over 45 minutes. In this group, 35 per cent report having skipped at least one morning lecture in the past month. For the remaining 120, the percentage is 20 per cent. The survey asks about travel times and self-reported class attendance, rather than using transport logs or electronic registers to measure both variables.

This comparison shows a clear association, which warrants deeper study. But it doesn’t imply that having to travel more than 45 minutes caused any given student to skip class. There are several possible confounders, factors such as employment, care responsibilities, disabilities, the construction of timetables and the existence of recorded lectures, that might affect both how long someone’s commute is, and how often they attend class. The survey didn’t control for any of these. In addition, even if all participants were honest, students who occasionally travel more than 45 minutes might count this as their typical commute, while a student who once missed a class because of travel difficulties might be more likely to recall that instance than an instance when travel didn’t cause them to miss a class.

The recruitment method also limits the usefulness of the data. The survey was advertised through two student email lists and participation was optional. Students who were having trouble travelling to campus might be more inclined to complete the survey, whereas students who were particularly pressed for time might be unlikely to do so. Therefore, the survey results can’t be considered representative of the student population at the university, let alone students elsewhere. While a larger sample size would be better for teaching purposes, it wouldn’t compensate for any systemic bias in how the survey was distributed.

Ideally, we would recruit a sample that spans multiple academic programmes, include measures of key demographics, track attendance over a longer period, and compare different class schedules while controlling for the reasons why students selected them. Without such a study, the survey we’ve described can inspire new lines of inquiry and discussions about whether morning classes should be scheduled. But it shouldn’t be cited as evidence that shortening every commute to under 45 minutes would lead to higher attendance rates. To do so would be to make a predictive claim that the current data doesn’t support.