Start with the people the study is meant to describe
Selection bias arises when the way people enter or remain in a study produces a misleading result. To spot it, first separate two groups: the population a finding appears to describe and the people who actually supplied data. A survey described as measuring “student study habits,” for example, may include only students who visit a library during exam week. Those visitors might study differently from students who do not use the library.
Write down the study’s apparent target population in ordinary language, then look for its eligibility rules and sampling frame: the list, place or process from which participants could be recruited. Ask who had no chance of appearing. A workplace survey sent only to full-time staff cannot, by itself, describe the experiences of contractors. This gap does not make every finding useless, but it narrows what the evidence can support.
Trace each step from invitation to final sample
A sample is shaped by more than its initial recruitment. People must be reached, decide whether to participate, meet eligibility criteria and sometimes complete follow-up. At each step, ask whether those who remain could differ on the characteristic being studied. In a survey about commuting stress, people with especially long commutes might have less time to respond. A low response rate alone does not show the size or direction of bias; the concern is whether response is related to the answer.
Exclusions deserve the same attention. Suppose a study of a training program analyzes only participants who finished every session. If people who found the program unhelpful were more likely to stop attending, results for completers could look more favorable than results for everyone who started. Look for counts at each stage, reasons for exclusion and whether the study compares those retained with those lost. Missing details should make you more cautious about a broad conclusion.
Ask what selection might change
Not every unrepresentative sample distorts every conclusion. The key question is whether selection is connected to the outcome, the factor being compared, or both. Imagine a study comparing two teaching methods using volunteers from an advanced course. Their test scores may not represent all students’ scores. If volunteers for one method also tend to have more prior experience than volunteers for the other, the comparison between methods may be misleading too.
Distinguish between a claim about how common something is and a claim about how two things relate. A survey of avid cyclists will probably be a poor basis for estimating how many residents cycle regularly. It might still reveal useful patterns among avid cyclists, if described that way. Conversely, even a study with many participants can produce a distorted comparison when the route into each group depends on qualities that also affect the outcome. Sample size cannot repair a faulty selection process.
Use practical clues, not a single warning sign
When reading a study, look for a clear account of where and when recruitment happened, who was eligible, how many people were approached and how many entered the analysis. Check whether a comparison group was selected through a similar process. A health survey conducted through a fitness app, for instance, may reach many people while missing those who do not use such apps. The issue is not that app users are inherently unreliable; it is that app use may be linked to the behavior the survey aims to describe.
Be alert to convenience samples presented with population-wide language, follow-up losses that differ between groups and results reported only for a selected subset. These are prompts for closer inspection, not automatic verdicts. Researchers may limit the claim to the group they studied, use recruitment methods suited to their question or explain why missing participants are unlikely to change the main finding. Judge the conclusion alongside those choices.
State the limitation precisely
A useful critique names the selection step and the possible consequence. Instead of saying, “The study is biased,” try: “Participants were recruited from library visitors, so the reported study time may not reflect students who rarely visit the library.” If you cannot tell which way a result would shift, say so. The absence of nonrespondents may make an estimate too high, too low or little changed, depending on how their answers would have differed.
Finally, ask what evidence would strengthen your confidence: a broader sampling frame, response information, comparisons with known population characteristics or similar findings from a differently recruited sample. Selection bias is a question about the path from the intended population to the analyzed participants. Following that path helps you decide not just whether a study has a limitation, but exactly which conclusions deserve a narrower reading.
