Do Research Findings Apply to My Students? Examining Study Samples and Sampling
Special educators are encouraged to use research to inform their instruction. However, it can be difficult to tell whether and how research findings apply to one’s own students. In the current issue of Learning Disabilities Research & Practice, Cook and Cook (2017; http://onlinelibrary.wiley.com/doi/10.1111/ldrp.12132/full) discussed two approaches for examining the degree to which research findings apply to one’s students: the classical generalization model and the case-to-case transferability model.
In the classical generalization model, generalization refers to the degree to which research findings derived from a sample of research participants can be extended or applied to the broader population. Generalizability depends on how representative a study sample is of a larger population. Representativeness refers to study participants accurately reflecting the population. For example, assume a state-level administrator needs to select a practice shown by research to improve reading outcomes for students with LD in her state. She should not choose a practice shown to be effective in a study that (a) was conducted in one affluent school and (b) included only students with LD with only reading comprehension difficulties (i.e., without problems in reading fluency) because this study sample is not representative of all students with LD in the state. Even though the practice worked for study participants, it might not be similarly effective for the larger population of students with LD in the state. Researchers can use probability sampling methods such as random selection to maximize the probability that their study sample is representative of a larger population. For example, if researchers randomly selected one out of every 50 students with LD in the state to participate in their study, it is highly probable that the study sample would be representative of students with LD in the state, and that study findings would therefore generalize to this population. Although examining whether study samples are representative of the population is a valid approach for identifying whether study findings apply to one’s own students, very few studies in special education use probability sampling.
The case-to-case transferability model is an alternative approach for examining the relevance of research findings to one’s own students that originated in qualitative research. In this approach, educators examine the match between their students and the participants in the research. The more research participants match one’s students on relevant characteristics (e.g., age, disability area, cultural and linguistic status), the more research findings are likely to transfer or apply to those students. For example, findings from a research study conducted with participants who were CLD fourth graders with LD are more likely to transfer or apply to other CLD fourth graders with LD than to non-CLD high school students with intellectual disabilities. It is important to note that (a) the match between study participants and one’s own students does not need to be exact for study findings to transfer and (b) that match and transferability exist on continuums. For example, although the match between (a) fourth-grade students with LD and (b) study participants who are fifth graders at risk for reading failure is not perfect, it is a much closer match (and, therefore, study results are much more likely to transfer) than for a study involving gifted high school students.
Among other considerations and caveats to bear in mind when examining the likelihood of research findings applying to one’s own students are:
- All other things being equal, the larger the number of participants in the study(ies), they greater the likelihood that research findings will generalize or transfer.
- Even if (a) a study sample is representative of the population to which one’s students belong or (b) the match between study participants and one’s students is very strong, there is no guarantee that results from research will apply to any particular student. Therefore, it is important to collect progress-monitoring data to examine the impact of any new interventions on individual learners.
- Examining the sample and sampling in research studies is just one consideration when deciding whether to adopt a practice. It is also important to consider factors such as the degree to which a practice addresses learner needs and goals, acceptability of the practice for the instructor and learner, and resource demands of the practice.
In conclusion, it is easy to overgeneralize research findings and for educators to inappropriately apply study results to their students. Special educators should examine the sample and the sampling method in studies to determine whether and the degree to which research findings are likely to generalize or transfer to their own students.PDF Back