Inference Validity
Inference
validity refers to the validity of a research design as a whole. It
refers to whether you can trust the conclusions of a study. Generally
the issue concerns causality. Statistical measures show relationships,
but it is the theory and the study design that affect what kinds of
claims to causality you can reasonably make.
Internal validity
Refers to whether claimed conclusions, especially
relating to causality, are consistent with
research results (e.g., statistical results) and research design
(e.g., presence of appropriate control variables, use of appropriate
methodology).
Internal validity can sometimes be checked via
simulation, which can tell you whether a
given theorized process can in fact yield the outcomes that you claim it
does.
External validity
Refers to generalizability of the results. Does it say anything
outside of the particular case?
|
A carpenter, a school teacher, and scientist were
traveling by train through Scotland when they saw a black sheep
through the window of the train.
"Aha," said the carpenter with a smile, "I see that Scottish
sheep are black."
"Hmm," said the school teacher, "You mean that some Scottish
sheep are black."
"No," said the scientist glumly, "All we know is that there is
at least one sheep in Scotland, and that at least one side of that
one sheep is black." |
Three strategies for strengthening external validity:
- Sampling. Select cases from a known population via a
probability sample, then claim the results apply to the population
as a whole
- Representativeness. Show the similarities between the
cases you studied with a population you wish your results to be
applied to
- Replication. repeat the study in multiple settings.
Use meta statistics to evaluate the results across studies. Although
journal reviewers might not agree, consistent results across many
settings with small samples is more powerful than a large sample of
a single settings.
|
