What makes a test valid? is a tricky question.
The short,
and rather obnoxious response is, “nothing.”
Like reliability,
validity is a property of test scores
rather than tests but more accurately, an interpretation
of the scores.
of the scores.
But it is
important to take the question seriously when test-takers and users are
wondering how much confidence to place in a test score. As with many aspects of
science, the answers can be simply stated but there is a complicated backstory.
Validity Traditions
For many, the traditional views of test score validity will
be sufficient. Tests measure constructs. Scientific constructs are ideas that
have features that can be measured like reading comprehension, dominance, short-term
memory, and verbal intelligence.
Construct validity
is not a single entity but rather the current state of knowledge about how a
test instrument’s scores have functioned in many settings and in relation to
criteria. Construct validity primarily includes findings from studies of content
validity, convergent validity and discriminant validity.
Content validity
is based on judgment analysis from
experts who mostly agree that test items measure the construct (e.g., marital
satisfaction).
The other types of validity are based on the concept of
correlations with a criterion. Researchers ask participants to take a specific
test X along with other tests Y and Z. Test X is the test of interest such as a
new math achievement test. Test Y represents other similar tests such as other
math tests. When test X and test Y yield similar scores we have evidence of convergent
validity.
When test X and test Z yield dissimilar results such as a
relationship between our test X math achievement and test Z vocabulary, we have
evidence of discriminant validity—a math test ought not to measure
vocabulary aside from the minimal vocabulary used in the instructions and word
problems. The relationship between the tests is based on a statistic called the
validity coefficient, which will vary anytime you have a group of people taking
two tests—even the very same people will get different scores on two different
testing dates.
Criterion validity
compares test scores to some criterion. The relationship between depression test
scores measuring depression today is called concurrent validity. The relationship between test scores today and
some future measurable performance is predictive
validity—for example, a pre-employment test may be correlated with
supervisor ratings after six months on the job.
Aside from content validity, most traditional studies are
looking at the strength of the relationship between one set of test scores and
another.
Factor analysis
is a complex correlational procedure that examines the underlying relationship
among test items and how they relate to other test items. For example, a set of
vocabulary items may be correlated with answers to questions about general
knowledge and be considered a “verbal factor” when the two sets of items may be
grouped as representing an underlying verbal factor. These abstract underlying
factors are sometimes called latent
variables or latent traits.
Read more about validity of surveys and tests in CREATING SURVEYS- Chapter 18.
Counselors, read more about validity of test scores in APPLIED STATISTICS: CONCEPTS FOR COUNSELORS-
Chapter 20.
