Assessment, Statistics, and Research

The reason counselor's tests are not reliable is that reliability is a property of scores not tests.  This isn't a matter of semantics. Think about it this way. Give all the students in one school an achievement test. The test items don't change so they appear stable, consistent, and reliable. However, when publishers report reliability values, they calculate the reliability statistics based on scores. Scores vary from one administration to another. If you ever took a test twice and got a different score, you know what I mean. Individuals change from day to day. And we change from year to year. Also, even a representative sample of students for a nation can be different each year. Everytime we calculate a reliability statistic, the statistic is slightly different. Reliability values vary with the sample. Reliability values also vary with the method used for calculation. You can get high reliability values using  coefficient alpha  with scores from a one-tim...

When counselors and psychologists report test scores, they often report one of the scores found in the table below. When several tests are used, it is helpful to know how the scores compare from one test to another. A good place to begin is to locate the average score-- that's the row where z = 0. Then look at the broad middle range between z = -1 and z = 1. About 68% of people score between z = -1 and z = 1. Intelligence Tests use Standard Scores abbreviated as SS. These scores take the place of the old IQ score. An average IQ is 100 -- about 68% of people score between 85 and 115. Here's a table from Appendix B of Applied Statistics: Concepts for Counselors Each row contains the equivalent score on a different scoring system. For example, a z -score of 1 equals a T score of 60, and a standard score of 115. The score is at the 84 th percentile. z T Standard Percentile Ra...

Why aren't tests reliable? The reason tests are not reliable is that reliability is a property of the interpretation of scores not the tests themselves.   This isn't a matter of semantics. Think about it this way. Give all the students in one school an achievement test. The test items don't change so they appear stable, consistent, and reliable. However, when publishers report reliability values, they calculate the reliability statistics based on scores. Scores vary from one administration to another. If you ever took a test twice and got a different score, you know what I mean. Individuals change from day to day. And we change from year to year. Also, even a representative sample of students for a nation can be different each year. Every time we calculate a reliability statistic, the statistic is slightly different. Reliability values vary with the sample. Reliability values vary with the method of calculation. Reliability values also vary with the method us...