Assessment, Statistics, and Research

The reason counselor's tests are not reliable is that reliability is a property of scores not tests.  This isn't a matter of semantics. Think about it this way. Give all the students in one school an achievement test. The test items don't change so they appear stable, consistent, and reliable. However, when publishers report reliability values, they calculate the reliability statistics based on scores. Scores vary from one administration to another. If you ever took a test twice and got a different score, you know what I mean. Individuals change from day to day. And we change from year to year. Also, even a representative sample of students for a nation can be different each year. Everytime we calculate a reliability statistic, the statistic is slightly different. Reliability values vary with the sample. Reliability values also vary with the method used for calculation. You can get high reliability values using  coefficient alpha  with scores from a one-tim...

Can you answer these questions that every counselor ought to know? Choose the BEST available answer. I'll post the answers below. 1. If the correlation between a test of intelligence and a test of achievement is usually between .88 and .92, how well can you use the intelligence test results to predict achievement test results? A. Very well B. Moderately well C. Not well at all D. None of the above 2. A personality test score was high on a scale of Extraversion. The validity of the Extraversion scale was reported as .52 to .57 compared to similar tests. How much confidence should the person have that their score is "valid?" A. A high degree B. A moderate degree C. A low degree D. None of the above 3. School counselors administered a questionnaire to 1,000 students. They calculated results for answers about four school improvements rated on a scale of 1 to 5. Most of the scores were in the range of 18 to 20. The counselors reported a mean rating o...

The Theistic Spirituality Outcome Scale (TSOS) has potential as a useful outcome measure. Recently, a group of us completed a study of clients who saw Christian counselors. We assessed their current well-being using two measures: The Schwartz Outcome Scale (SOS) and the Theistic Outcome Scale (TSOS). (See references below.) The TSOS was designed by Richards (2005) as a measure of well-being for people associated with a theistic religion like Christianity, Judaism or Islam. We used the 17-item version, which uses a 5-point response format from  1 = never to 5 = almost always to rate each item (e.g., “I felt spiritually alive.”). Reliability We only calculated coefficient alpha, which was strong at .95. Validity The TSOS was significantly correlated with ratings of satisfaction with Christian counseling (.65) and likelihood of returning to Christian counseling (.62). It was significantly correlated with the SOS measure of general well-being (.84). Other signi...

Why aren't tests reliable? The reason tests are not reliable is that reliability is a property of the interpretation of scores not the tests themselves.   This isn't a matter of semantics. Think about it this way. Give all the students in one school an achievement test. The test items don't change so they appear stable, consistent, and reliable. However, when publishers report reliability values, they calculate the reliability statistics based on scores. Scores vary from one administration to another. If you ever took a test twice and got a different score, you know what I mean. Individuals change from day to day. And we change from year to year. Also, even a representative sample of students for a nation can be different each year. Every time we calculate a reliability statistic, the statistic is slightly different. Reliability values vary with the sample. Reliability values vary with the method of calculation. Reliability values also vary with the method us...