Assessment, Statistics, and Research

What are age scores? Age scores, also called age-equivalent scores , are supposed to help people understand how a person’s test score compares to other people of the same age. They are often provided to teachers and parents to show how children scored on achievement tests compared to their age peers. A common age-equivalent abbreviation is AE .  Age scores are reported with a hyphen. The first number refers to age in years and the second number refers to the age in months. A score of 8-4 is supposed to mean a test performance typical of children age 8-years and 4-months. The scores appear convenient and make a kind of common sense. An age score of 7-6 is supposed to mean that a child earned a test score similar to children age 7-years and 6-months. But there are problems with the scores. What tests report age scores? Age scores are commonly reported with results of achievement tests. They are sometimes reported with results of intelligence tests. Old i...

The Theistic Spirituality Outcome Scale (TSOS) has potential as a useful outcome measure. Recently, a group of us completed a study of clients who saw Christian counselors. We assessed their current well-being using two measures: The Schwartz Outcome Scale (SOS) and the Theistic Outcome Scale (TSOS). (See references below.) The TSOS was designed by Richards (2005) as a measure of well-being for people associated with a theistic religion like Christianity, Judaism or Islam. We used the 17-item version, which uses a 5-point response format from  1 = never to 5 = almost always to rate each item (e.g., “I felt spiritually alive.”). Reliability We only calculated coefficient alpha, which was strong at .95. Validity The TSOS was significantly correlated with ratings of satisfaction with Christian counseling (.65) and likelihood of returning to Christian counseling (.62). It was significantly correlated with the SOS measure of general well-being (.84). Other signi...

Who would think that a simple statistic like a mean or a median would make a difference? In large samples involving thousands of people, and when data are normally distributed (close to the shape of a bell curve), the mean and median will be nearly the same. In fact, in a theoretical distribution called the normal curve , the mean , median , and   mode are in the middle. But, many samples are not normal distributions . Instead, the often contain extreme scores called outliers or a lot of scores bunched up at high or low levels ( skewed ). Sadly, even people that understand statistics, continue to report the mean as if they are not thinking about their samples. Suppose you work for a company where the top person earns $300,000 but most folks earn $30,000 to $60,000. Well that $300,000 is gonna skew results and the mean will look much higher than the median. I ran some fictitious data on a sample of 10 people. Nine earn between $30 and $60K and one earns $300K. The Mean ...

The Schwartz Outcome Scale (SOS ) has the potential to be a useful measure of counseling outcomes. Overview I and my colleagues used the 10-item version of the Schwartz Outcome Scale in two recent studies about psychotherapy patients. The scale measures several aspects of well-being:  physical, relational, and psychological functioning and the person’s capacity to be peaceful, interested, excited, and satisfaction with life. RELIABILITY In our two studies, coefficient alpha  reliability values were .93 and .96 (Sutton et al., 2018). VALIDITY In previous research, the SOS was linked to hope, self-esteem, affect, mental health, and life satisfaction    (Young, Waehler, Laux, McDaniel, & Hilsenroth, 2003). Some validity findings based on positive correlations with other measures may be of interest to clinicians and researchers (Sutton et al. (2018).    Satisfaction with counseling .63    Likely to return to counse...

  What makes a test valid? is a tricky question.  The short, and rather obnoxious response is, “nothing.”  Like reliability , validity is a property of test scores  rather than tests but more accurately, an interpretation of the scores. But it is important to take the question seriously when test-takers and users are wondering how much confidence to place in a test score. As with many aspects of science, the answers can be simply stated but there is a complicated backstory. Validity Traditions For many, the traditional views of test score validity will be sufficient. Tests measure constructs. Scientific constructs are ideas that have features that can be measured like reading comprehension, dominance, short-term memory, and verbal intelligence. Construct validity is not a single entity but rather the current state of knowledge about how a test instrument’s scores have functioned in many settings and in relation to crite...

Photo by Geoff W. Sutton, 2017 Researchers define religious fundamentalism in different ways. One recent model focuses on the way religious people view their sacred text. I have written about the Intratextual Fundamentalism model in a previous post ( October 2013) . In this post, I provide some data related to the 5-item version of the Intratextual Fundamentalism Scale (IFS), which I have found useful in research projects. The revised version of the scale (IFS) has five items--each measuring a dimension of intratextuality (Williamson, Hood, Ahmad, Sadiq, & Hill, 2010). Here are the five dimensions (from my previous blog): Divine : The sacred text is a revelation from God (or of divine origin) to humans. Regardless of the involvement of people in the writing of the text, God (or a deity) is the author. Inerrant : The sacred text does not contain errors, inconsistencies, or contradictions. The text is objectively true. Privileged : The sacred text of the fundamentalist gr...

One way to think about the components of religion is three-dimensional, which includes  beliefs, practices, and experiences.  A few years ago, a group of us studied Christian counseling to discover what Christian counselors actually did that was different from other counselors (Sutton, Arnzen, & Kelly, 2016). We wanted to get more specific about the identity of Christian counselors--beyond a simple checklist of their affiliation with a large group such as Presbyterian or a movement such as Pentecostal. As part of our plan to be more specific about spirituality, we created a few measures.  Previously, I reported on a scale for assessing spiritual practices . This time I present a measure of beliefs, the Christian   Beliefs Index .             T he wording of the items clearly applies to the Christian faith, but the point of our measure was to be more precise about the diversity of beliefs within C...