Assessment, Statistics, and Research

Averages Can Be Deceiving What You Need to Know About Averages Most people assume that a simple statistic like a mean or a median tells the whole story. It doesn’t. Averages can clarify—but they can also mislead—depending on which one you use and how your data are distributed. Understanding the mean, median, and mode helps you interpret test scores, salaries, evaluations, and research findings with greater accuracy. CALCULATOR : I have included a basic calculator at the bottom of this page. The Mean, Median, and Mode Mean The mean is the arithmetic average of a set of numbers. You calculate it by adding all the values and dividing by the number of values. Example: For the data 1, 2, 3, 4, 5, the sum is 15 and the mean is 3. The mean uses every value in the dataset, which makes it useful but also sensitive to extreme scores. Median The median is the middle value when numbers are arranged from lowest to highest. Half the values fall above it and half fall below it. Example: In the set 1,...

The Schwartz Outcome Scale (SOS ) has the potential to be a useful measure of counseling outcomes. Overview I and my colleagues used the 10-item version of the Schwartz Outcome Scale in two recent studies about psychotherapy patients. The scale measures several aspects of well-being:  physical, relational, and psychological functioning and the person’s capacity to be peaceful, interested, excited, and satisfaction with life. RELIABILITY In our two studies, coefficient alpha  reliability values were .93 and .96 (Sutton et al., 2018). VALIDITY In previous research, the SOS was linked to hope, self-esteem, affect, mental health, and life satisfaction    (Young, Waehler, Laux, McDaniel, & Hilsenroth, 2003). Some validity findings based on positive correlations with other measures may be of interest to clinicians and researchers (Sutton et al. (2018).    Satisfaction with counseling .63    Likely to return to counse...

  What makes a test valid? is a tricky question.  The short, and rather obnoxious response is, “nothing.”  Like reliability , validity is a property of test scores  rather than tests but more accurately, validity is an interpretation of the scores. But it is important to take the question seriously when test-takers and users are wondering how much confidence to place in a test score. As with many aspects of science, the answers can be simply stated but there is a complicated backstory. Validity Traditions For many, the traditional views of test score validity will be sufficient. Tests measure constructs. Scientific constructs are ideas that have features that can be measured like reading comprehension, dominance, short-term memory, and verbal intelligence. Construct validity is not a single entity but rather the current state of knowledge about how a test instrument’s scores have functioned in many settings and in relat...

Photo by Geoff W. Sutton, 2017 Researchers define religious fundamentalism in different ways. One recent model focuses on the way religious people view their sacred text. I have written about the Intratextual Fundamentalism model in a previous post ( October 2013) . In this post, I provide some data related to the 5-item version of the Intratextual Fundamentalism Scale (IFS), which I have found useful in research projects. The revised version of the scale (IFS) has five items--each measuring a dimension of intratextuality (Williamson, Hood, Ahmad, Sadiq, & Hill, 2010). Here are the five dimensions (from my previous blog): Divine : The sacred text is a revelation from God (or of divine origin) to humans. Regardless of the involvement of people in the writing of the text, God (or a deity) is the author. Inerrant : The sacred text does not contain errors, inconsistencies, or contradictions. The text is objectively true. Privileged : The sacred text of the fundamentalist gr...

Overview The Christian Beliefs Index is a four item self-report 5-point rating scale of Christian beliefs. One way to think about the components of religion is three-dimensional, which includes  beliefs, practices, and experiences.  A few years ago, a group of us studied Christian counseling to discover what Christian counselors actually did that was different from other counselors (Sutton, Arnzen, & Kelly, 2016). We wanted to get more specific about the identity of Christian counselors--beyond a simple checklist of their affiliation with a large group such as Presbyterian or a movement such as Pentecostal. As part of our plan to be more specific about spirituality, we created a few measures.  Previously, I reported on a scale for assessing spiritual practices . This time I present a measure of beliefs, the Christian   Beliefs Index .             T he wording of the items clearly applies to the C...

Why aren't tests reliable? The reason tests are not reliable is that reliability is a property of the interpretation of scores not the tests themselves.   This isn't a matter of semantics. Think about it this way. Give all the students in one school an achievement test. The test items don't change so they appear stable, consistent, and reliable. However, when publishers report reliability values, they calculate the reliability statistics based on scores. Scores vary from one administration to another. If you ever took a test twice and got a different score, you know what I mean. Individuals change from day to day. And we change from year to year. Also, even a representative sample of students for a nation can be different each year. Every time we calculate a reliability statistic, the statistic is slightly different. Reliability values vary with the sample. Reliability values vary with the method of calculation. Reliability values also vary with the method us...