Showing posts with label Reliability. Show all posts
Showing posts with label Reliability. Show all posts

Saturday, February 19, 2022

Spearman-Brown

 



The Spearman-Brown formula estimates test score reliability of a full-length test when using a split-half method of reliability.

The split-half method divides a test in two and calculate a correlation between the scores of the two halves. Longer tests are yield higher reliability values so the Spearman-Brown formula estimates the reliability value for the full test.

Other names for the Spearman-Brown formula are Spearman-Brown Prophecy Formula, Spearman-Brown Correction.


Learn more about test statistics in

Buy Applied Statistics for Counselors

 

GOOGLE BOOKS

 

AMAZON





Please check out my website   www.suttong.com

   and see my books on   AMAZON       or  GOOGLE STORE

Also, consider connecting with me on    FACEBOOK   Geoff W. Sutton    

   TWITTER  @Geoff.W.Sutton    

You can read many published articles at no charge:

  Academia   Geoff W Sutton     ResearchGate   Geoffrey W Sutton 


Monday, April 26, 2021

Coefficient Alpha or Cronbach's Alpha

 Coefficient Alpha (also called "alpha") is a statistical value indicating the degree of internal consistency of items in a multiple-item scale like survey items or Likert-type scales.

Internal consistency is one measure of reliability for scores from scales, measures, and survey items.

The alpha statistic was developed by Lee Cronbach in 1951 thus it is also called Cronbach's alpha. In research reports, you may just see the Greek lower case letter alpha, α.


The procedure to calculate alpha can be found in SPSS under Analyze > Scale > Reliabilty.

For research purposes, scales with alpha levels equal to or above alpha = .70 are acceptable. The best scales have values of alpha = .9 or higher.

The alpha method works best to evaluate unidimensional measures. If there are two or more dimensions in a set of items, the alpha value will be lower so, when alpha values are low, consider which item or items do not support the primary dimension.

Cite this Post

Sutton, G.W. (2021, April 26). Coefficient Alpha or Cronbach’s Alpha. Retrieved from https://statistics.suttong.com/2021/04/coefficient-alpha-or-cronbachs-alpha.html


Learn more about statistics in Applied Statistics Concepts for Counselors on AMAZON   or   GOOGLE








Learn more about surveys and statistics in Creating Surveys on    AMAZON or    GOOGLE












Please check out my website   www.suttong.com

   and see my books on   AMAZON       or  GOOGLE STORE

Also, consider connecting with me on    FACEBOOK   Geoff W. Sutton    

   TWITTER  @Geoff.W.Sutton    

You can read many published articles at no charge:

  Academia   Geoff W Sutton     ResearchGate   Geoffrey W Sutton 

Monday, December 14, 2020

Metaphors Can Interfere with Understanding Survey Items and Results

Photo for illustration purposes only

“If Jesus is God, how could he create the world if he wasn’t born yet.”

                     —Girl, age 7

It will be a while until this 7-year-old passes through the stage of concrete operations and begins to pull apart various mental constructs in a serious fashion. Along the way she’ll pick up many metaphors, including those that unravel men’s thinking about God hundreds of years ago. And all sorts of other metaphors.

Americans are known for being religious and in particular, for being Christian; however, as is commonly said, the devil is in the details.

In this post, I look at religious survey items to make a point about being careful when writing and interpreting survey items containing concepts with a range of meaning.

*********

God- Who is God?

Gallup keeps tabs on Americans’ views on God. In an interesting article, Hrynowski (2019) reveals a different response rate for beliefs in God depending on how the question is asked. Specifically, they asked the question about belief in God three ways.

1. The simple question, “Do you believe in God?” gets the highest response—86 to 89% in recent years.

2. When given a few options the percentage of belief drops to 79% in recent polling.

3. When asked if they are convinced that God exists and given other options, the percentage of believers in God falls to 64%

Jesus—Who is Jesus?

Centuries after Jesus life on earth ended, religious leaders argued about his nature and formulated statements essentially saying Jesus is both God and man. For Christians, the widely accepted doctrine of the trinity declares God to be “Father, Son, and Holy Spirit.”

In 2020, the State of Theology survey asked Americans about Jesus by presenting a statement: “Jesus was a great teacher, but he was not God.” Survey participants could select five options representing levels of agreement. Only 28% strongly agreed. Another 23% chose the “agree” option. So if you combine the two levels of agreement, you get 52% and if you add the opposite two choices of strongly disagree (27%) and somewhat disagree (10%) you obtain a level of disagreement of 36%.

What about evangelicals? Despite the difficulty in defining who is and who isn’t an evangelical, the researchers analysed the results to see how those participants who identified as evangelical answered the question. It turns out, 30% agreed.

The Bible—What is Truth?

When it comes to Christians’ ideas about God and Jesus, it’s reasonable to turn to the source material. Thus, the researchers asked participants their beliefs about the Bible. As with all surveys, how the question is asked can make a difference. Here’s the Statement of Theology survey statement:

“The Bible, like all sacred writings, contains helpful accounts of ancient myths but is not literally true.”

As with other items, participants selected from the range of strongly disagree to strongly agree. The strongly agree level for the US population is 20%.

The statement suggests a dichotomy that ignores an understanding of truth revealed in metaphors. Can you rewrite the statement?

Metaphors

Writers like Marcus Borg (1995) attempt to help Christians deal with various conundrums by pointing out the biblical metaphors about Jesus. So, some writers refer to Jesus as the Son of God. But Jesus is also presented as a lamb and the word.

There are many words in the Bible about God. A dominant presentation is that God is a male figure. Sometimes God is presented as a husband (e.g., Isaiah 54:5)—even a jealous one—and sometimes God is presented as a Father (e.g., Matthew 6: 9-13). But the Bible also refers to God as a Spirit (John 4:24). And in Christian teaching, people lose their distinctiveness as male or female (Galatians 3:28).

So, the point is, are these descriptions of God just reflections of men using metaphors that made sense to people living in male dominated cultures thousands of years ago or must we view God as a man to have a correct understanding? Surely, if Christians do not think metaphorically, the idea that God is like a man is rather limiting.

Of course, there are many religions besides Christianity in the world and those religious people understand God or gods in ways that are different from the diverse views of Christians about the God of the Bible.

Survey Limitations

1. Survey results are interesting but I hope these examples show that people appear to have different perspectives on the central person in their faith.

2. When researchers expand the wording of survey items, the investigators may obtain  more nuanced responses.

3. In some cases, giving participants the opportunity to add a text response to a survey item can help clarify nuances of meaning.

4. In some cases, metaphors make a difference in interpreting the results of a survey. The familiarity of the survey writer and participant with relevant metaphors can enhance or obscure the meaning of the results.

5. When a survey does not consistently tap the same domain of knowledge, reliability is negatively affected to an unknown degree.

6. I once examined the relationship between the intelligence of graduate students to the results obtained when they administered and scored intelligence tests. I never published those data. But I am left with a hypothesis related to this topic— the intelligence of survey writers can affect not only the wording of the items but the interpretation of the results as well. And of course, there is the unknown factor of the intelligence of the people responding to a survey.

Creating Surveys on AMAZON    or   GOOGLE  Worldwide



Final Note

If a 7-year-old girl can ask a thoughtful question about biblical literalism, imagine the difficulty in ascertaining what thoughtful adults really think about a survey topic.

 

Links to Connections

Checkout My Page    www.suttong.com

  

My Books  AMAZON          and             GOOGLE STORE

 

FOLLOW me on   FACEBOOK   Geoff W. Sutton   TWITTER  @Geoff.W.Sutton

 

PINTEREST  www.pinterest.com/GeoffWSutton

 

Articles: Academia   Geoff W Sutton   ResearchGate   Geoffrey W Sutton 

 

 

 

Tuesday, April 21, 2020

Measurement Error Standard Error of Measurement

In testing, measurement error usually refers to the fact that the same people can obtain different scores on the same test at different times. In a broad sense, measurement error can also refer to the degree of accuracy of a test to correctly identify a condition, which is discussed as test validity.

Recall that test score reliability is a necessary but insufficient condition for test score validity.

Many tests in psychology, medicine, and education are useful. The reliability of the scores will vary depending on such factors as the properties of the test itself as well as how well the user follows standard procedures in administering the test, environmental factors that can affect the scores, and factors within the person taking the test.


The scores on many tests conform to the pattern called the normal curve or bell curve. In classical test theory, the scores people obtain on tests are simply called obtained scores (symbol X). Statisticians consider the variation in scores to estimate a "true score." Variations of obtained scores around the theoretical true score (symbol T) indicate error because a reliable test ought to yield the same score every time it is used. The deviations of those obtained scores are referred to as error (symbol E). In a formula, X = T + E.

Theoretically, the reliability of test scores depends on the ratio of variances of the true scores divided by the variances of the obtained scores. A perfectly reliable test would yield a reliability value of rxx = 1.0. In reality, most of the better tests yield average reliability values above .90. Test publishers are obligated by professional ethics to include reliability values in their test manuals.

Studies of score patterns allow statisticians to calculate the average variability of score error. Thus, for any given published test, there ought to be a statistic known as the Standard Error of Measurement, which is abbreviated as SEM.


Once the history of the SEM for a test is known based upon large scale studies, users can use that value to estimate how the scores of test takers might vary if the test taker were to take the same test again under similar conditions. The estimates are based on the properties of the normal curve thus, the test must yield scores that conform to the normal score pattern to use a SEM based on this model.

Example, suppose a student obtains an IQ score of 100 and one SEM = 4 then on future administration of the same test, the student would likely score between 96 and 104 68% of the time.

The process of forming a range of values around the obtained score should remind users and test takers that scores are not fixed properties. Scores vary and they tend to vary in a "standard" pattern. In this theory, the error variance has been standardized. Clearly, a user who wanted to be careful could use 2 SEMs, which would then allow a range of plus and minus 8 points. In the example, the IQ could range between 108 and 92.

It is important to keep in mind that tests are neither reliable or unreliable because reliability is the property of scores not tests. Thus it is incorrect to refer to a test as reliable or unreliable. We can speak about the degree of reliability of the scores.

There are other theories about testing and reliability.

The concept of how well a test accurately identifies a criterion, see the discussion of validity.


Cite this Blog Post

Sutton, G.W. (2020, April 21). Measurement error standard error of measurement. Assessment, Statistics, & Research. https://statistics.suttong.com /2020/04/measurement-error-standard-error-of.html


Read more about statistics in these two books.

Read more about basic statistics in APPLIED STATISTICS: CONCEPTS FOR COUNSELORS at  AMAZON













Creating Surveys on AMAZON    or   GOOGLE  Worldwide












Connections

My Page    www.suttong.com
  
My Books  AMAZON                       GOOGLE STORE

FACEBOOK   Geoff W. Sutton
TWITTER  @Geoff.W.Sutton

Publications (many free downloads)
 
Academia   Geoff W Sutton   (PhD)     

  ResearchGate   Geoffrey W Sutton   (PhD)



Structural Equation Modeling (SEM)

  Structural Equation Modeling (SEM) is a statistical technique that is widely used in psychology and related fields to examine the relatio...