Assessment, Statistics, and Research

  A post hoc test is a statistical test used to determine if a pair of values are significantly different from each other after the primary analysis has been completed. The term post hoc is a Latin phrase meaning after the event. A common use of post hoc tests is the comparison of group means after an F -test in an ANOVA has revealed significant differences among the groups. The reason to test for differences after an overall test like ANOVA is to reduce the risk of finding a significant difference by chance. That is, if researchers perform a large number of tests on a sample, they may find one or more tests significant by chance. There are many post hoc tests. Following are some examples of tests that compare the means of two groups. Bonferroni Test This is a popular test. By dividing the significance level by the number of comparisons, the risk of finding a significant difference by chance is reduced. This procedure is called the Bonferroni Correction. Tukey's Honest Significant

In statistics, an effect size ( ES ) indicates the strength of the relationship between two variables. In psychological experiments, researchers are interested in the strength of the effect of the Independent Variable on the Dependent Variable. In psychotherapy studies, researchers may be interested in the effects of treatment on a measure of the dependent variable. A research questions may be framed: How effective is a set of 6 CBT sessions on the reduction of depression? Psychologists have often described effect sizes as small, medium, or large. Cohen's d Cohen's d is a measure of effect size between two groups. The mean of one group is subtracted from a second group and divided by the pooled standard deviation of the two groups. ES = (M1 - M2) / SD Effect Size  Label 0.2     Small 0.5     Medium 0.8     Large Pearson Correlation Coefficient ( r ) 0.1 to 0.3  Small 0.3 to 0.5  Medium 0.5 to 1.0   Large Converting Cohen's d to the correlation coefficient r =   d / √ d 2

  Factor Analysis and Assessment In testing, factor analysis is a mathematical strategy to analyze groups of items within a large test to see how well they relate to each other. The goal will be to reduce the large number of items to a set of factors that appear to measure different but related constructs; hence, factor analysis is a method of data reduction. (Sutton, 2020) A large test of various abilities may be analyzed for ways to group different abilities. Short tests of vocabulary, verbal analogies, and synonyms might form a factor that a researcher could label as "Verbal Abilities." A factor is a group of variables that are highly correlated with each other and, although different, they appear to have something in common. Researchers choose names for groups of variables based on the content of the variables in the factor. In large research projects, each participant may have scores on a large number of variables. Factor analysis can be used to identify patterns amo

  Scale name: Attitudes to Disability Scale (ADS) Scale overview: The Attitudes to Disability Scale (ADS) is a 16-item rating scale designed to measure attitudes toward disability. The ADS was translated into multiple languages.   Response Type: Items are rated on a 5-point Likert scale of agreement.   Scale items = 16      Scale subscales = 4 Using factor analysis, the authors identified four factors in the 16-item Attitudes to Disability Scale: Inclusion, Discrimination, Gains, and Prospects. Inclusion People with a disability find it harder than others to make new friends Discrimination People often make fun of disabilities Gains Having a disability can make someone a stronger person Prospects People with a disability have less to look forward to than others   Reliability: The authors reported ADS  Cronbach’s alpha values of.795 and .764 in two samples. Validity: The authors examined the structure of the scale using Confirmatory Factor Analysis.

  Scale name: Attitudes Toward Abortion Scale Scale overview: The Attitudes Toward Abortion Scale is a 12-item, four-choice agreement rating scale. Authors: Michael Taylor and George Whitehead Response Type: Items are rated on a scale of agreement: strongly agree, agree, disagree, strongly disagree. Subscales = 2 with sample item Moral/Legality 1. Abortion is not acceptable under any circumstances. External Influence 10. Abortions should be partially covered by insurance companies.   Reliability: Cronbach’s alpha = .92 in the scale development sample. Validity: The scale was significantly correlated with the Hess & Rueb abortion scale and the James religiosity scale. The results of a factor analysis yielded two factors.   Availability: The scale is available online in the article at https://scholar.utc.edu/cgi/viewcontent.cgi?article=1339&context=mps Reference for the scale Taylor, M. G. and Whitehead, G. I. (2014) "The measurement of a

  Scale name: Attitudes Toward Abortion Questionnaire Scale overview: The Attitudes Toward Abortion Questionnaire is a 4-item scale of attitudes toward abortion. Author(s) Matthew Feinberg et al. (See references below). Response Type: Items are rated on a 7-point scale of attitudes: 1 = Should definitely be illegal 4 = Unsure 7 = Should definitely be legal Sample item If the woman’s own health is seriously endangered by the pregnancy.   Reliability:   Cronbach’s alpha was . 87 in study 1 of the research article (Feinberg et al. 2014). Validity: Correlations with other scales are included in the research article. Availability: The full set of items can be found in the PsycTESTS reference.   Permission Test content may be reproduced and used for non-commercial research and educational purposes without seeking written permission. Distribution must be controlled, meaning only to the participants engaged in the research or enrolled in the educational a