Showing posts with label F test. Show all posts
Showing posts with label F test. Show all posts

Sunday, July 24, 2022

Post Hoc Tests and Data Analyses

 A post hoc test is a statistical test used to determine if a pair of values are significantly different from each other after the primary analysis has been completed.

The term post hoc is a Latin phrase meaning after the event.

A common use of post hoc tests is the comparison of group means after an F-test in an ANOVA has revealed significant differences among the groups. The reason to test for differences after an overall test like ANOVA is to reduce the risk of finding a significant difference by chance. That is, if researchers perform a large number of tests on a sample, they may find one or more tests significant by chance.

There are many post hoc tests. Following are some examples of tests that compare the means of two groups.

Bonferroni Test

This is a popular test. By dividing the significance level by the number of comparisons, the risk of finding a significant difference by chance is reduced. This procedure is called the Bonferroni Correction.

Tukey's Honest Significant Difference Test (HSD)

The Tukey HSD is a commonly used test, which adjusts for the number of comparisons.

Scheff├ęs Test

Scheff├ęs Test is similar to the Tukey HSD but it is slightly more conservative.

More post hoc tests

Additional post hoc tests are available. I will list them so you can recognize the test as one that evaluates a pair of means for significant differences after an overall test (such as an ANOVA).

Duncan's New Multiple Range Test (MRT)

Dunn's Multiple Comparison Test

Fisher's Least Significant Difference (LSD)

Holm-Bonferroni Procedure


Rodger's Method

Dunnett's correction

Benjamin-Hochberg (BH) procedure

Cite this post

Sutton, G. (2022, July 24). Post hoc tests and data analyses. Assessment, Statistics, and Research. Retrieved from 

Reference for using scales in research:

Creating Surveys 



Reference for clinicians on understanding assessment

Applied Statistics Concepts for Counselors on



Resource Link:  A – Z Test Index

Please check out my website

   and see my books on   AMAZON       or  GOOGLE STORE

Also, consider connecting with me on    FACEBOOK   Geoff W. Sutton    

   TWITTER  @Geoff.W.Sutton    

You can read many published articles at no charge:

  Academia   Geoff W Sutton     ResearchGate   Geoffrey W Sutton 


Tuesday, January 5, 2021

ANOVA in Counseling & Psychology Research

There are several types of ANOVA procedures. The term ANOVA refers to Analysis of Variance. Variance is a statistical term we will review later. Variance refers to differences, so the ANOVA procedures examine differences in scores among groups of people who complete a survey, a test, or produce a scorable response.

For example, an ANOVA can be used to assess the effects of three temperatures on math. The Independent Variable is temperature varies three ways (75, 85, 95 degrees F). The dependent variable is math. The dependent measure of math is a math test.

When there is only one independent variable (IV), the ANOVA is called a one-way ANOVA. If there are two IVs the ANOVA is a two-way ANOVA, and so forth. It is rare to go beyond a four-way because the interpretation of interactions is complicated.

The ANOVA procedure is usually reported with an F value. The larger the F value, the more likely it is that the differences the researchers found are not due to chance.

There may be several independent variables in a project. The effect of each variable is tested with an F test. When there are two or more variables, researchers also test for possible interaction effects, which results in additional F tests for each interaction. Interactions refer to the possibility that two or more variables combine to produce a change in the dependent variable. As with t tests, researchers include a probability (p) value with each F test. A common effect size associated with F tests is partial eta squared. An ANOVA is used when there are one or more independent variables but only one dependent variable.

Independent or grouping Variable = 1 or more IV

Dependent or criterion Variable = 1 DV

Dependent Measure = DM = a test score or quantitative measure of the DV

IV Groups or levels = 2 or more. An IV may have many levels (e.g., temperature, dosages) or groups (e.g., therapy groups, learning groups, work groups).

Overall tests are used to determine significant effects or differences among the groups or levels of the IVs.

An test indicates significance overall and for specific effects or relationships.

A commonly reported measure of effect size is eta squared. In psychology, researchers have been encouraged to focus on effect sizes rather than p values when analyzing and reporting research results.

value reveals the probability of a significant relationship-- one that is not due to chance factors. The level of significance is set by the researchers. A common level is p is .05. F values yielding a probability below 0.05 are commonly considered significant in psychology and education.

The concept or idea is that a difference between means yields a large F value that would only occur 95 times out of a hundred due to chance. When researchers analyze the differences, they are analyzing variance hence, ANOVA.

If the overall F test is significant, then researchers may compare group means two at a time to determine possible significant differences between pairs of groups. There are many tests of pairs. For example t tests, Tukey HSD, Bonferroni, Neuman-Keuls. These tests are called post hoc tests because they are used only if the overall F test is significant.

Read more about ANOVA and data analyses in the following books.

Applied Statistics Concepts for Counselors on AMAZON   or   GOOGLE

Creating Surveys on AMAZON    or   GOOGLE  Worldwide

Illustration of a three group design project. Volunteers in a math class are randomly assigned to three levels of room temperature (Fahrenheit) to determine if room temperature has an effect on the score. The random assignment should mean that individual characteristics of the participants would not be a factor in the score.

(The temperature is in the 90s today and expected to reach the 100s this week!)




75 degrees

Math score

85 degrees

Math score

95 degrees

Math Score

If the researchers worried that randomization did not truly even out the math skills of the participant's then they could give a math  pretest.

The IV is a research design term. In statistics, the IV is often represented by the letter X and the three groups would be X1, X2, and X3.

The DV if often referred to using the letter Y in statistics. In this study, there would be three Y values Y1, Y2 and Y3.

If the overall F value indicated a significant difference for the variation in scores, then the researchers could compare each pair. There would be three comparisons.

Y1 and Y2
Y2 and Y3
Y1 and Y3

What is compared?

In ANOVA, the differences among the mean scores are analyzed.
In the post hoc tests, the differences between the pairs of means are analyzed.

Serious stats help

If you need help with detailed analysis, one of the best statistics books for details is Discovering Statistics by Andy Field. I've used an earlier edition as a course textbook. It is of course not needed for those who desire only a conceptual understanding. See Discovering Statistics on AMAZON for more information.

Other notes
The letter F in the F value comes from the surname of the British statistician, Sir Ronald Fisher (1890-1962) born in my hometown, London, England.

Daniel Fahrenheit of temperature fame was born in Poland to a German family.

Links to Connections


Checkout My Website


See my Books






JOIN me on


   FACEBOOK   Geoff W. Sutton  


   TWITTER  @Geoff.W.Sutton




Read many published articles:


  Academia   Geoff W Sutton   


  ResearchGate   Geoffrey W Sutton 

Structural Equation Modeling (SEM)

  Structural Equation Modeling (SEM) is a statistical technique that is widely used in psychology and related fields to examine the relatio...