Repeated Measures ANOVA

Repeated measures ANOVA tests whether there are statistically significant differences in three or more dependent samples. In a dependent sample, the same participants are measured multiple times under different conditions or at different time points.

The one-way analysis of variance with repeated measures is the extension of the t-test for dependent samples for more than two groups.

In the t-test for dependent samples, we examined whether there is a difference between two dependent samples. If we want to test whether there is a difference between more than two dependent samples, we use the analysis of variance with repeated measures.

What are dependent samples?

In a dependent sample, the measured values are connected. For example, if a sample is drawn of people who have knee surgery and these people are interviewed before the surgery and one week and two weeks after the surgery, it is a dependent sample. This is the case because the same person was interviewed at two points in time.

Now, of course, it doesn't have to be about people or points in time, in a generalized way, we can say: In a dependent sample, the same test units are measured several times under different conditions. The test units can be people, animals or cells, for example, and the conditions can be time points or treatments, for example.

Difference of analysis of variance with and without repeated measurements

If three or more independent samples are available, ANOVA without repeated measures is used. But be careful, of course the assumptions have to be checked! More about this later!

Example of repeated measures ANOVA

You might be interested to know whether therapy after a slipped disc has an influence on the patient's perception of pain. For this purpose, you measure the pain perception before the therapy, in the middle of the therapy and at the end of the therapy. Now you want to know if there is a difference between the different times.

So, your independent Variable is time, or therapy progressing over time. Your dependent variable is the pain perception. You now have a history of the pain perception of each person over time and want to know whether the therapy has an influence on the pain perception.

To put it simply, in the left case the therapy has an influence and in the right case the therapy has no influence on the pain sensation. In the course of time, the pain sensation does not change on the right hand case, but it does on the left hand one.

Research question and hypotheses

What is the research question in a repeated measures ANOVA? The research question is: Is there a significant difference between the dependent groups in terms of the mean?

The null and alternative hypotheses result in:

• Null hypothesis: there are no significant differences between the dependent groups.
• Alternative hypothesis: there is a significant difference between the dependent groups.

Assumptions ANOVA with repeated measures

Now we come to the assumptions of ANOVA with repeated measures and finally I will show you how you can easily calculate it online. So what are the assumptions?

• Dependent samples: The samples must be dependent samples.
• Normality: The data should be approximately normally distributed and have metric scale level. This assumption is especially important when the sample size is small. When the sample size is large, ANOVA is somewhat robust to violations of normality.
• Sphericity: The variances of the differences between all combinations of factor levels (time points) should be the same.
• Homogeneity of Variances: The variance in each group should be equal. Levene's test can be used to check this assumption.
• Homogeneity of Covariances (Sphericity): The variances of the differences between all combinations of the different groups should be equal. This assumption can be tested using Mauchly's test of sphericity.
• No significant Outliers: Outliers can have a disproportionate effect on ANOVA, potentially leading to misleading results.

For example, the QQ plot or the Kolmogorov smirnov test can be used to test whether data are normally distributed or not. If the data are not normally distributed, the Friedman test is used.

Whether the assumption of sphericity is violated can be tested using Mauchly's test for sphericity. If the resulting p-value is greater than 0.05, it can be assumed that the variances are equal and the condition is not violated.

If this assumption is violated, adjustments such as Greenhouse-Geisser or Huynh-Feldt can be made.

Results of the repeated measures ANOVA

The analysis of variance with repeated measurement calculates a p-value for your data. This p-value tells you whether there is a significant difference between the repeated measurements.

If the calculated p-value is smaller than the predefined significance level, which is usually 0.05, the null hypothesis is rejected. In this example, the p-value is 0.01, which is less than 0.05. Therefore the null hypothesis is rejected and it can be assumed that there is a difference between the different time points.

Effect size for repeated measures ANOVA

In a repeated measures ANOVA, the effect size can be calculated via the partial eta squared (η²_p). Here, the variance within individuals is related to the variance that cannot be explained, i.e. the error variance or residual.

Bonferroni Post-hoc-Test

As soon as there is a significant difference between the different time points, it is of course also of interest to identify between which exact time points that difference exists. This can be found out with the help of the Bonferroni post-hoc test.

In the Bonferroni post-hoc test in a repeated measures ANOVA, multiple t-tests are calculated for dependent samples. However, the problem with multiple testing is that the so-called alpha error (the false rejection of the null hypothesis) increases with the number of tests. To counteract this, the Bonferroni post-hoc test calculates the obtained p-values times the number of tests.

In the present case, 3 tests were performed, so for the calculation of the Bonferroni post-hoc test, the p-value obtained from the t-test was multiplied by 3 in the background. If one or more p-values are less than 0.05, a significant difference between the two groups is assumed. In this case, we therefore have a significant difference between Before and End and between Middle and End.

Calculate repeated measures ANOVA with DATAtab

ANOVA with repeated measures can be easily calculated with DATAtab. To do this, simply visit the repeated measures ANOVA calculator on DATAtab and copy your own data into the table.

Now you just need to select your variables. If you select three or more metric variables, an analysis of variance with repeated measures is automatically calculated.

And you get the results. You can read the p value in the table and if you don't know exactly how to interpret the results, just click on interpretation in words. In addition, the results are displayed in a boxplot. Finally, the Bonferroni post toc test is calculated.

Calculate a repeated measures ANOVA by hand

How do you calculate an analysis of variance with repeated measures by hand? Here you can find the formulas to calculate an ANOVA. Let's say this is our data. We have 8 people, each of whom we measured at three different points in time (start, middle and end).

First we can calculate the necessary mean values. With the mean values we can calculate the Sum of squares and the Mean Square. Now we can calculate the F value, which is calculated by dividing the mean square of the treatment by the mean square of the residual or error. Finally we can calculate the p value using the F-value and the degrees of freedom from the treatment and error. To calculate the p-Value we use the F-distribution.