The Statistics Cheat Sheet: Which Test, When
A plain English decision guide to choosing the right statistical test, the assumptions to check first, and what your output actually means.
Start with your question and your data
Choosing a test comes down to three questions: are you comparing groups or looking at a relationship, how many groups or variables are involved, and is your data continuous or categorical? Answer those and the test almost picks itself.
Comparing means
When you are comparing averages between conditions:
- — Two groups, different people: independent samples t-test.
- — Two conditions, same people: paired samples t-test.
- — Three or more groups: ANOVA (one-way, factorial, repeated measures or mixed, depending on design).
- — Non-parametric alternatives when assumptions fail: Mann-Whitney, Wilcoxon, Kruskal-Wallis.
Looking at relationships
When you want to know whether variables move together or predict one another:
- — Two continuous variables, strength of association: Pearson correlation (or Spearman if non-parametric).
- — Predicting one continuous outcome from one or more predictors: linear regression.
- — Predicting a categorical outcome: logistic regression.
- — Nested or repeated data: linear mixed effects models.
Categorical data
When everything is counts or categories:
- — Association between two categorical variables: chi-square test of independence.
- — Goodness of fit against expected frequencies: chi-square goodness of fit.
Check assumptions first
Most marks, and most retractions, come down to assumptions. Before you trust a parametric test, check them.
- — Normality: Shapiro-Wilk and a look at the distribution.
- — Homogeneity of variance: Levene’s test.
- — Sphericity for repeated measures: Mauchly’s test.
- — Linearity, independence of observations and absence of severe multicollinearity for regression.
Read the output honestly
A p-value below .05 means the result is unlikely if the null hypothesis were true; it does not tell you the effect is large or important. Always report an effect size (Cohen’s d, eta squared, r) and, where you can, a confidence interval. A significant result with a tiny effect size is rarely the story it first appears to be.
Want help applying this to your own course or deadline?
Book a free discovery call →