Essential 10

10. Results If applicable, the effect size with a confidence interval. explanation

For each experiment conducted, including independent replications, report:

10a Summary/descriptive statistics for each experimental group, with a measure of variability where applicable (e.g. mean and SD, or median and range).
10b If applicable, the effect size with a confidence interval.
Explanation
Examples

In hypothesis-testing studies using inferential statistics, investigators frequently confuse statistical significance and small p-values, with biological or clinical importance [1]. Statistical significance is usually quantified and evaluated against a preassigned threshold, with p < 0.05 often used as a convention. However, statistical significance is heavily influenced by sample size and variation in the data (see item 2 – Sample size). Investigators must consider the size of the effect that was observed and whether this is a biologically relevant change.

Effect sizes are often not reported in animal research, but they are relevant to both exploratory and hypothesis-testing studies. An effect size is a quantitative measure that estimates the magnitude of differences between groups, or strength of relationships between variables. It can be used to assess the patterns in the data collected and make inferences about the wider population from which the sample came. The confidence interval for the effect indicates how precisely the effect has been estimated, and tells the reader about the strength of the effect [2]. In studies where statistical power is low, and/or hypothesis-testing is inappropriate, providing the effect size and confidence interval indicates how small or large an effect might really be, so a reader can judge the biological significance of the data [3,4]. Reporting effect sizes with confidence intervals also facilitates extraction of useful data for systematic review and meta-analysis. Where multiple independent studies included in a meta-analysis show quantitatively similar effects, even if each is statistically non-significant, this provides powerful evidence that a relationship is ‘real’, although small.

Report all analyses performed, even those providing non-statistically significant results. Report the effect size to indicate the size of the difference between groups in the study, with a confidence interval to indicate the precision of the effect size estimate.

 

References

  1. Wasserstein RL, Schirm AL and Lazar NA (2019). Moving to a World Beyond “p < 0.05”. The American Statistician. doi: 10.1080/00031305.2019.1583913
  2. Altman DG (2005). Why we need confidence intervals. World J Surg. doi: 10.1007/s00268-005-7911-0
  3. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M and Altman DG (2010). CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. doi: 10.1136/bmj.c869
  4. Nakagawa S and Cuthill IC (2007). Effect size, confidence interval and statistical significance: a practical guide for biologists. Biological reviews of the Cambridge Philosophical Society. doi: 10.1111/j.1469-185X.2007.00027.x