ReproNim Statistics Module: Reference

Key Points

An introduction to the Statistics in reproducibility module
  • Reproducible analysis is strongly impacted by statistical analyses.

  • Reproducible research requires understanding the notions of sampling, testing, power, model selection.

Statistical basis for neuroimaging analyses: the basics
  • Be familiar with the concept of sampling

  • Know what we call a distribution, a p-value, a confidence interval

  • Have some knowledge of Bayesian statistics and model comparison

  • This is in line with our overall goal of making science (including scientific training) more open.

Effect size and variation of effect sizes in brain imaging
  • Effect sizes come in many forms

  • Significance is not relevance

  • Difference between the raw effect size and the cohen’s d effect size

  • How can the effect size vary? Why is it important to know about this?

  • Effect sizes are under reported, not well understood, and are crucial for our scientific understanding. Let’s fix this.

P-values and their issues
  • A p-value does not give you an idea of the importance of the result

  • A p-value should always be complemented by other information (effect size, confidence interval)

About statistical power
  • The lack of power is much more problematic that it seems at first sight. - It would usually lead to wasted resources - If an under powered study yields some significant effects, these are likely to be overestimated - If an under powered study yields some significant effects, these are less likely to replicate

Cultural and psychological issues
  • A summary of everything so far

FIXME: more reference material.