"Things I [Jacob Cohen] Have Learned", your nostrils, and avoiding the path of least resistance

Steven Hirsch -

📝 Weekly paper summary

This week, I want to summarise a classic editorial of Jacob Cohen, a renowned statistician.

Things I have Learned (So Far) (1990)

Category

Editorial

Contributions

  • Some things you learn aren't so
    • Cohen talks about some of the issues with small (at the time, a sample of <30) and large sample statistics. The individual anecdote is entertaining to read, but I appreciated the example he provided. A two-independent-group-mean comparison with n=30 per group that a medium-sized effect would be labelled as significant is about 50%. Of course, reporting effect sizes now is mandated by most journals, but sometimes (at least for me), it's easy to forget just how important they might be when interpreting data from experiments.
  • Less is more
    • Although biomechanists have heeded the advice of statisticians regarding effect sizes, one area we haven't is with reporting too many independent variables. It's easy to compute endless variables with current software, but as Cohen pointed out, it becomes very easy to end up with spuriously significant results, which leaves us wondering which ones are actually significant. In addition to other regression-related issues that so many variables cause, another "less is more" scenario that we (i.e., I) often forgot is the number of decimal places to include when reporting data (usually, we report way too many and should consider not only the precision of the measurement equipment but also how many decimal places are required to tell the story for your data).
  • Simple is better
    • This advice is something I think we know somewhat implicitly. However, although Cohen's specific advice from this section is less relevant today (and pertains to leveraging graphics software to present data rather than strictly reporting the moments of distributions), one piece of pertinent advice is when simple is not actually better. For example, it is common in many applied biomechanics research to dichotomize the data to simplify it. However, the discarding of information can reduce a variable's squared correlation (i.e., explained variance) by about 36%! So, yes, simple is better, but don't get carried away and lack rigour!
  • The Fischerian Legacy
    • I recommend reading this section (it's quite short, but very sweet). Mainly, Cohen has great insight regarding the history of p-values and how they are often well suited for making decisions, but the development of scientific theories is not necessarily so decision-oriented.
  • The Null Hypothesis Tests Us
    • Cohen brings up an issue that researchers (including myself, go check out my papers to see where I've made this mistake) often report that there are "no differences between groups" or "no relationship between variables" when p>0.05. This wording is problematic since the p-value itself has zero predictive capability for telling us whether we might reject or accept the null again in a future replication of an experiment. As a reminder, a p-value does not tell us how likely the null hypothesis is to be true (we need Bayesian statistics for this), but rather the probability of the observed data given the assumption the null hypothesis is true. Furthermore, another issue with the null hypothesis is that it is always false (because any difference other than 0, which can include 10^(-1000000), is enough to reject the hypothesis. Finally, the best quote from this section is as follows:

"...hypothesis testing has been greatly overemphasized in psychology and in the other disciplines that use it. It has diverted our attention from crucial issues.Mesmerized by a single all-purpose, mechanized, "objective" ritual in which we convert numbers into other numbers and get a yes-no answer, we have come to neglect close scrutiny of where the numbers came from."

  • How to use statistics
    • plan the research in advance (sample size, dependent variables)
    • researchers should contextualize their findings with effect sizes, not solely p-values
    • one of the most important parts of the scientific process is the judgements made by the scientists, which shouldn't rest solely on the p-value.

🧠 Fun fact of the week

Perhaps this is a weird one, but your nostrils work one at a time! The process of nasal cycling (which has a duration of about 2.5 hours) allows for alternation between congestion and decongestion. This process is advantageous since the cilia on the congested side suspend their motility until it decongests, ensuring that one nostril is always available to humidify incoming air. Further, it benefits our sense of smell since some odours need more time, and others less, to bind to olfactory receptor cells. Therefore, cycling the airflow rate between the two nostrils helps us detect a more extensive range of odours. Neat!

🎙 Podcast recommendation

This podcast was released a couple of weeks ago now, but I enjoyed the overall discussion and thought you will too!

🗣 Quote of the week

"I thought I'd solved a problem when really I was creating new ones by taking the path of least resistance"

- David Goggins