Is Self-Controlled Practice Actually Advantageous?

📝 Weekly paper summary

Meta-Analytic Findings in the Self-Controlled Motor Learning Literature: Underpowered, Biased, and Lacking Evidential Value (McKay et al., 2021)

Category

Meta Analysis (Pre-Print; still needs to be peer-reviewed)

Context

In recent years, theories such as OPTIMAL have advocated for self-controoled practice structures and feedback schedules to enhance learning rather than pre-planned (i.e., yoked) structures. The first experiments examining this practice date back to the mid 1990's, but really picked up again around the 2010's. However, some researchers have questioned the validity of these reported findings, citing biased study designs and publication bias. Therefore, this review's purpose was to critically examine the validity of the self-controlled learning literature

Correctness

I think the authors did a very thorough job with their statistical analyses, which is the main strength of this paper. If I'm being honest, I don't have a strong enough grasp on the theoretical developments in this space to provide anything other than praise for the way the article was put together. I think this is a very good read for anyone interested in this space.

Contributions

I strongly recommend reading this paper yourself, but here are some of the key highlights:

• The authors found a high risk of bias in the study designs published
• Of the published studies, the authors detecting a high risk of publication bias (mainly, published experiments showed a moderate-large effect size in favour of self-controlled practice, but non-published studies showed a near zero effect size. On aggregate, the effect size is very small and statistically indistinguishable from zero).

Here are two quotes I found quite interesting from the authors:

"The effectiveness of self-control was not moderated by choice-type, suggesting that self-controlled practice may be ineffective regardless of the nature of the choices provided. Indeed, the only factor we tested that moderated the effect of self-controlled practice was publication status."

This is quite damning if the theoretically relevant factors did not "hold up" to more rigorous analyses, but publication status was a strong statistical predictor for the success of self-controlled practice.

"the substantial self-controlled learning literature is, as of now, insufficient to provide evidence that self-controlled practice is more effective than a yoked practice."

🧠 Fun fact of the week

I watched Top Gun: Maverick this weekend, so the fun fact is centered around that.  I found it incredible that it costed the movie about 11K USD per hour to rent the fighter jets. This sounds crazy expensive until you realize that many military aircrafts cost in excess of 30K USD per hour to fly, largely due to the maintenance costs of keeping these aircraft operational! So perhaps the movie got a decent price for their rentals :)

🎙 Podcast recommendation

I haven't listend to this podcast yet, but I'm always curious to see the similarities and differences in how people in other domains (e.g., investing) think about identifying talent.

🗣 Quote of the week

"Smart people learn from everything and everyone, average people from their experiences, stupid people already have all the answers."

- Socrates