3 Statistical Stuff-Ups That Made Everyday Items Look Healthier (or Riskier) Than They Really Are

Conducting Scientific Studies: The Importance of Accuracy and Transparency

Conducting scientific studies is never easy, and there are often major disasters along the way. A researcher accidentally spills coffee on a keyboard, destroying the data. Or one of the chemicals used in the analysis is contaminated, and the list goes on. However, when we read the results of the study in a scientific paper, it always looks pristine. The study went smoothly with no hiccups, and here are our results.

The Consequences of Statistical Mistakes

But studies can contain errors, not all of which independent experts or “peer reviewers” weed out before publication. Statistical stuff-ups can be difficult to find, as it really takes someone trained in statistics to notice something wrong. When statistical mistakes are made and found, it can have profound impacts on people who may have changed their lifestyle as a result of the flawed study.

Case Studies in Statistical Errors

Here are three examples of inadvertent statistical mistakes that have had major consequences for our health and shopping habits.

1. Did You Throw Out Your Black Plastic Spoons?

Late last year, I came across a news article about how black plastic kitchen utensils were dangerous as they could potentially leak toxic flame-retardant chemicals into your food. Being a natural sceptic, I looked up the original paper which was published in the journal Chemosphere. The article looked genuine, the journal was reputable. So – like perhaps many other people – I threw out my black plastic kitchen utensils and replaced them with silicone ones.

Unfortunately, the authors made a mistake in their calculations. They were out by a factor of ten. This meant the level of toxic chemicals was well under the daily safety limits. In recent weeks, the authors apologised and corrected their paper.

2. Did You Avoid HRT?

A landmark study raised safety concerns about hormone replacement therapy or HRT (now also known as menopausal hormone therapy). This highlights a different type of statistical error.

The Women’s Health Initiative (WHI) study involved 10,739 postmenopausal women aged 50-79 recruited from 40 clinical centres in the US. It compared the health of women randomised to take HRT with those who took the placebo. Neither the researchers nor the women knew which treatment had been given.

The authors reported higher rates of invasive breast cancers in the HRT group. However, they used a unit called “person-years”. Person-years is a way to measure the total time a group of people spends in a study. For example, if 100 people are in a study for one year each, that makes 100 person-years. If someone leaves the trial after only six months, only that half-year is counted for them.

The authors showed a rate of 38 invasive breast cancers per 10,000 person-years in the HRT group, compared to 30 per 10,000 person-years in the placebo group. This gives a rate ratio of 1.26 (one rate divided by the other). However, the rate difference is much more modest – 0.08% breast cancer cases in the HRT group. The authors should have been more careful when describing their results.

3. Did Popeye’s Spinach Change Your Meals?

Cartoon character Popeye is a one-eyed, pipe-smoking sailor with mangled English, in love with the willowy Olive Oyl. He is constantly getting into trouble, and when he needs extra energy, he opens a can of spinach and swallows the contents. His biceps immediately bulge, and off he goes to sort out the problem.

But why does Popeye eat spinach?

The story begins in about 1870, with a German chemist, Erich von Wolf or Emil von Wolff, depending on which version of events you read. He was measuring the amount of iron in different types of leafy vegetables. According to legend, he was writing the iron content of spinach down in a notebook and got the decimal point wrong, writing 35 milligrams instead of 3.5 milligrams per 100 gram serve of spinach. The error was found and corrected in 1937. By then the Popeye character had been created and spinach became incredibly popular with children.

Conclusion

Conducting scientific studies is a complex and challenging process. Statistical errors can have far-reaching consequences, and it is essential to ensure the accuracy and transparency of our results. These case studies demonstrate the importance of attention to detail and careful analysis in scientific research. By recognizing the potential for errors and taking steps to prevent them, we can ensure that our findings are reliable and trustworthy.

FAQs

• What are the consequences of statistical mistakes in scientific studies?
  • They can have far-reaching consequences, affecting people’s lifestyles, health, and well-being.
• Can statistical errors be fixed?
  • Yes, errors can be identified and corrected, but it is essential to do so in a timely and transparent manner.
• How can we prevent statistical errors in the future?
  • By ensuring the accuracy and transparency of our results, using rigorous methods, and peer review to identify and correct errors.
• What can we learn from these case studies?
  • The importance of attention to detail, careful analysis, and transparency in scientific research.