The Practical Diabetic

Not all medical information is equal, even if it comes from a reliable source. To help me filter the wheat from the chaff, I created these ten levels ranging from Idle Speculation up to verified Medical Fact. Let us get into it and, as is often the case, we have the tl;dr at the end.

Level 1: Idle Speculation

The least reliable medical fact, this is conjecture with literally nothing to back it up. An example might be “I reckon lies cause head colds”.

Level 2: Secondary Source Anecdote

Something someone has heard from somewhere else. The evidence is a “friend of a friend” who had success with the approach. An example might be “Yoga cured my aunt’s diabetes”. Perhaps she was cured, perhaps her management improved. Perhaps she was pre-diabetic or, perhaps, it was gestational diabetes which went away after pregnancy.

Level 3: Primary Source Anecdote

It worked for the person telling you. A good example of this is my “Practical Diabetic Solution”. While, since publishing the article, many people have said things along the lines of “I have a similar approach which works for me”, the fact is, at the time of writing this article, the only person to try the Practical Diabetic Solution is me because I literally wrote about it seven days before writing this article.

Level 4: Multiple, Corroborating Anecdotes

Many people have tried a similar approach and claim to have success. The quotes you see on dodgy supplement sites or on the back of books fall under this category. While the quote may be genuine, sample selection is often biased (when was the last time you saw a bad review on the back of a book?).

Level 5: Observed Under Controlled Conditions

Social experiments often fall under this category; the rules are set and then let to play out to see what happens. The movie “Super Size Me” is a good example of this where the movie’s maker followed a set of rules for engaging with the fast food restaurant, McDonald’s, and monitored his health to see the effects.

Level 6: Observed and Confirmed Independently Under Identical, Controlled Conditions

By Level 6, we are starting to see some rigour in the analysis. An example might be Alcoholics Anonymous (AA) if they released their statistics. As an aside, a Stanford researcher did confirm in 2020 that AA is more effective at keeping people sober than therapy.

Level 7: Published in a Peer-Reviewed Journal

Even when a study is peer-reviewed and published, it can be wrong or misleading. A great example is the Wakefield Vaccine-Autism study published in The Lancet in 1998. With evidence of fraud, the paper was retracted in 2010. Dr. Bernstein’s Diabetes Solution meets this level because of its publication of results in Pediatrics.

Level 8: Published in a Peer-Reviewed Journal, Conducted in a Double-Blind Study and/or with a Control Group (When Ethical/Appropriate)

Let us explain some of these terms by pretending we are testing a new drug. A control group is a group of people, similar in characteristics to the active group who do not receive the drug (or a placebo, explained below). The control group allows us to compare the fates of the control group to the group receiving the drug.

A double-blind study is when the subjects of the study AND the people conducting the study do not know if the subjects are receiving the drug or a fake version (sometimes called a placebo). Why such extreme measures? To remove bias from the experiment and, in the case of a triple-blind study, from the analysis of the data afterwards. The “placebo effect” is probably the most famous form of bias being addressed in this kind of setup.

Generally, if a paper is published following this level of protocol, it likely has medical findings worth further investigation.

Level 9: The Same as Level 8 Plus the Results are Statistically Significant

Claims are often made in science but they do not always have a necessary level of statistical significance to back them up. Understanding p-values and confidence intervals are key in seeing what data are valid and which are not. Often journalists are not well versed in such things and will publish “breakthroughs” where there are none.

Level 10: The Same as Level 9 Plus Verified By Independent Third Parties

Anything meeting this standard can be considered, in my opinion, medicine. One study is compelling but multiple independent studies, being conducted under the strictest of conditions is more compelling. In most developed countries, all vaccines and medications have achieved this level of scrutiny, as a minimum, before being released onto the population.

Using the Levels

Pharmacies (chemists) sell pretty much anything from Level 2 or above e.g. ear candles. Alternative medicine generally gets up to Level 6 or 7 because, beyond that, it starts becoming actual medicine.

This is not to say alternative medicine or diets are bad or wrong, they are just not scientifically proven to the level of other potential treatments. In the absence of other verified treatments or as an adjunct to other therapies, they may be worth considering.

Journalists/mainstream media generally publish “breakthroughs” down to Level 7 but, I would argue it is only in the public interest at Level 9 or 10. Wakefield’s autism claims are the poster child for why this is the case.

Medical research can get to Level 9 and then hit a dead end because it cannot be easily replicated. The famous Rat Park experiments of the late 70s is a good example. While linking environmental conditions to addictive behaviours, the results proved difficult to replicate elsewhere.

In terms of general science, the same ten levels apply although the need for control groups and double blind studies are less important in other science areas. A good example of a Level 9 physics error was the 1989 cold fusion hoax highlighting that science is indeed fallible, at all levels, and before embracing something you read on the internet, ensure it meets the highest possible standard.

tl;dr

Here I present ten levels for assessing medical information. The ten levels are:

• Level 1: Idle Speculation
• Level 2: Secondary Source Anecdote
• Level 3: Primary Source Anecdote
• Level 4: Multiple, Corroborating Anecdotes
• Level 5: Observed Under Controlled Conditions
• Level 6: Observed and Confirmed Independently Under Identical, Controlled Conditions
• Level 7: Published in a Peer-Reviewed Journal
• Level 8: Published in a Peer-Reviewed Journal, Conducted in a Double-Blind Study and/or with a Control Group (When Ethical/Appropriate)
• Level 9: The Same as Level 8 Plus the Results are Statistically Significant
• Level 10: The Same as Level 9 Plus Verified By Independent Third Parties

Using these levels as a guide we can assess the medical information we receive and how reliable it may be.