Tools for Skeptical Thinking

Forbes Magazine, 2012.  Is it Real or Fake Science?  10 Questions.

 

1. What is the source? Is the person or entity making the claims someone with genuine expertise in what they’re claiming? Are they hawking on behalf of someone else? Do they use a website that’s made to look “sciencey” or newsy when it’s really one giant advertisement for something that is being marketed to you? The scientific studies on healitself.com are taken directly from the U.S. National Library of Medicine National Institutes of Health website, pubmed.gov.  They have been published in peer-reviewed journals and posted for the world to see the results.  These sources are the results of thoughtful and diligent work in using the scientific method to study and find solutions to real health problems.  Other sources are from user testimonials that show that the science really works in real people's lives.

2. What is the agenda? You must know this to consider any information in context. In a scientific paper, look at the funding sources. If you’re reading a non-scientific anything, remain extremely skeptical. What does the person or entity making the claim get out of it? The agenda of healitself.com is to get the information from scientific journals to the public so that people will be healthier and happier.  The developer of this website used his own time and money to study the issues and paid out of his own pocket to get the info on this website, and in over two years of this website being published, has never earned a single penny from this venture.  The scientists just do their jobs and I find their research to show on this site.  The people who have submitted testimonials to this site have done it to tell personal stories, and do not get paid for it.

3. What kind of language is used? Does it use emotion words (miracle cure)? Or use language that sounds highly technical or jargon-y but is really meaningless in the therapeutic or scientific sense? If you’re not sure, take a term and Google it. Be on the lookout for sciencey-ness. If peddlers feel that they have to toss in a bunch of jargony science terms to make you think they’re the real thing, they probably don’t know what they’re talking about.  It can't be helped that scientists use scientific terms in their scientific reports.  It is their job to be accurate.  However, healitself.com has taken the liberty to look up some of the terms and develop an interpretation of many of the studies.  Anybody can do the same.  Nrf-2 activators are not advertised to be a "miracle cure" - they only work to reduce the specific problem of excess oxidative stress, not every human condition.  And the disclaimer should clearly state that it doesn't work on everybody.  Science says it will probably work for you, but then it might not in your case.  

4. Does it involve testimonials? If all the person or entity making the claims has to offer is testimonials without any real evidence of effectiveness or need, be very, very suspicious. Anyone can write a testimonial and put it on a website. If the only thing “showing” effectiveness are testimonials, then you know the science is not there.  Yes, efforts have been made to include testimonials on this website.  This is because most people like to hear interesting stories, not read boring science.  Both are considered important - but testimonials are definitely not the only thing showing effectiveness.  The science is actually the most important part of this website.  Science proves Nrf2 activation works, and real people have real experience showing that is is actually effective. 

5. Are there claims of exclusivity? New findings arise out of existing knowledge and involve the contributions of many people. It’s quite rare that a new therapy is something completely novel without a solid existing scientific background to explain how it works. Watch for words like “proprietary” and “secret.” These terms signal that the intervention has likely not been exposed to the light of scientific critique.  Everybody has known their whole lives and for generations before to "eat their fruits and vegetables."  This is not a new concept.  But science is discovering WHY it's important, and have studied which are the best fruits and vegetables to do certain things to the body, to get it to heal itself better.  Even many drugs are plant-based.  The new findings only add on to the solid science we already know - it is not replacing anything.  This website is helping get the new information out:  scientists know that good foods and herbs cause a healing power to work in your body.

6. Are there mentions of conspiracy? Do they use words like “...only clinicians know how to do this, not those in the ivory towers.” Is there a belief that they are the only ones in the know? Is there a sense they feel like they are being put down or suppressed because of their unique approach?  Nrf2 activators have been advertised on national TV and by famous personalities like Chuck Norris and Donny Osmond, have been sold in health food stores, and are actively advertised word-of-mouth by network marketers of many different companies.  The word is getting out, but it seems like it is going very slowly.  It is hoped that everybody learns about this knowledge about how the body fights its diseases, because a lot of people would get better and not have to take as many of their medicines anymore, which the billion-dollar drug companies would not like to have happen because they would lose a lot of life-long paying customers - now that may be the only conspiracy.  Are the rich drug companies somehow preventing this healing knowledge from getting to the public?

7. Does the claim involve multiple unassociated disorders? Does it involve assertions of widespread therapeutic benefit for unrelated disorders? Claims that a specific intervention will cure cancer, allergies, ADHD, and autism are frankly irrational.  Frankly, if you understand the science, oxidative stress is at the root of many disorders, so the disorders that Nrf2 activators help the body deal with are very rationally associated from a science perspective.  Just ask the many people who had 3 or more disorders, and the disorders all simultaneously went away after taking Nrf2 activators.  In this case, disorders associated with oxidative stress are indeed related.  

8. Is there a money trail or a passionate belief involved? It is always important to follow the money. The ones who benefit financially are those who market cures or therapies, act as consultants, and/or give paid talks. Because of all of our biases and our passion for improvement, we often fail to be as skeptical as necessary.  Of course there's a money trail to every product.  Grocers who sell apples (which by the way are linked to cancer prevention) advertise their apples, and make a profit every time one is sold.  Some people who promote Nrf2 Activators are in network marketing companies, which is the face-to-face distribution method which that company has chosen to explain its product better, and are users of the products themselves, so they also have personal experiences with the product, so they may have a passionate belief by their own positive experiences, and yes they may earn money if they sell it to others.  However, most companies are regular in that they know about the science, develop a  product, and sell it for a retail profit.  Healitself.com does not earn money by telling people about Nrf2 activators, and is not involved in network marketing.  This is simply an informational website, hoping that people will feel better and be happier if they are in better health.  It tells about and lists all companies that sell Nrf2 activators as that info is learned.  

9. Were real scientific processes involved? Is there evidence that the product or intervention on offer has been tested scientifically? Were the results published in scientific journals? Was there true peer-review that is unbiased? Be careful of self-published books, websites, etc.  The scientific articles were all published in scientific journals, and the researchers had to use appropriate scientific processes or their work would have been immediately rejected and not published. The info on this site has been gathered from various sources, and is not self-made.  In fact, the author's own experiences are positive but are not even included in this website at all lest this website be considered to be "biased" in some way.

10. Is there expertiseMake sure that these people have studied the topic deeply.  It does not matter that they have a particular job title, or are employed by a particular establishment.   Remember, just because someone has a PhD does not necessary make them an expert.  The researchers who do these Nrf2 studies are all experts in their field, getting their work published in professional peer-reviewed scientific journals, which is very hard to do even while they are well-educated.  They know their stuff well enough to convince other experts that the data is good. 

 

The CRAAP Test.  (The CRAAP test had its origins in the US to help students successfully evaluate and find reliable information).

C is for Currency: The timeliness of the information.  Is it outdated? When was the information published or posted? Has the information been revised or updated? Will older sources work for your purposes? Are the links functional?  Every month, dozens of new studies appear on the benefits of Nrf2 activation.  Sometimes, information will even appear on pubmed.gov before the physical paper copy of the journal has been sent out to the professional community. Healitself.com cannot keep up with all the influx of studies, so readers are encouraged to go to pubmed.gov themselves, and type in "oxidative stress, (their disorder)."  If studies come up, then chances are that oxidative stress is related to that disorder, and they will have the most current studies that talk about it, usually current within 4 months if it is a major disorder.  We should keep up with what scientists are discovering.  If people wait for it to someday make the evening news, it may be years before they learn about the best ways to treat their disorders.  Reminder: doctors are people too, and may be using old methods of treating disorders if they have not learned the current best practices. 

R is for Relevance: The importance of the information for your needs. Does the information relate to your question? Have you looked at a variety of sources? Would you be comfortable citing this source?  Thousands of researchers from around the world have been working on finding the cure to diabetes, cancer, and a myriad of other conditions afflicting the human population.  Their research is extremely relevant to what is happening to people in the world today. 

A is for Authority: What is the source of the information? Who is the author/publisher /source/sponsor? What are the author’s credentials or organizational affiliations? Is the author qualified to write on this topic? Does the URL reveal anything about the author source? .com .edu .cog. .org .net.    If you go to pubmed.gov, look at the top of every article, which will tell you who the authority is.  Click on their name to find out more about them, such as their degree and where they work.  People who submit testimonials may be considered an expert on their own lives and how something worked for them, but may not be a good consultant for you because their environment, diet, and other conditions are generally less controlled than a real scientific study.  It may be noted that some doctors (whom most people judge as the authority on diseases) can be influenced by companies that provide money or other incentives to keep prescribing products that don't really heal. 

A is for Accuracy: Reliability, truthfulness, correctness of the content. Where does the information come from? Is the information supported by evidence? Has the information been reviewed or refereed? By whom? Can the information be verified in another source? Does the language or tone seem unbiased? Does the author or publisher seem unbiased?   It is best to trust the scientific sources for accuracy.  Testimonials may be good for general effectiveness of a method, but the base effectiveness may be inflated if the teller has a money motive in mind, such as trying to get others to buy the product.  That is why healitself.com tries to provide both scientific studies and testimonials. 

P is for Purpose: Reason the information exists. Why this information? Inform? Sell? Persuade? Do the authors/sponsors make their intentions clear? Is the information fact, opinion, propaganda? Does the point of view appear objective and impartial?    Healitself.com has the purpose of information dissemination.  The intent of this website is to inform, and if possible, pursuade people to live a healthier, happier life by getting their body to heal itself like it's designed to do.  The author does not earn any money as a result of this website.  

 

Sagen, 1996:  Tools for Skeptical Thinking: Baloney Detection..

These ideas can help you remain appropriately skeptical when encountering new therapeutic techniques so you can test and analyze the purported findings.

Independent confirmation: Can other clinicians/researchers come up with the same findings?

Encourage debate on the evidence: There must be open and free dialogue in order for the science of new techniques to be evaluated.

Believe data not “experts”: Don’t let testimonials and non-scientific findings sway you…these may be interesting and may lead us to ask important questions, but arguments from authorities without proper data should be meaningless.

Spin more than one hypothesis: If there are no conceivable reasons for something to work, then it must be questioned if it really does work.

Don’t overly attach to a hypothesis: Believe the research, not the emotions of yourself and others, especially parents.

Quantify the findings: Testimonials cannot be used. We must quantify the results of the techniques and interpret the findings accurately and fairly.

Every link in the argument chain must work: When following the logic of the argument ALL of the pieces must fit together, not just some.

Count the HITS and the MISSES: We cannot overlook the misses and only concentrate on the hits.

A case study is not experimental: A case study cannot and never has been a methodology for explaining cause-effect relationships.

If it is too good to be true, it probably is NOT true: We cannot let our “excitement” dictate over our thinking of the issues.

Follow the scientific methodology.

Be wary of information from the popular press: Only information from peer-reviewed reputable journals can be believed, and then appropriate skepticism must still be applied.