We’ve all seen and heard of supplements touting that their health claims are backed by “scientific” or “clinical study.” In the consumer mind, these terms elicit trust and a notion that the product must be safe and effective. In truth, this is not always the case. At LabDoor, we encourage you to be more critical of the “clinical studies” in question. Let me be clear—some products have gone through high quality clinical trials and likely do have the safety and efficacy profiles they boast. Other products, however, may be supported to a larger extent by manufacturer marketing than by science.
The danger of marketing is an issue of exaggeration. Results of low-quality testing, a disregard for conflicting results (or author’s conflicts of interest), and skimming potentially serious side-effects may often be replaced with big, but controversial, health claims. So, then, short of reading the Materials and Methods section of every paper and analyzing the experiment’s study design, what can you believe?
The most informative way of assessing the efficacy and quality of multivitamins, just as any other medicine, is to conduct well-designed clinical trials – ones that account for the placebo effect, reduce allocation bias, and effectively isolate variability down to the treatment being administered (in this review, we focus on multivitamin supplementation). In an effort to help you make more informed decisions about the supplements you choose to consume, we provide a basic guide to the qualities of a well-designed clinical study—and, importantly, those whose safety and efficacy claims we should be able to trust.
The Basics
Length of Study: Short-term studies may not always provide definitive conclusions. These studies often neglect to answer important questions, such as how long supplementation should be maintained for noticeable benefits or how long it takes for adverse effects to present. Multivitamin intake may take years to show possible health effects, whether they be beneficial or adverse. One criterion for higher quality study (and their claims’ validity) should be reserved for longer length of the study.
Size of Study: The effects seen in a small group of people of a single race provide information only for that group of people, and possibly for people sharing that specific racial profile (i.e. similar genetic makeup) but not for populations at large. Large, multiethnic studies assessing thousands of people would provide more meaningful and accurate information regarding a supplements beneficial and/or harmful effects than would small-scale studies.
Keys to Quality
In an ideal clinical trial, the effects of the drug in question (called the intervention) are isolated—in an effort to guarantee that any effects seen at the study’s end are due to the drug alone. Variability from any source outside of the intervention needs to be effectively eliminated. In practice, however, every clinical trial is subject to some degree of variability, but some clinical design best practices are encouraged to minimize variability and maximize quality.
Placebo-Controlled: Placebo—you may have heard this referred to as a “sugar pill”—is a treatment specifically designed to have no true effects on health. The treatment is given to the study’s control group and meant to account for and eliminate the placebo-effect, a percieved effect not due to the treatment in question.
Randomized: Randomizations refers to the process of assigning study participants to various groups in the study. Study groups are generally separated into two broad categories: those receiving the actual treatment and those receiving placebo. Those receiving treatment—the so called intervention group—may be further split into groups receiving varying dosages of the drug in question. Randomization should ideally give every participant an equal chance of being assigned to any of the study’s groups.
The primary purpose of randomization is to avoid something called allocation bias—a situation in which either the researcher and/or study participants know which treatment they are receiving. This may influence researcher/participant behavior (and even thought) during the trial, lead to false-positive reports (those not due to the treatment alone), and effectively invalidate the results.
Double-Blind: Double-blinding refers to a process in which neither the researcher nor the study participants know which of the treatments—either the intervention drug or placebo—are being administered. The purpose of double-blinding during clinical trials, like randomization, is to minimize bias and attempt to accurately determine of effects of the intervention drug alone.
Types of Study
Trials may be observational, prospective, or controlled and randomized in nature. These studies are built upon fundamentally different designs—some may not be placebo-controlled, randomized, and/or double-blind. The validity of results is similarly dependent on design.
Observational studies simply draw associations between a certain treatment and an effect seen in the test subjects, but do not prove a cause-effect relationship. In large part, this is because test subject in observational analysis are not compared against controls.
According to Richard Nahin, Senior Advisor for Scientific Coordination and Outreach at the National Center for Complementary and Alternative Medicine (NCAAM), observational studies “cannot provide definitive evidence of safety, efficacy, or effectiveness, [but] they can:
- provide information on ‘real world’ use and practice
- detect signals about the benefits and risks of complementary therapies use in the general population
- help formulate hypotheses to be tested in subsequent experiments
- provide part of the community-level data needed to design more informative pragmatic clinical trials
- inform clinical practice.”
Prospective studies simply assess the rate at which risk factors emerge, as well as their magnitude, after intervention was started. In this study design, all subjects are healthy at the beginning of the trial, are followed for, often, long periods of time, and assessed for presence of designated health outcomes frequently throughout the study’s duration. These test subjects are then compared to controls, individuals not affected by the intervention of choice.
These types of studies provide more meaningful information than do observational studies, but may still feature significant bias.
A randomized controlled trial (RCT) is widely considered the gold-standard for clinical trials. In this study design, subjects are first assessed for eligibility and then randomized into either the treatment or placebo group. Post-randomization, both groups are followed identically (procedures, tests, visits, follow-up calls) with the purpose of isolating the effects of the treatment under study as the only variable between the groups.
Conclusion
Understanding the differences and quality of the research experiments on which important claims are made becomes an invaluable tool in assessing the validity of those claims. As a note, scientific research on the health effects of multivitamins is still ongoing. A greater number of high-quality clinical trials will provide more definitive answers.
- Header Image: r2hox (Flickr)
- Assessing the Risks of Bias – Cochrane Bias Methods Group
- How To Randomize – Journal of the Society for Integrative Oncology
- Nahin R. Observational Studies and Secondary Data Analyses to Assess Outcomes in Complementary and Integrative Health Care. National Center for Complementary and Alternative Medicine. (2012)
- Bookwala A, et al. The three-minute appraisal of a prospective cohort study. Indian Journal of Orthopaedics. (2011)
- Moher D, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomized trials. British Medical Journal. (2010)