Pharmacy & Therapeutics - December 2008 - (Page 700)

Interpreting Estimates of Treatment Effects Implications for Managed Care Stephen V. Faraone, PhD INTRODUCTION How should health care professionals choose among the many therapies claimed to be efficacious for treating specific disorders? The practice of evidence-based medicine provides an answer. Advocates of this approach urge health care professionals to base treatment choices on the best evidence from systematic research on both the efficacy and adverse effects of various therapeutic alternatives. Ideally, health care professionals would compare different treatments by referring to randomized, double-blind, head-to-head trials that compared the treatment options. Although individual medications are typically well researched when these placebo-controlled studies are performed, studies that directly compare treatments are rare. In the absence of direct head-to-head trials, other evidence comes from indirect comparisons of two or more therapies by examining individual studies involving each treatment. This article provides an introductory review of methods of such indirect comparisons of therapies across studies, provides examples of how these methods can be used to make treatment decisions, and presents a general overview of relevant issues and statistics for readers interested in understanding these methods more thoroughly. STATISTICAL SIGNIFICANCE VERSUS MAGNITUDE OF EFFECT Before one considers the meaning of a treatment effect, it is necessary to document that the effect is “statistically significant” (i.e., the effect observed in a clinical trial is greater than what would be expected by chance). If a treatment effect is not larger than that expected by chance, the “magnitude of effect” computed from the trial is questionable if one is making comparative therapeutic choices. Sometimes a small trial suggests a large benefit, but the result might not be statistically significant because the study is underpowered. In that case, the apparent large benefit should be viewed cautiously and should be considered when one is designing future studies aimed at replicating the finding. When the results of clinical trials are statistically significant, treatment choices should not be made based on comparisons of statistical significance, because the magnitude of statistical significance is heavily influenced by the number of patients studied. Therefore, a small trial of a highly effective Dr. Faraone is Director of the Medical Genetics Research Center and of Child and Adolescent Psychiatric Research at SUNY (State University of New York) Upstate Medical University in Syracuse, New York. He is also a Professor of Psychiatry in the Department of Psychiatry and Behavioral Sciences. therapy could have a statistically significant result that is smaller than a result from a large trial of a modestly effective treatment. Although the results of statistical analyses provide crucial information, the magnitude of statistical significance does not necessarily indicate the magnitude of the treatment effect. As such, it is impossible to determine from the degree of statistical significance how, for example, a novel therapy evaluated in one study compares with the efficacy of other established or emerging treatments for the same condition. This problem of interpretation of statistical significance can be addressed if we use the concept of magnitude of effect, which was developed to allow clinically meaningful comparisons of efficacy between clinical trials. The magnitude of an effect can help clinicians and P&T committee members decide whether the often modest increases in efficacy of newer therapies are important enough to warrant clinical or formulary changes. One way to make these determinations is to examine acceptable effects of widely recognized therapies for specific disorders. If this approach is not taken, comparing two clinical trials can be difficult. As the name suggests, an effect magnitude estimate places an interpretable value on the direction and magnitude of an effect of a treatment. This measure of effect can then be used to compare the efficacy of the therapy in question with similarly computed measures of effect of a treatment’s efficacy in other studies that use seemingly noncomparable measures. When indirect comparisons are conducted, measures of effect magnitude are essential in order to make sensible evaluations. For example, if one study measured the efficacy of a therapy for back pain using a five-point rating scale for pain intensity and another study used a 10-point rating scale, we could not compare the results, because a one-point decrease has a different meaning for each scale. Even if two studies use the same measure, we cannot simply compare changed scores between treatment and placebo, because these studies may differ in their standards for precision of measurement. These problems of differing scales of measurement and differences in precision of measurement make it difficult to compare studies. Fortunately, these problems can be overcome if we use estimates of effect magnitude, which provide the difference in improvement between therapy and placebo, adjusted for the problems that make the statistical significance level a poor indicator of treatment efficacy. Although using measures of effect magnitude to indirectly Disclosure: Dr. Faraone has acted as a consultant to and has received research support from McNeil, Pfizer, Shire, Eli Lilly, and Novartis. He has also served as a speaker for or has been on the advisory boards of these companies. This article was supported by funding from Shire Development, Inc. Accepted for publication September 22, 2008. 700 P&T® • December 2008 • Vol. 33 No. 12

Table of Contents Feed for the Digital Edition of Pharmacy & Therapeutics - December 2008

Pharmacy & Therapeutics - December 2008 Contents Editorial Medication Errors Prescription: Washington New Drugs/Drug News/ New Medical Devices Drug Forecast Interpreting Estimates of Treatment Effects Chemotherapy Foundation Symposium XXVI: Innovative Cancer Therapy for Tomorrow Third Annual Onmark National Payor/Provider Forum Fourth Annual Ophthalmic Drug Development and Delivery Summit Classification and Pharmacological Management Of Obesity CE Test and Forms Season’s Greetings: Thanks to Our Readers and Reviewers

Pharmacy & Therapeutics - December 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.