Pharmacy & Therapeutics - December 2008 - (Page 701)

Interpreting Estimates of Treatment Effects compare therapies is helpful, this method has some limitations. Bucher et al. 1 presented an example of comparing sulfamethoxazole–trimethoprim (Bactrim, Women First/Roche) with dapsone/pyrimethamine for preventing Pneumocystis carinii in patients with human immunodeficiency virus (HIV) infection. The indirect comparison using measures of effect magnitude suggested that the former treatment was much better. In contrast, direct comparisons from randomized trials found a much smaller, nonsignificant difference. Song et al.2 examined 44 published meta-analyses that used a measure of effect magnitude to compare treatments indirectly. In most cases, results obtained by indirect comparisons did not differ from results obtained by direct comparisons. However, for three of the 44 comparisons, there were significant differences between the direct and the indirect estimates. Chou et al.3 compared initial highly active antiretroviral therapy (HAART) with a protease inhibitor (PI) against a nonnucleoside reverse transcriptase inhibitor (NNRTI). The investigators conducted a direct meta-analysis of 12 head-tohead comparisons and an indirect meta-analysis of six trials of NNRTI-based HAART and eight trials of PI-based HAART. In the direct meta-analysis, NNRTI-based regimens were better than PI-based regimens for virological suppression. By contrast, the indirect meta-analyses showed that NNRTI-based HAART was worse than PI-based HAART for virological suppression. From these studies, although it seems reasonable to conclude that indirect comparisons usually agree with the results of head-to-head direct comparisons, nevertheless when direct comparisons are lacking, the results of indirect comparisons using measures of effect magnitude should be viewed cautiously. Many variables, including the quality of the study, the nature of the population studied, the setting for the intervention, and the nature of the outcome measure, can affect the apparent efficacy of treatments. If these factors differ between studies, indirect comparisons may be misleading. The magnitude of effect can be computed in a couple of ways: 1. Relative measures express the magnitude of effect in a manner that clearly indicates the relative standings of the two treatments being considered. This method results in comparative statements such as “the improvement rate for treatment X is five times the improvement rate for treatment Y.” 2. Absolute measures express the magnitude of effect without making such comparative statements. Instead, they define a continuous scale of measurement and then place the observed difference on that scale. For example, a simple absolute measure is the difference in improvement rates between two groups. Ideally, both relative and absolute measures would be reported by treatment studies. doubles the probability of a successful outcome. The absolute effect of the treatment depends on the baseline (or control) probability of a successful outcome. If it is low, say 1%, the therapy increases successful outcomes by only one percentage point to 2%, a fairly small increase in absolute terms. In contrast, if the baseline rate of success is 30%, the treatment success rate is 60%, a much large increase in absolute terms. For continuous variables, one simple approach to computing an absolute measure is the weighted mean difference, which is created by pooling results of trials that have used the same outcome measure in a manner that weights the results of each trial by the size of the trial. The weighted mean difference is readily interpretable, because it is on the same scale of measurement as the clinical outcome measure. The problem with using this method is that different trials typically use different outcome measures even when they are focused on the same concept. Standardized Mean Difference and Cohen’s d: Effect Size Measurement The standardized mean difference (SMD) measure of effect is used when studies report efficacy in terms of a continuous measurement, such as a score on a pain-intensity rating scale. The SMD is also known as Cohen’s d.5 The SMD is sometimes used interchangeably with the term “effect size.” Generally, the comparator is a placebo, but a similar calculation can be used if the comparator is an alternative active treatment. The SMD is easy to compute with this formula: SMD = new treatment improvement comparator (placebo) improvement pooled standard deviation (1) The pooled standard deviation (SD) in Equation 1 adjusts the treatment-versus-placebo differences for both the scale and precision of measurement and the size of the population sample used. An SMD of zero means that the new treatment and the placebo have equivalent effects. If improvement is associated with higher scores on the outcome measure, SMDs greater than zero indicate the degree to which treatment is more efficacious than placebo, and SMDs less than zero indicate the degree to which treatment is less efficacious than placebo. If improvement is associated with lower scores on the outcome measure, SMDs lower than zero indicate the degree to which treatment is more efficacious than placebo and SMDs greater than zero indicate the degree to which treatment is less efficacious than placebo. Examination of the numerator of Equation 1 shows that the SMD increases as the difference between treatment and placebo increases, which makes sense. Less intuitive is the meaning of the denominator. Because the SD assesses the precision of measurement, the denominator of Equation 1 indicates that the SMD increases as the precision of measurement increases. The SMD is a point estimate of the effect of a treatment. Calculation of 95% confidence intervals (CIs) for the SMD can facilitate comparison of the effects of different treatments. EFFECT MAGNITUDE: ABSOLUTE MEASURES Although absolute measures of effect seem to be associated with a straightforward interpretation, these measures can be misleading if the baseline rates of outcomes are not taken into account.4 For example, let’s suppose we know that a therapy Vol. 33 No. 12 • December 2008 • P&T® 701

Table of Contents Feed for the Digital Edition of Pharmacy & Therapeutics - December 2008

Pharmacy & Therapeutics - December 2008 Contents Editorial Medication Errors Prescription: Washington New Drugs/Drug News/ New Medical Devices Drug Forecast Interpreting Estimates of Treatment Effects Chemotherapy Foundation Symposium XXVI: Innovative Cancer Therapy for Tomorrow Third Annual Onmark National Payor/Provider Forum Fourth Annual Ophthalmic Drug Development and Delivery Summit Classification and Pharmacological Management Of Obesity CE Test and Forms Season’s Greetings: Thanks to Our Readers and Reviewers

Pharmacy & Therapeutics - December 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.