WSO December 2023 – Issue 3 - 1162

1162
International Journal of Stroke 18(10)
Key messages
1. What is the issue and what we know so far about it?
 Credibility of subgroup effects in clinical trials remains a major challenge because of the perils of subgroups that include
increased risk of false-positive and false-negative conclusions
 This study aims to assess the credibility of reported subgroup analyses in published stroke trials using the Instrument for
Assessing the Credibility of Effect Modification Analyses (ICEMAN) criteria
2. What are the key findings from this study?
 The credibility of reported subgroup analyses in stroke trials remained generally low.
 Most stroke trials still do not report on the justification or direction for effect modification for the selected subgroups
 Many of the studies that were reviewed used the incorrect statistical method to assess subgroup effects
3. What are the implications of the findings?
 More effort is needed to train trialists on the best methods for designing and performing subgroup analyses, and how to
report the results
 Journals need to include the requirements to assess the credibility of reported subgroup effects as part of their reporting
Recommendations
1. Trialists must be better educated on the best methods for conducting subgroup analyses to prevent avoid misleading
results
2. Biological and sample size considerations should take precedence to minimize data dredging
3. Journal editors and professional societies play an important role in improving the credibility of reported subgroup
analyses
4. Checklists, such as the ICEMAN checklist, should be considered necessary for the publication of trial protocols and
subsequent subgroup findings
Introduction
Evaluation of treatment effect heterogeneity is an integral
part of the analysis of clinical trials.1-3 Identifying subgroups
with different treatment effects generate new
hypotheses for future clinical trial research and enables
interventions to be tailored to specific patients. Subgroup
analyses are commonly performed in stroke randomized
controlled trials (RCTs) to evaluate the heterogeneity of
treatment effects.4,5 Reporting the overall treatment effect
without accounting for subgroup effects can be misleading
when patients with different characteristics respond differently
to the same intervention. In addition, incorrectly
reported subgroup analyses lead to wrong conclusions,
which harm clinical policies and practices.6,7
Several studies have drawn the attention of the clinical
trials community to the prevalence and impact of incorrect
reporting of subgroup analysis to support claims of treatment
effect heterogeneity.8-11 Subgroup analysis was incorporated
into the Consolidated Standards of Reporting Trials
(CONSORT) statement in response to these findings.12
Despite the addition of subgroup analysis to the CONSORT
guidelines, selective reporting of significant subgroup analyses,
a lack of prior evidence on potentially relevant subgroups,
and a failure to use appropriate statistical analysis
remain major concerns.13,14
International Journal of Stroke, 18(10)
Many authors have developed guidelines and checklists
to improve the conduct and reporting of subgroup analysis
findings.15-19 However, these checklists vary in the length
and type of criteria included, which creates ambiguity in
their applications. Schandelmaier et al.20 recently created
the Instrument for Assessing the Credibility of Effect
Modification Analyses (ICEMAN) checklist, a shorter
five-item Likert-validated checklist for assessing the credibility
of subgroup analyses in RCTs. The ICEMAN checklist
reduces the number of recommendations and assigns a
credibility rating to each subgroup effect ranging from very
low to high. While Farrokhyar et al.21 mentioned ICEMAN
as a detailed checklist for systematic reviews, Kilpeläinen
et al.22 used the ICEMAN checklist to investigate the credibility
of subgroup effects in urology trials. Until now, the
credibility of subgroup analyses reported in stroke trials has
not been examined. This study aims to assess the credibility
of reported subgroup analyses in published stroke trials.
Methods
Design and registration
This systematic review followed the guidelines outlined
in the Cochrane Handbook of Systematic Reviews of
Interventions.23
The review followed the Preferred
WSO December 2023 – Issue 3

Table of Contents for the Digital Edition of WSO December 2023 – Issue 3
