Mistakes in ... Booklet 2020 - 34

ueg education

used in this regard, though other methods
also exist.20
Of note, certain projects have more than one
outcome. In these cases, the primary outcome
and any secondary outcomes should be clearly
predefined, and the analysis should be performed
accordingly. Finally, important discoveries can still
be made outside those predicted in the analysis
plan or even by the original question. In these
cases, it is important to realize whether the
findings are indeed a true signal or an artefact
from multiple comparisons. Additional
confirmatory studies, focusing on the new
question, may be needed.

Mistake 6 Conducting the study on the
wrong number of participants
Fundamental to conducting a good study is the
awareness of sample size (or study size).21 If an
effect truly exists then the study should be
performed on a sufficiently large sample to detect
it. Too small a sample size can also increase the
likelihood of generating a false-positive effect
because smaller sample sizes can be more prone
to certain biases (see mistake 7).
What the sample size should be for any
particular study depends on a number of
parameters. There are several pieces of
knowledge that are fundamental to all sample
size calculations (Figure 3).
The first consideration is the prevalence of the
disease in the unexposed population, which may
be known from previously conducted research or
from other similar populations. There is a level
of guestimation involved in determining this
parameter, but basing it on prior knowledge is, of
course, beneficial. The second factor is the
minimum effect size, that is, the clinically
relevant difference that we wish to detect (e.g.
a reduction of bilirubin levels of 1mg/dL or
0.1mg/dL). The smaller the difference to be
detected, the larger the sample size required.
The third parameter is the significance level (or
alpha value) of the study, which is the probability
of rejecting a null hypothesis given that it is true
(type-I error) and is frequently set at 5%. The final
component when calculating sample size is the
power of a study, the probability that it will detect
an effect when there is one to be detected-power
is frequently set at 80% or 90%.
With these parameters known to us or chosen,
the required sample size can be calculated.
Other factors that may need to be taken into
consideration include the ratio of controls to
cases and the likelihood of loss to follow-up in
the patient population. The more complicated
the study design, the more complicated the
calculation of sample size becomes. Whilst there
are several statistical packages available that
allow the calculation of sample sizes (some
opensource and others proprietary),22 as with so
many aspects of study design it is best to ensure

Mistakes in... 2020

What is the prevalence
of the disease
in the unexposed

What is the


What is the minimum
effect size you
wish to be
able to detect?

What is the

Figure 3 | Sample size considerations.

you have the correct skills mix and support for
your project from the outset. Consulting or
collaborating with a statistician at this stage
of project development can be extremely
In practice, sample size considerations are
often based as much, if not more, on resource
implications as they are on the purely
mathematical components mentioned above.
If we know there is a finite number of cases, an
adjustment to other parameters will have to be
justified to allow the study to go ahead, such
as increasing the numbers of controls to cases,
or increasing the effect size to be detected.
Conducting a study on too many cases (perhaps
by wishing to detect a very small effect size) may
be considered a waste of resources and,
therefore, the study runs a higher risk of not
being funded (see mistake 8).

Mistake 7 Not considering potential
sources of bias from the outset
The avoidance of bias is one of the cornerstones
of good epidemiologic study design. Bias has
been descrided as "... the deviation of results or
inferences from the truth or processes which lead
to such deviation."23 There are a huge number of
defined biases and it is not possible to discuss
every single one of them here. Broadly, bias in
epidemiological studies can be categorized into
selection bias and misclassification (information
bias; see table 1).
Selection bias arises when there is a systematic
difference in the characteristics of people within
a study compared with those who are not. This
can include differences or errors in those enrolled
in the study in the first place compared with the
target population (sampling bias), through the
inclusion of an inappropriate control population
(e.g. choosing hospital controls who have a
particular condition), and through following
up participants differently dependent on
their exposure status (attrition bias or
follow-up bias).
Misclassification is a deviation in the
measurement of one or more factors of interest
in a study. Misclassification may affect all groups
equally (nondifferential misclassification) or,
perhaps more frequently, it may be more likely to
occur in one group than another (differential
misclassification, or information bias).

Information bias includes observer bias (where
a study team member differentially records
observations based on known exposure or
outcome status), recall bias (where a participant
is more or less likely to remember accurately an
exposure based on their outcome status) and
reporting bias (where a participant is more
or less likely to reveal or suppress certain
information, particularly when there may be
stigma attached to it [e.g. alcohol use]).
We also include confounding in this section.
There is debate as to whether confounding is
strictly speaking a type of bias because it does
not arise from a systematic error in the design of a
piece of research, but rather from the nonrandom
distribution of risk factors in the population being
studied. A confounder is a third variable that is
associated both with the disease and exposure of
interest in any given study. If a confounding
variable has been measured then it is usually
possible to adjust for it in the analytical stages of
the study through stratification or modelling. The
study design can also be adjusted to minimize or
remove confounding by restricting the study
population or matching the population based on
certain known (or suspected) confounding
factors. If this is not possible or appropriate,
careful thought as to the likely confounding
variables in any study will ensure that appropriate
data have been collected to allow adjustment at
the analysis stage.

Mistake 8 Undertaking unfeasible projects
Resources for research are often limited,
especially for those in the early stages of their
career. Research that requires recruitment of
patients is often orders of magnitude more
expensive than research that can be performed
on data collected via other mechanisms. Of
course, although funding is a crucial point, it is
not the only one, and once a research question
has been framed, and a protocol has been
finalized, it is important to evaluate whether the
proposed project is feasible or not.
At times, a protocol that on paper seems
perfectly tailored to answer a research question is
not practicable and will be impossible to execute
in the real world. In such cases, the protocol and
research question should be revisited, and a more
practical approach considered. Although this may
be frustrating because plenty of time and work
will have already been invested in the project, it is
vital to recognize unfeasible projects, and identify
them as 'nonstarters'. This course of action may
ultimately be more helpful to the researcher's
professional development than the alternative of
starting a neverending project.
Many factors may render a project
unfeasible. Insufficient protected time for
carrying out the research, and an insufficiently
framed research question are pitfalls that
early-stage researchers should avoid. Finding


Mistakes in ... Booklet 2020

Table of Contents for the Digital Edition of Mistakes in ... Booklet 2020

Mistakes in ... Booklet 2020 - 1
Mistakes in ... Booklet 2020 - 2
Mistakes in ... Booklet 2020 - 3
Mistakes in ... Booklet 2020 - 4
Mistakes in ... Booklet 2020 - 5
Mistakes in ... Booklet 2020 - 6
Mistakes in ... Booklet 2020 - 7
Mistakes in ... Booklet 2020 - 8
Mistakes in ... Booklet 2020 - 9
Mistakes in ... Booklet 2020 - 10
Mistakes in ... Booklet 2020 - 11
Mistakes in ... Booklet 2020 - 12
Mistakes in ... Booklet 2020 - 13
Mistakes in ... Booklet 2020 - 14
Mistakes in ... Booklet 2020 - 15
Mistakes in ... Booklet 2020 - 16
Mistakes in ... Booklet 2020 - 17
Mistakes in ... Booklet 2020 - 18
Mistakes in ... Booklet 2020 - 19
Mistakes in ... Booklet 2020 - 20
Mistakes in ... Booklet 2020 - 21
Mistakes in ... Booklet 2020 - 22
Mistakes in ... Booklet 2020 - 23
Mistakes in ... Booklet 2020 - 24
Mistakes in ... Booklet 2020 - 25
Mistakes in ... Booklet 2020 - 26
Mistakes in ... Booklet 2020 - 27
Mistakes in ... Booklet 2020 - 28
Mistakes in ... Booklet 2020 - 29
Mistakes in ... Booklet 2020 - 30
Mistakes in ... Booklet 2020 - 31
Mistakes in ... Booklet 2020 - 32
Mistakes in ... Booklet 2020 - 33
Mistakes in ... Booklet 2020 - 34
Mistakes in ... Booklet 2020 - 35
Mistakes in ... Booklet 2020 - 36
Mistakes in ... Booklet 2020 - 37
Mistakes in ... Booklet 2020 - 38
Mistakes in ... Booklet 2020 - 39