IEEE Robotics & Automation Magazine - September 2017 - 128

introduce a number of enhancements and refinements in the
fundamental methods mentioned earlier [68], or apply other
techniques, such as saliency maps [29] or regions of interest
[69], prior to background subtraction.
Despite the wide research on this topic, there are still some
issues to be resolved, such as how to arrange for a training
period with foreground objects in dynamic, real environments; the adaptation to minor dynamic, uncontrolled changes like the passage of time, blinking of a screen, or shadows;
the adaptation to sudden, unexpected changes in illumination; or the differentiation between foreground and background objects in terms of motion and motionless situations.
With the purpose of overcoming these problems, we
proposed a hybrid algorithm based on frame differencing
and background subtraction along with a single-Gaussian
background model and a mechanism for its effective maintenance (which is described in depth in [70]). The underlying idea of this method is to mutually reinforce frame
difference and background subtraction so that the drawbacks of both approaches are overcome while keeping their
original advantages.
In a first stage, an initial background model is built.
Unlike most background estimation algorithms, another
technique for controlling the activity within the system
workspace is performed. As computational and time costs
are critical issues, this control is performed by means of a
combination of difference techniques: frame difference with
reference frame subtraction. Frame difference allows the system to identify objects that have moved from one frame to
the next one. However, it is important to take into account
that both the previous position and the current one are
detected. This problem was solved by using background subtraction because the only highlighted position is the current
one. Note that, as the reference frame is the first taken frame,
it might be possible that it contains objects that are not part
of the background. For that reason, some additional
constraints have been defined to solve this kind of situation.
Furthermore, the used thresholds for those subtraction
approaches are automatically set for each pixel from pixel
neighborhood information. In a similar way, the stationary
object problem has been solved with the combination of
both subtraction techniques. Therefore, there is no danger of
missing foreground objects while the initial model is being
built. Moreover, the obtained background model does not
contain information about those moving targets thanks to
the use of a simple frame-difference approach that detects
moving objects within the robot workspace.
In a second stage, adjacent frame difference, background
subtraction, and background maintenance techniques are
used. The detection and identification of moving objects is
composed of two processes.
1) The adaptive background model, built initially, is used to
classify pixels as foreground or background. This is possible because each pixel belonging to the moving object has
an intensity value that does not fit into the background
model. That is, the used background model associates a
128

IEEE ROBOTICS & AUTOMATION MAGAZINE

SEPTEMBER 2017

Gaussian distribution to each pixel of the image, as defined
by its mean color value and its variance. Then, when an
interest object enters or moves around the system workspace, there will be a difference between the background
model values and the object's pixel values. A criterion
based on stored statistical information is defined to deal
with this classification, and it can be expressed as follows:
b ^r, c h = '

1 if | i ^r, c h 0 otherwise

n r, c | 2

k # v r, c

(1)

where b ^r, ch is the binary value of the pixel at row r and
column c to be calculated, i ^r, c h represents the pixel
brightness in the current frame, n r, c and v r, c are the mean
and standard deviation values estimated by the background model, and k is a constant value that depends on
the point distribution.
2) The second process is the improvement of the raw classification based on the background model as well as detection and adaptation of the background model when a
global change in illumination occurs. The proper combination of subtraction techniques is used to improve the
segmentation carried out at pixel level by using
background subtraction. Furthermore, this difference
processing allows the system to identify global illumination changes. It is assumed that a significant illumination
change has taken place when there is a change in more
pixels than two-thirds of the image size. When an event
of this type occurs, a new adaptive background model is
built because, otherwise, the application would detect
background pixels as targets, as the model is based on
intensity values, and a change in illumination produces a
variation of them.
Once the whole image is processed, those pixels classified
as background are incorporated into the adaptive background
model. For that, the following formulas are used:
Z
] n ^ t + 1h = )^ 1 - ah n r, c ^ t h + ai t + 1 ^ r, ch if background
] r, c
n r, c ^ t h
otherwise
.
[
^
h
a
v
1
r, c ^ t h + ai t + 1 ^ r, c h if background
]] v r, c ^ t + 1h = )
v r,c ^ t h
otherwise
\
(2)
Here, the constant α (0 1 a 1 1) controls the adaptation
rate, and it is given by the number of pixels that are part of the
Gaussian distribution. However, sometimes the pixel gray
level might change quicker than the background model, as
when illumination gradually brightens. As the proposed
updating process is too slow, after a certain period of time, the
background model might not be suitable for foreground pixel
detection. For that reason, a new updating process was
designed. During the updating phase, two different tasks are
carried out.
● The background model is being updated with each new
frame by using (2).

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2017