In Broadcast - November 2020 - 48

www.inbroadcast.com | Vol: 10 - Issue 11 | November 2020

InUSA

Video Processing
Inspired By The
Human Brain

By
Michael Grotticelli...

lthough you might not realize it, making closer to the data
there have been many applications source. This reduces delay
in video (and movie) production where and latency and improves
"deep learning" has been applied, that near- to real-time results.
is, now that the media industry has a Since the advent of
better understanding of what deep IoT, there has been an
learning actually means.
enormous increase in
Basically, deep learning is a subset the number of connected
of machine learning where artificial devices on a media
neural networks - that is, algorithms processing network. This
inspired by the human brain - learn number is sure to increase
commands from large amounts of going forward. Due to
data. One of the more complex ways of the current challenges
doing this is by mimicking the neurons faced in connecting a very
in a human brain. A neural network large number of devices to the cloud,
is also a type of machine learning media organizations are now more
that enables a computer to learn open to using EI - mainly because they
and improve upon the task at hand. understand it better than they did
When these artificial brains, or neural previously.
networks, are made more complex to
Many people draw comparisons
handle massive processing, we call between EI and edge computing.
that deep learning.
However, edge computing relates to
Many are calling it the new frontier computing at the edge - leveraging
for the video industry, as it allows gateways with minimal processing video professionals to do things but edge intelligence goes one step
automatically that would have taken ahead with edge devices that can truly
weeks of work in the past. It can also work independently.
complete tasks that wouldn't have
Deep learning can also be used to
been possible at all. For example, match generated speech with human
it allows a computer to take all the speech, so text-to-speech programs
pixels in a frame of video and output sound more natural. In a similar task,
something
equally
complex,
such as all the pixels in a new,
altered, frame of video. It may
be shown frames with unwanted
grain as input, and have its output
compared to clean frames. By trial
and error, it learns how to remove
the grain from the input. As more
and more images are passed
through it, it can learn how to do
the same thing for images that it
was never shown.
Another element of deep
learning is called Edge Intelligence,
(EI), which is used to change the
way data is acquired, stored and
extracted by shifting the process
from the storage devices in the
cloud to the edge (e.g. a camera). As a part of machine learning, Deep learning
is an automatic algorithm that mimics human
EI makes edge devices independent perception and is the technique that comes
of the network by moving decision- closest to the way humans think

Yet another problem
that can be solved with
deep learning algorithms
is missing or damaged
frames from a video.
You can't do reshoots on
something that happened
years ago. Restoring that
type of movie before
meant editing around
the missing frames. Now,
deep learning networks
are being deployed to
change that. Google has
The use of deep learning in film and broadcast developed a technology
has only begun to touch the surface of what is possible
that
can
realistically
it is used by translation companies to recreate part of a scene based on start
teach computers how to translate from and end frames.
one language to another.
By detecting the faces of everyone in
In Hollywood, Warner Bros. movie a video, deep learning can allow you
studio spend US$25M on reshoots to search for any clip or movie that
for its latest instalment of "Justice has a given performer. Alternatively,
League" and part of that money went you could use the technology to count
to digitally removing a moustache on the exact screen time for every actor
its star actor. Deep learning is being in a video. Sky News recently used
called a game changer for these are facial recognition, powered by AWS, to
types of highly detailed tasks.
identify famous faces during coverage
Video restoration is another of Harry and Meghan's royal wedding.
application for deep learning. The
However, the technology is not
UCLA Film & Television Archive said limited to detecting just faces. Sports
that nearly half of all films produced broadcasts now rely on camera
prior to 1950 have disappeared. operators to track the movements
And 90 percent of the classic film of the ball, or to identify other key
prints that do exist are in poor elements of the game, such as the
condition. The process of restoring goal. Using object recognition, AIthese films is long, tedious, and powered tools can be also used to
expensive. Deep learning is now automate the production of a sports
helping to bring those films back to broadcast - such as calling shot angles
their original glory.
during a telecast as accurately as a
The process of colorizing black human director sitting in an OB Van
and white footage has always been outside a stadium would.
lengthy. There are thousands of
Going forward, the use of deep
frames of footage in a movie and learning in film and broadcast has only
coloring each one takes a long begun to touch the surface of what is
time. Even with advanced tools, possible. As with any new technology,
the process can only be automated the video industry and tech experts
so much. Deep learning can now must come together to develop the
speed up the process significantly, standards of how tomorrow's new
with tools that only require an normal might look. However, with
artist to color one frame of a scene. the right approach, deep learning will
From there, the deep learning network take film and television to a whole new
automatically handles the rest.
level.

http://www.inbroadcast.com

In Broadcast - November 2020

Table of Contents for the Digital Edition of In Broadcast - November 2020