IEEE Technology and Society Magazine - June 2020 - 50

done to select an energy source.
However, since increasing quantities
of models are trained on the cloud,
it is entirely possible and even very
easy to choose the location of a
server used for training and, therefore, the energy source it is connected to. Therefore, while practically
speaking it is hard to calculate precisely how much carbon is emitted
by a model during its training on the
cloud, this amount can be estimated using information about the local
energy grid if we assume that all
servers are connected to local grids
at their physical location and no
alternative source of energy exists.1

Cloud Providers
When running on the cloud, it is necessary to choose a cloud provider,
whose servers are scattered throughout the word and therefore connected to grids with different emissions
factors. In order to create our ML
emissions calculator, we gathered as
much publicly-available data as we
could regarding the carbon emissions of different energy grids, at
varying levels of granularity, from
regional to national. We then crossreferenced these with cloud server
locations from the three major cloud
providers: Google Cloud Platform,
Microsoft Azure, and Amazon Web
Services. We found that emissions
can vary by up to a factor of forty
depending on the energy mix of the
grid. For example, in a region like
Québec, Canada, which relies mostly
on hydroelectricity, this can be roughly 20 grams of carbon per kWh,
whereas in a place like Queensland,
Australia, this can go up to 800 grams
of carbon, since Queensland's power
1

This is not always the case, since sometimes
data centers rely in part or in full on local energy sources and are not connected to the energy
grid, e.g., Google's St. Ghislain data center,
which has solar panels on its roof (https://

blog.google/around-the-globe/google-europe/
time- shine-new- sola r-facilit y-a nd-additiona l
-data-center-belgium/).

50

ware. While this case should be considered common practice, there is a
general trend that can be observed
towards more data, more powerful
computing hardware, and longer
training time. Recent data published
by OpenAI shows that the computing power required for key ML tasks
has doubled every 3 months or so,
increasing 300 000 times between
2012 and 2018. At this rate, not
only are emissions from AI going to
rise exponentially, but so will the
barriers to entry in the domain,
since few individuals and smaller
companies and research labs will be
able to afford the sheer computational power required to innovate
and to beat existing leaderboards.
As we mentioned above, while the
initial network training procedure definitely accounts for a large portion of
its carbon emissions, there are also
other factors to consider: for instance,
whether it is necessary to fine-tune the
network and to what extent. In fact,
in a recent article comparing the carbon emissions produced by a neural
network to the average lifetime emissions of five cars, only a small fraction
of those emissions were produced by
a single model training, whereas the
majority was a result of the architecture search and hyperparameter tuning [6]. This is an important factor to
keep in mind, since many recent powerful models in areas like machine
translation and image recognition
have been shared online with the
broader research community, meaning that it is no longer necessary to
train models from scratch, and it is
possible to take a pre-trained model
and use it as is, or train it for less time
on a more specific task. This means
that, for example, if a neuro-linguistic
programing (NLP) model was initially
trained to translate English to French
texts on a huge corpus with millions
of documents and shared publicly, it
can then be customized on a smaller set of documents to carry out

grid relies almost solely on fossil
fuels. The carbon emissions can really add up for a big neural network
trained on a few graphical processing
units (GPUs) for several weeks, resulting in almost a ton of carbon, or the
equivalent of a transatlantic flight [2].
Something to keep in mind is
that some major cloud providers are
already carbon neutral, meaning that
they have a net zero carbon footprint.
This is done via carbon offsetting
and renewable energy credits (RECs),
which involve purchasing an equivalent amount of renewable energy for
the amount of non-renewable energy
used. This does not mean, however,
that the associated cloud computing
does not emit carbon - it just means
that there is a significant effort to balance the carbon emitted by incentivizing and democratizing the use
of renewable energy. There is also a
general tendency towards more transparency and accountability, which
can help when choosing a given cloud
provider when several operate in the
same region. For instance, Microsoft
recently announced that they will be
"carbon negative" by 2030, meaning
that they will remove all the carbon
that they have emitted either directly
or by electrical consumption since
their founding in 1975.2

Hardware and Training Time
While computing hardware has been
getting more powerful, capable of
carrying out more calculations in
less time, efficiency is being reduced
because computing is being used for
longer and longer to learn more and
more complex tasks. For instance,
MegaFace, a popular facial recognition dataset, has almost 5 million
images [5], and training a state-of-theart facial recognition model with millions of parameters can take months
on even the most powerful hard2

S o u r c e : h t t p s : // b l o g s . m i c r o s o f t . c o m /
blo g / 2020/01/16/m icro sof t-w i l l -be - c a rbon
-negative-by-2030/.

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

∕

JUNE 2020


https://blog.google/around-the-globe/google-europe/time-shine-new-solar-facility-and-additional-data-center-belgium https://blogs.microsoft.com/blog/2020/01/16/microsoft-will-be-carbon-negative-by-2030/ https://blog.google/around-the-globe/google-europe/time-shine-new-solar-facility-and-additional-data-center-belgium https://blog.google/around-the-globe/google-europe/time-shine-new-solar-facility-and-additional-data-center-belgium https://blogs.microsoft.com/blog/2020/01/16/microsoft-will-be-carbon-negative-by-2030/ https://blog.google/around-the-globe/google-europe/time-shine-new-solar-facility-and-additional-data-center-belgium https://blogs.microsoft.com/blog/2020/01/16/microsoft-will-be-carbon-negative-by-2030/

IEEE Technology and Society Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2020

Contents
IEEE Technology and Society Magazine - June 2020 - Cover1
IEEE Technology and Society Magazine - June 2020 - Cover2
IEEE Technology and Society Magazine - June 2020 - 1
IEEE Technology and Society Magazine - June 2020 - Contents
IEEE Technology and Society Magazine - June 2020 - 3
IEEE Technology and Society Magazine - June 2020 - 4
IEEE Technology and Society Magazine - June 2020 - 5
IEEE Technology and Society Magazine - June 2020 - 6
IEEE Technology and Society Magazine - June 2020 - 7
IEEE Technology and Society Magazine - June 2020 - 8
IEEE Technology and Society Magazine - June 2020 - 9
IEEE Technology and Society Magazine - June 2020 - 10
IEEE Technology and Society Magazine - June 2020 - 11
IEEE Technology and Society Magazine - June 2020 - 12
IEEE Technology and Society Magazine - June 2020 - 13
IEEE Technology and Society Magazine - June 2020 - 14
IEEE Technology and Society Magazine - June 2020 - 15
IEEE Technology and Society Magazine - June 2020 - 16
IEEE Technology and Society Magazine - June 2020 - 17
IEEE Technology and Society Magazine - June 2020 - 18
IEEE Technology and Society Magazine - June 2020 - 19
IEEE Technology and Society Magazine - June 2020 - 20
IEEE Technology and Society Magazine - June 2020 - 21
IEEE Technology and Society Magazine - June 2020 - 22
IEEE Technology and Society Magazine - June 2020 - 23
IEEE Technology and Society Magazine - June 2020 - 24
IEEE Technology and Society Magazine - June 2020 - 25
IEEE Technology and Society Magazine - June 2020 - 26
IEEE Technology and Society Magazine - June 2020 - 27
IEEE Technology and Society Magazine - June 2020 - 28
IEEE Technology and Society Magazine - June 2020 - 29
IEEE Technology and Society Magazine - June 2020 - 30
IEEE Technology and Society Magazine - June 2020 - 31
IEEE Technology and Society Magazine - June 2020 - 32
IEEE Technology and Society Magazine - June 2020 - 33
IEEE Technology and Society Magazine - June 2020 - 34
IEEE Technology and Society Magazine - June 2020 - 35
IEEE Technology and Society Magazine - June 2020 - 36
IEEE Technology and Society Magazine - June 2020 - 37
IEEE Technology and Society Magazine - June 2020 - 38
IEEE Technology and Society Magazine - June 2020 - 39
IEEE Technology and Society Magazine - June 2020 - 40
IEEE Technology and Society Magazine - June 2020 - 41
IEEE Technology and Society Magazine - June 2020 - 42
IEEE Technology and Society Magazine - June 2020 - 43
IEEE Technology and Society Magazine - June 2020 - 44
IEEE Technology and Society Magazine - June 2020 - 45
IEEE Technology and Society Magazine - June 2020 - 46
IEEE Technology and Society Magazine - June 2020 - 47
IEEE Technology and Society Magazine - June 2020 - 48
IEEE Technology and Society Magazine - June 2020 - 49
IEEE Technology and Society Magazine - June 2020 - 50
IEEE Technology and Society Magazine - June 2020 - 51
IEEE Technology and Society Magazine - June 2020 - 52
IEEE Technology and Society Magazine - June 2020 - 53
IEEE Technology and Society Magazine - June 2020 - 54
IEEE Technology and Society Magazine - June 2020 - 55
IEEE Technology and Society Magazine - June 2020 - 56
IEEE Technology and Society Magazine - June 2020 - 57
IEEE Technology and Society Magazine - June 2020 - 58
IEEE Technology and Society Magazine - June 2020 - 59
IEEE Technology and Society Magazine - June 2020 - 60
IEEE Technology and Society Magazine - June 2020 - 61
IEEE Technology and Society Magazine - June 2020 - 62
IEEE Technology and Society Magazine - June 2020 - 63
IEEE Technology and Society Magazine - June 2020 - 64
IEEE Technology and Society Magazine - June 2020 - 65
IEEE Technology and Society Magazine - June 2020 - 66
IEEE Technology and Society Magazine - June 2020 - 67
IEEE Technology and Society Magazine - June 2020 - 68
IEEE Technology and Society Magazine - June 2020 - 69
IEEE Technology and Society Magazine - June 2020 - 70
IEEE Technology and Society Magazine - June 2020 - 71
IEEE Technology and Society Magazine - June 2020 - 72
IEEE Technology and Society Magazine - June 2020 - 73
IEEE Technology and Society Magazine - June 2020 - 74
IEEE Technology and Society Magazine - June 2020 - 75
IEEE Technology and Society Magazine - June 2020 - 76
IEEE Technology and Society Magazine - June 2020 - 77
IEEE Technology and Society Magazine - June 2020 - 78
IEEE Technology and Society Magazine - June 2020 - 79
IEEE Technology and Society Magazine - June 2020 - 80
IEEE Technology and Society Magazine - June 2020 - Cover3
IEEE Technology and Society Magazine - June 2020 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2013
https://www.nxtbookmedia.com