ASHRAE Journal - December 2020 - 30
TECHNICAL FEATURE
LBNL's High Performance Computing Center
Continuously Improving
Energy and Water
Management
BY JINGJING LIU, P.E., BEAP; NORMAN BOURASSA, ASSOCIATE MEMBER ASHRAE
High performance computing (HPC) centers are unique in certain aspects such as task
scheduling and power consumption patterns. However, they also share commonalities with other data centers, for example, in the infrastructure systems and opportunities for saving energy and water. The success and lessons learned at LBNL's National
Energy Research Scientific Computing Center (NERSC) can be useful for other data
centers with proper adoption considerations.
NERSC HPC Facility Today
Lawrence Berkeley National Laboratory's (LBNL)
HPC center, NERSC, has a mission to support U.S.
Department of Energy (DOE) Office of Science-funded
scientific research through providing HPC resources to
science users at high-availability with high utilization of
the machines.1
NERSC has been located in Shyh Wang Hall (Figure 1), a
LEED Gold-certified building, on LBNL's main campus
since 2015. The current main production system is Cori
*Unit
(Figure 2), a 30 petaflops* high performance computing
system. The facility consumes an average 4.8 gigawatthours per month. To track its energy efficiency, the
NERSC team has implemented a rigorous 15-minute
interval measurement of power usage effectiveness
(PUE),† drawing from an extensive instrumentation
and data storage system referred to as Operations
Monitoring and Notification Infrastructure (OMNI).3
So far, the team has achieved over 1.8 gigawatt-hours of
energy savings and 0.56 million gallons (2.1 million L) of
of computing speed equal to one thousand million million (1015) floating point operations per second.
†PUE
is the ratio of total facility energy to IT energy; it's a measure of how effectively the IT equipment is served by the power distribution
and cooling systems. A lower PUE number is better, and 1.0 is the lowest theoretical number. There are three levels for PUE measurement per Green Grid:2 Level 1, Level 2 and Level 3 (basic, intermediate and advanced). At each level, the IT load is measured at different
points: the UPS output, power distribution unit (PDU) output and directly from the input of IT equipment.
Jingjing Liu, P.E. is a program manager and researcher at Lawrence Berkeley National Laboratory (LBNL) in Berkeley, Calif. Norman Bourassa works in LBNL's National Energy
Research Scientific Computing (NERSC) building infrastructure group.
30
ASHRAE JOURNAL
ashrae.org
D ECEM BER 2020
https://www.ashrae.org/
ASHRAE Journal - December 2020
Table of Contents for the Digital Edition of ASHRAE Journal - December 2020
Contents
ASHRAE Journal - December 2020 - Intro
ASHRAE Journal - December 2020 - CT1
ASHRAE Journal - December 2020 - CT2
ASHRAE Journal - December 2020 - Cover1
ASHRAE Journal - December 2020 - Cover2
ASHRAE Journal - December 2020 - 1
ASHRAE Journal - December 2020 - Contents
ASHRAE Journal - December 2020 - 3
ASHRAE Journal - December 2020 - 4
ASHRAE Journal - December 2020 - 5
ASHRAE Journal - December 2020 - 6
ASHRAE Journal - December 2020 - 7
ASHRAE Journal - December 2020 - 8
ASHRAE Journal - December 2020 - 9
ASHRAE Journal - December 2020 - 10
ASHRAE Journal - December 2020 - 11
ASHRAE Journal - December 2020 - 12
ASHRAE Journal - December 2020 - 13
ASHRAE Journal - December 2020 - 14
ASHRAE Journal - December 2020 - 15
ASHRAE Journal - December 2020 - 16
ASHRAE Journal - December 2020 - 17
ASHRAE Journal - December 2020 - 18
ASHRAE Journal - December 2020 - 19
ASHRAE Journal - December 2020 - 20
ASHRAE Journal - December 2020 - 21
ASHRAE Journal - December 2020 - 22
ASHRAE Journal - December 2020 - 23
ASHRAE Journal - December 2020 - 24
ASHRAE Journal - December 2020 - 25
ASHRAE Journal - December 2020 - 26
ASHRAE Journal - December 2020 - 27
ASHRAE Journal - December 2020 - 28
ASHRAE Journal - December 2020 - 29
ASHRAE Journal - December 2020 - 30
ASHRAE Journal - December 2020 - 31
ASHRAE Journal - December 2020 - 32
ASHRAE Journal - December 2020 - 33
ASHRAE Journal - December 2020 - 34
ASHRAE Journal - December 2020 - 35
ASHRAE Journal - December 2020 - 36
ASHRAE Journal - December 2020 - 37
ASHRAE Journal - December 2020 - 38
ASHRAE Journal - December 2020 - 39
ASHRAE Journal - December 2020 - 40
ASHRAE Journal - December 2020 - 41
ASHRAE Journal - December 2020 - 42
ASHRAE Journal - December 2020 - 43
ASHRAE Journal - December 2020 - 44
ASHRAE Journal - December 2020 - 45
ASHRAE Journal - December 2020 - 46
ASHRAE Journal - December 2020 - 47
ASHRAE Journal - December 2020 - 48
ASHRAE Journal - December 2020 - 49
ASHRAE Journal - December 2020 - 50
ASHRAE Journal - December 2020 - 51
ASHRAE Journal - December 2020 - 52
ASHRAE Journal - December 2020 - 53
ASHRAE Journal - December 2020 - 54
ASHRAE Journal - December 2020 - 55
ASHRAE Journal - December 2020 - 56
ASHRAE Journal - December 2020 - 57
ASHRAE Journal - December 2020 - 58
ASHRAE Journal - December 2020 - 59
ASHRAE Journal - December 2020 - 60
ASHRAE Journal - December 2020 - 61
ASHRAE Journal - December 2020 - 62
ASHRAE Journal - December 2020 - 63
ASHRAE Journal - December 2020 - 64
ASHRAE Journal - December 2020 - 65
ASHRAE Journal - December 2020 - 66
ASHRAE Journal - December 2020 - 67
ASHRAE Journal - December 2020 - 68
ASHRAE Journal - December 2020 - 69
ASHRAE Journal - December 2020 - 70
ASHRAE Journal - December 2020 - 71
ASHRAE Journal - December 2020 - 72
ASHRAE Journal - December 2020 - 73
ASHRAE Journal - December 2020 - 74
ASHRAE Journal - December 2020 - 75
ASHRAE Journal - December 2020 - 76
ASHRAE Journal - December 2020 - 77
ASHRAE Journal - December 2020 - 78
ASHRAE Journal - December 2020 - 79
ASHRAE Journal - December 2020 - 80
ASHRAE Journal - December 2020 - 81
ASHRAE Journal - December 2020 - 82
ASHRAE Journal - December 2020 - 83
ASHRAE Journal - December 2020 - 84
ASHRAE Journal - December 2020 - 85
ASHRAE Journal - December 2020 - 86
ASHRAE Journal - December 2020 - 87
ASHRAE Journal - December 2020 - 88
ASHRAE Journal - December 2020 - 89
ASHRAE Journal - December 2020 - 90
ASHRAE Journal - December 2020 - 91
ASHRAE Journal - December 2020 - 92
ASHRAE Journal - December 2020 - 93
ASHRAE Journal - December 2020 - 94
ASHRAE Journal - December 2020 - 95
ASHRAE Journal - December 2020 - 96
ASHRAE Journal - December 2020 - 97
ASHRAE Journal - December 2020 - 98
ASHRAE Journal - December 2020 - 99
ASHRAE Journal - December 2020 - 100
ASHRAE Journal - December 2020 - 101
ASHRAE Journal - December 2020 - 102
ASHRAE Journal - December 2020 - 103
ASHRAE Journal - December 2020 - 104
ASHRAE Journal - December 2020 - 105
ASHRAE Journal - December 2020 - 106
ASHRAE Journal - December 2020 - 107
ASHRAE Journal - December 2020 - 108
ASHRAE Journal - December 2020 - 109
ASHRAE Journal - December 2020 - 110
ASHRAE Journal - December 2020 - 111
ASHRAE Journal - December 2020 - 112
ASHRAE Journal - December 2020 - 113
ASHRAE Journal - December 2020 - 114
ASHRAE Journal - December 2020 - 115
ASHRAE Journal - December 2020 - 116
ASHRAE Journal - December 2020 - 117
ASHRAE Journal - December 2020 - 118
ASHRAE Journal - December 2020 - 119
ASHRAE Journal - December 2020 - 120
ASHRAE Journal - December 2020 - 121
ASHRAE Journal - December 2020 - 122
ASHRAE Journal - December 2020 - 123
ASHRAE Journal - December 2020 - 124
ASHRAE Journal - December 2020 - 125
ASHRAE Journal - December 2020 - 126
ASHRAE Journal - December 2020 - 127
ASHRAE Journal - December 2020 - 128
ASHRAE Journal - December 2020 - Cover3
ASHRAE Journal - December 2020 - Cover4
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_MCDEBX
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_WNYSQY
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_XATVOD
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_FJSHSS
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_CCBZDS
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_XDEFVG
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2023november_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2023november
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_VHGNBL
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_WPKBNJ
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_UUVCDE
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_RTGDEW
https://www.nxtbook.com/nxtbooks/ashrae/ashraemexico_2023
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_LKRFXS
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_AZSOFG
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_ERCDBH
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_QWDFRV
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_JHGVDF
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_OPUYHG
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_SREIBM
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_LRTGLK
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_OKRFGH
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2022november_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2022november
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_TZSERA
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_LVRUIX
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_RPTVYZ
https://www.nxtbook.com/nxtbooks/ashrae/mini_pub_catalog
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_XIYTGD
https://www.nxtbook.com/nxtbooks/ashrae/ashraemexico_2022
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_RFGDOB
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_PABXNU
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_REMKLS
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_PICVBT
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_AOYTVW
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_JQOPLS
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_IOYTBC
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_SGAJJF
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_IGHYER
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_PDRKLS
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2021november
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2021november_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_XCODFR
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_QSLFGO
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_ILKVNM
https://www.nxtbook.com/nxtbooks/ashrae/ashraemexico_2021
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_OPDJKD
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_VJKSRY
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_SDHUTC
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_JPPKRR
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_SDLTTH
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_CKLLES
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_SLDOX
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_HJETUK
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_OLUHGE
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2020october
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2020october_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_ZERDGH
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_QVMNEO
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_RTPOKE
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_BBATRE
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_STUBMW
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_TPEMPE
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_JNMKDS
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_FBTTPA
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_WQMMNE
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_TVBRYN
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_showguide2020
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_KTUZMA
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_ABEDGD
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201910
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201909
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2019septmeber_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2019september
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201908
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201907
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201906
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201905
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201904
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_2019april
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201903
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_2019march
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201902
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201901
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_showguide2019
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_2018december
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_2018november
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2018fall_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2018fall
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_2018october
https://www.nxtbook.com/nxtbooks/ashrae/ashraemexico_2018
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201810
https://www.nxtbook.com/nxtbooks/ashrae/ashraeinsights_201806
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201805
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201804
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201803
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201712
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201711
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201710
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2017fall_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2017fall
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201709
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201705
https://www.nxtbook.com/nxtbooks/ashrae/ashrae_meetinginsert_201610
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2016fall_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2016fall
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_acrexindia
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2015summer_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_amca_2015summer
https://www.nxtbook.com/nxtbooks/amca/2014summer2
https://www.nxtbook.com/nxtbooks/amca/2014summer
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_acma_2014summer
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201311
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201309
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_acmasupp_2013fall
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201305
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201303
https://www.nxtbook.com/nxtbooks/ashrae/pubcatalog_2013winter
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201211
https://www.nxtbook.com/nxtbooks/ashrae/achr_expo_mexico2012
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201209
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201208_v3
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201208_v2
https://www.nxtbook.com/nxtbooks/ashrae/pubcatalog_2012summer
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201205
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201203
https://www.nxtbook.com/nxtbooks/ashrae/pubcatalog_2012winter
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201111_v2
https://www.nxtbook.com/nxtbooks/ashrae/ashraejournal_201109_v2
https://www.nxtbook.com/nxtbooks/ashrae/pubcatalog_2011summer
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201105
https://www.nxtbook.com/nxtbooks/ashrae/meetingplanner_201103
https://www.nxtbookmedia.com