IEEE Solid-States Circuits Magazine - Summer 2022 - 68

The OSD minimizes the read/write
energy of the partial sum, which
requires higher precision than the
input or weights.
Digital AI Accelerators
Featuring Low Precision
Different DNN models have different
optimal weight precisions, and different
layers in a given DNN model
have different optimal precisions.
Therefore, to obtain the optimal
energy-accuracy tradeoff, supporting
variable weight precision is
important for DNN accelerators.
In the UNPU accelerator [22], variable
weight precision from 1 to 16 b
is supported by bit serial processing,
where MAC operations with N-b
weight precision are computed sequentially
from the LSB to the MSB of the
weights for N cycles by shifting and
accumulating the partial sums.
Higher energy efficiency is achieved
for lower-precision weights, while
the accuracy degradation could occur
depending on the DNN. In addition,
by efficiently reusing input feature
maps, the same UNPU chip can be
fully shared for convolution, recurrent,
and fully connected layers.
In [23], Intel presented a digital
binary neural network (BNN) accelerator
in 10-nm CMOS. When N-b
precision is reduced to 1-b precision,
the memory storage is reduced linearly,
while the compute complexity
is reduced quadratically (e.g., 8-b
MAC energy is 10-100× higher than
1-b MAC energy). To that end, the proposed
BNN accelerator employs much
higher parallelism of very-low-precision
MACs and data reuse to amortize
the cost of memory access and data
movement across many operations.
The accelerator chip has 131,000
binary MAC units or XNOR gates split
between a total of 128 memory execution
units. At 0.37 V, 617 TOPS/W
energy efficiency is achieved.
IBM presented a 7-nm AI chip [24]
that supports both fixed-point precision
inference and floating-point
precision training. This chip consists
of four AI cores, where each core has
two corelets with a private L0 and a
shared L1 scratchpad.
Each corelet contains an 88#
array of mixed-precision engines
(MPEs), where MPEs implement separate
compute pipelines for various
precisions. Inference workloads can
be executed with 2- or 4-b fixedpoint
precision and training workloads
can be operated with hybrid
8- or 16-b floating-point precision to
meet diverse application demands
for both AI inference and training.
For inference with 4-b fixed-point
precision, with a 0.55-V core and
0.7-V SRAM supply, 16.5 TOPS/W
are reported. A scaled-up chip with
32 cores achieves >60% utilization
for ResNet-50 and >70% utilization
for the Google Neural Machine Translation
model.
Digital AI Accelerators
Featuring Pruning/Sparsity
The STICKER-T accelerator [25] employed
block-circulant weights as a structured
compression technique, where
each row vector circulantly rotates
one element to the right side to generate
the next row vector. Therefore,
the first row includes all
information
in this matrix block, leading
to N# storage reduction. By training
weights in the block-circulant
matrix format, the matrix-vector
multiplication can be performed
with frequency-domain elementwise
production, and FFT operations can
reduce the computation complexity
from ()
On2
to ()
Onlogn .
With the block-circulant technique,
STICKER-T has a frequency-domain
16 16# MAC array with bit serial
processing to flexibly support 1-12-b
precision. Sixteen activations are
shared by the same PE row, and 16
weights are shared by the same PE
Input
Activation
W11
W12
W13
P11
P12
P13
Weight
Partial Sum
Weight Memory
W21
W22
W23
P21
Partial Sum
W31
W32
(a)
W33
P31
P32
(b)
FIGURE 2: (a) The weight-stationary dataflow [19], [20] and (b) output-stationary dataflow [21]. P: partial sum; W: weight.
68 SUMMER 2022
IEEE SOLID-STATE CIRCUITS MAGAZINE
P22
P23
Weight
P33
Activation Memory
Activation Memory

IEEE Solid-States Circuits Magazine - Summer 2022

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Summer 2022