IEEE Solid-States Circuits Magazine - Fall 2021 - 75

(e.g., floating point, 16-b, 8-b, and binarized)
may not be completely fair;
however, it is the standard practice.
COTS Devices
The NVIDIA Pascal-family GPU is
loaded with 8 GB of memory and 256
cores that can operate at a clock frequency
of 1.3 GHz and deliver up to
1.33 teraflops for high-performance
computing workloads, like matrix-matrix
multiplication using developed
libraries, such as cuBLAS. Such performance
is only very high for large
workloads, and it degrades when the
workload is small. CuSPARSE is another
useful library that performs sparse
matrix operations, such as a pruned
matrix-vector multiplication, and it
outperforms cuBLAS if the matrix is
sufficiently sparsified.
We conduct a set of experiments
of matrix-vector multiplication over
the GPU using both the cuBLAS and
cuSPARSE libraries as well as over the
CSC-AP for a CSC matrix of size 1,024
by 1,024 with various compression
rates. We measure the performance
and efficiency of these implementations
only after the data are loaded
to the on-chip memory of the two
devices. Figure 7(a) shows the true
efficiency of the CSC-AP at 100 MHz
(0.62 V) and compares it with that of
the GPU at 1.3 GHz using either of the
two NVIDIA Compute Unified Device
Architecture libraries over the given
workload with various compression
rates. Figure 7(b) plots the equivalent
efficiency of the experiments by
taking the compression rate into account
according to (13).
According to this experiment, the
performance and efficiency of cuBLAS
is constant for variously pruned matrices,
and, since it is agnostic to the
sparsity of its workload, its equivalent
performance and efficiency are the
same as its actual ones. On the other
hand, the equivalent performance and
efficiency of both the CSC-AP and cuSPARSE
libraries increase proportionally
to the compression rate of the CSC
matrix. Consequently, for a matrix-
vector multiplication with N = 1,024,
the efficiency of the CSC-AP ranges beTABLE
3. A COMPARISON WITH RELATED WORK.
EYERISS [8]
Layer type
Sparsity mode
Tech (nm)
Die area (mm2)
Data width (b)
Multiplier count
On-chi buffer (kB)
Supply voltage (V)
Power (mW)
Frequency (MHz)
Peak efficiency (TOPJ)
CONV: convolutional.
CONV
Dense
65
12.25
16
168
108
0.82-1.17
235-332
100-250
0.31
UNPU [9]
CONV + FC
Dense
65
16
1-16
13,824
256
0.63-1.1
3.2-297
5-200
3.08
STICKER [10]
CONV + FC
Sparse
65
7.8
8
256
170
0.67-1
20.5-284.4
20-200
0.41-62.1
CSC-AP
FC
CSC
65
7
8
16
32
0.62-1
5.6-74.3
1-100
0.04-3.61
tween 43 and 2,756 GOPJ compared to
that of the cuSPARSE, ranging between
2 and 66 floating point operations per
joule (FLOPJ) and efficiency of cuBLAS,
which is constant 4.2 FLOPJ for compression
rates ranging between one
and 64 times.
ASICs
Table 3 summarizes three accelerators
fabricated in 65-nm technology
that process neural network layers
and compares them to the CSC-AP.
The main purpose of this table is to
compare the related work and review
the potentials that the 65-nm technology
brings about, given different
approaches and architectures.
Eyeriss [8] is an accelerator for
deep convolutional neural networks
optimized for energy efficiency and
16-b models. It encompasses a wide
range of neural network layers; however,
it delivers the same amount of
efficiency given pruned DNN layers,
whereas the efficiency of the CSC-AP
can approach 3.6 TOPJ for highly compressed
CSC layers.
A unified neural processing unit
(UNPU) [9] is a unified DNN accelerator
that supports variable weight
precision ranging from 1 to 16 b for
the efficient implementation of DNNs
within a mobile environment. While
the CSC-AP consumes two times less
silicon area, it can consume as little
as 5.6 mW at low-frequency settings,
which is comparable to the UNPU's
low power consumption of 3.2 mW at
the same frequency range.
Lastly, STICKER [10] is an 8-b accelerator
that explores a neural network's
sparsity both in weights and
the fmap data and can deliver up to
62.1 TOPJ, a high amount of equivalent
efficiency, which is accounted
for by an fmap and neural network
layer both with 10% sparsity. Compared
to STICKER, the CSC-AP has
16 times fewer multipliers and five
times less on-chip memory, thus delivering
approximately 10-20 times
less efficiency yet consuming approximately
four times less power.
Conclusions
CSC architectures are structurally
sparse graphs that follow a circular
arrangement in their design and have
a density of ()O logNN ; they can be
used to reduce the memory/computation
complexity of FC layers of ()O N2
where N is the number of nodes in a
given layer. CSC architectures can effectively
compress the traditional FC
layers of neural networks on par with
pruning methods, and then can be implemented
using a hardware-friendly
style consuming minimal hardware resources
and power as well as providing
high energy efficiency.
The CSC-AP is a chip with a die size
of 7mm2
fabricated in 65 nm that can
process neural networks compressed
IEEE SOLID-STATE CIRCUITS MAGAZINE
FALL 2021
75
IEEE Solid-States Circuits Magazine - Fall 2021

Table of Contents for the Digital Edition of IEEE Solid-States Circuits Magazine - Fall 2021
