IEEE Circuits and Systems Magazine - Q2 2022 - 26

developers should look beyond them so as to potentially
revolutionize online learning. It is widely accepted that
the biological learning rules, such as STDP, is unsupervised
and local. How to achieve useful machine intelligence
using the unsupervised local learning is another
area to be explored. This may require novel network architectures
that provide local feedback or reward signals
during the learning process. Since unsupervised learning
in general leads to associative memories, a study on the
application development and learning capacity of associative
memory is worthwhile.
Finally, like all online learning algorithms, the online
learning of SNN will also suffer from catastrophic forgetting
and slow convergence. The low data precision in SNN
deprives us the flexibility of controlling the weights precisely.
Hence, techniques such as meta learning, which
carefully move the synaptic weights to a specific combination
that works for multiple input domains, may not be
applicable for SNN. New techniques to improve the quality
of online learning must be studied.
B. Challenges in Architectural Design
At the architecture level, the challenges come from offchip
memory access latency, on-chip memory capacity,
highly diverse SNN models, reconfigurability, massive
connection, neuron density and network parallelism. The
architectural design has to balance these divergent and
tightly coupled aspects. The higher degree of flexibility
and reconfigurability comes at the cost of additional
hardware cost. For example, Loihi suffers a 2 # reduction
in the neuron density compared with TrueNorth after
process normalization [105]. SpiNNaker achieves even
higher flexibility as it adopts general purpose ARM core
and off-chip storage. To mitigate the memory access latency,
SpiNNaker stacks the SDRAM on the chip. Digital
designs such as Loihi, TrueNorth, and SpiNNaker all work
at the speed comparable to a biological system, while
BrainScaleS adopts an analogue design, and it achieves
10 ,000 # speedup over biological speed [298]. These design
trade-offs are made to serve specific purposes. Loihi
and TrueNorth are mainly designed for machine learning
applications, hence use relatively simple models. SpiNNaker
is designed as a super computing system for various
biological research, hence it uses an ARM core rather
than a model-specific core to guarantee flexibility.
Although cost, flexibility, performance, and energy dissipation
are always contradictory goals during the hardware
design, a better architecture can push the design
point for a more efficient trade-off. Optimized resource
allocation and scheduling that maps neurons to physical
cores while maintaining workload balance and minimum
communication will further help to improve the performance
and lower the energy dissipation.
26
IEEE CIRCUITS AND SYSTEMS MAGAZINE
C. Emerging Devices
At the device level, the emerging technologies in nano
devices and materials provide a potential for extremely
small, ultra-fast and extremely low-lower neuromorphic
hardware if they are successfully married with suitable
algorithms. These works share similar ideas as analog/
mixed-signal design, i.e., using the physical process to
naturally and efficiently emulate neuron and synapse dynamics.
In
his work [299], Chua hypothesized the existence of
the missing element, memristor, defined by relation between
flux-linkage z and charge q [300]:
ddz = Mq q
()
(24)
where M(q) is a function of the amount of charge q flowed
through it. Such a memristor behaves like a non-linear resistor
with memory [299].
The synaptic weights in biological systems can be
adjusted by the ionic flow. This is analogous to the resistance
of the memristor, which can be adjusted by
the charge or flux, hence [301] demonstrated that the
synapse function can be implemented by a memristor.
Furthermore, it showed that STDP can be achieved by
a hybrid system, which consists of CMOS neurons and
memristive synapses. Since then, the memristor has attracted
attentions as a promising implementation technique
for neuromorphic systems [217], [302]-[311].
Most of these works adopt a memristor crossbar as it
is capable of providing a high density connection and
efficient implementation of matrix-vector multiplication
[303], [312], [313] and can be used as an accelerator
for neuromorphic computing [310], [314]-[316]. How to
realize synaptic plasticity has also drawn a lot of interest
[307], [317]-[323]. [318] built a single layer perceptron
and implemented in situ training by the delta rule.
[324] realized triplet STDP learning rule on memristors.
[306] also demonstrated the feasibility of implementing
ReLU neuron, convolution layer, fully connected layer
and unsupervised synaptic weight update on memristor
arrays.
A Photonic device is another promising direction for
their ultra-fast operation speed and virtually unlimited
bandwidth [325]-[336]. Recent works show that it is feasible
to implement synapses and neurons by photonic devices.
[325] implemented synapses in the optical domain
via a photonic integrated-circuit based on phase-change
materials (PCMs) cells. Because the PCM can be adjusted
by optical pulses, the PCM cell serves as non-volatile photonic
memories. Synaptic plasticity is also demonstrated
[325]. [326] realized a scalable all-optical spiking neural
network circuit. A network of four input neurons, three
hidden-layer neurons and two output neurons were built
SECOND QUARTER 2022

IEEE Circuits and Systems Magazine - Q2 2022

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2022

IEEE Circuits and Systems Magazine - Q2 2022 - Cover1
IEEE Circuits and Systems Magazine - Q2 2022 - Cover2
IEEE Circuits and Systems Magazine - Q2 2022 - 1
IEEE Circuits and Systems Magazine - Q2 2022 - 2
IEEE Circuits and Systems Magazine - Q2 2022 - 3
IEEE Circuits and Systems Magazine - Q2 2022 - 4
IEEE Circuits and Systems Magazine - Q2 2022 - 5
IEEE Circuits and Systems Magazine - Q2 2022 - 6
IEEE Circuits and Systems Magazine - Q2 2022 - 7
IEEE Circuits and Systems Magazine - Q2 2022 - 8
IEEE Circuits and Systems Magazine - Q2 2022 - 9
IEEE Circuits and Systems Magazine - Q2 2022 - 10
IEEE Circuits and Systems Magazine - Q2 2022 - 11
IEEE Circuits and Systems Magazine - Q2 2022 - 12
IEEE Circuits and Systems Magazine - Q2 2022 - 13
IEEE Circuits and Systems Magazine - Q2 2022 - 14
IEEE Circuits and Systems Magazine - Q2 2022 - 15
IEEE Circuits and Systems Magazine - Q2 2022 - 16
IEEE Circuits and Systems Magazine - Q2 2022 - 17
IEEE Circuits and Systems Magazine - Q2 2022 - 18
IEEE Circuits and Systems Magazine - Q2 2022 - 19
IEEE Circuits and Systems Magazine - Q2 2022 - 20
IEEE Circuits and Systems Magazine - Q2 2022 - 21
IEEE Circuits and Systems Magazine - Q2 2022 - 22
IEEE Circuits and Systems Magazine - Q2 2022 - 23
IEEE Circuits and Systems Magazine - Q2 2022 - 24
IEEE Circuits and Systems Magazine - Q2 2022 - 25
IEEE Circuits and Systems Magazine - Q2 2022 - 26
IEEE Circuits and Systems Magazine - Q2 2022 - 27
IEEE Circuits and Systems Magazine - Q2 2022 - 28
IEEE Circuits and Systems Magazine - Q2 2022 - 29
IEEE Circuits and Systems Magazine - Q2 2022 - 30
IEEE Circuits and Systems Magazine - Q2 2022 - 31
IEEE Circuits and Systems Magazine - Q2 2022 - 32
IEEE Circuits and Systems Magazine - Q2 2022 - 33
IEEE Circuits and Systems Magazine - Q2 2022 - 34
IEEE Circuits and Systems Magazine - Q2 2022 - 35
IEEE Circuits and Systems Magazine - Q2 2022 - 36
IEEE Circuits and Systems Magazine - Q2 2022 - 37
IEEE Circuits and Systems Magazine - Q2 2022 - 38
IEEE Circuits and Systems Magazine - Q2 2022 - 39
IEEE Circuits and Systems Magazine - Q2 2022 - 40
IEEE Circuits and Systems Magazine - Q2 2022 - 41
IEEE Circuits and Systems Magazine - Q2 2022 - 42
IEEE Circuits and Systems Magazine - Q2 2022 - 43
IEEE Circuits and Systems Magazine - Q2 2022 - 44
IEEE Circuits and Systems Magazine - Q2 2022 - 45
IEEE Circuits and Systems Magazine - Q2 2022 - 46
IEEE Circuits and Systems Magazine - Q2 2022 - 47
IEEE Circuits and Systems Magazine - Q2 2022 - 48
IEEE Circuits and Systems Magazine - Q2 2022 - 49
IEEE Circuits and Systems Magazine - Q2 2022 - 50
IEEE Circuits and Systems Magazine - Q2 2022 - 51
IEEE Circuits and Systems Magazine - Q2 2022 - 52
IEEE Circuits and Systems Magazine - Q2 2022 - 53
IEEE Circuits and Systems Magazine - Q2 2022 - 54
IEEE Circuits and Systems Magazine - Q2 2022 - 55
IEEE Circuits and Systems Magazine - Q2 2022 - 56
IEEE Circuits and Systems Magazine - Q2 2022 - 57
IEEE Circuits and Systems Magazine - Q2 2022 - 58
IEEE Circuits and Systems Magazine - Q2 2022 - 59
IEEE Circuits and Systems Magazine - Q2 2022 - 60
IEEE Circuits and Systems Magazine - Q2 2022 - 61
IEEE Circuits and Systems Magazine - Q2 2022 - 62
IEEE Circuits and Systems Magazine - Q2 2022 - 63
IEEE Circuits and Systems Magazine - Q2 2022 - 64
IEEE Circuits and Systems Magazine - Q2 2022 - 65
IEEE Circuits and Systems Magazine - Q2 2022 - 66
IEEE Circuits and Systems Magazine - Q2 2022 - 67
IEEE Circuits and Systems Magazine - Q2 2022 - 68
IEEE Circuits and Systems Magazine - Q2 2022 - Cover3
IEEE Circuits and Systems Magazine - Q2 2022 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q1
https://www.nxtbookmedia.com