IEEE Circuits and Systems Magazine - Q2 2022 - 11

until a sufficiently large number of spikes have been received.
While a typical neuron firing rate is between 1 and
200 Hz, in realistic biological neural networks, there is not
enough time to integrate spikes to get the spike count. For
example, a fly can respond to a visual object after one or
two spikes are received [12]. Secondly, rate coding is not
energy efficient. It represents large values using high spike
frequency, which increases the switching activities in computing
hardware, and may even pose challenges to neuromorphic
chip design [38]-[40]. Without extended latency
or escalated spiking frequency, rate coding will suffer from
high quantization error. When spikes are generated as stochastic
events, there will be sampling errors too.
Temporal coding takes spike timing into account [41]-
[43] such that the temporal structure of a spike train can
convey information. Two spike-trains with the same spike
count could represent distinct information, as shown by
[44], and produce significantly different postsynaptic current.
When considering the spike timing, the information
capacity of a spike train is significantly increased [45].
However, temporal coding is still not well understood.
There are many hypotheses, which lead to different variations
of temporal coding schemes. For example, [46] shows
that the spatial structure of an image is encoded by retinal
ganglia using the relative timing of first spikes, referred to
as latency coding. Latency coding assumes that the first
spike carries the most significant information, while the
subsequent spikes are less important. The latency coding
is also known as Time-to-first-spike (TTFS) coding [12]. It is
noteworthy that there are some subtle differences between
the latency coding observed in a biological system and the
TTFS in the context of neuromorphic computing. The latter
utilizes at most one spike per neuron to encode information
by applying a long refractory period or a strong inhibition
[47], [48], while there is no such restriction in biological systems.
[49] proposes reverse coding, which assumes that a
stronger stimulus is encoded by a later spike time. This can
be interpreted as a variation of TTFS. Training algorithms
and neuromorphic hardware have been designed specifically
for TTFS coding [47]-[50]. TTFS usually allows more
efficient hardware because it substantially reduces spike
numbers, hence the communication workload is less. Furthermore,
neurons using TTFS do not have to accumulate
multiple spikes to produce output, hence the computation
latency is also reduced. As another variation of temporal
coding, phase coding considers the entire spike train. Information
is represented by the relative spike timing with
respect to periodical background oscillation [12].
[51] and [52] suggest that different coding schemes
may co-exist in the nervous system, and the brain uses
different coding for different tasks. The variety and task
specialization of coding schemes can also be seen in existing
research in SNN. For example, [53], [54] encode imSECOND
QUARTER 2022
age as spatial spike patterns. [55], [56] proposed to convert
audio signals into time-varying spike patterns.
The choice of neural coding scheme is closely related to
the decision on SNN training algorithms, neuron models and
even the hardware architecture. For example, to recognize
different temporal spike patterns, [57] employs LIF neuron
with dual-exponential synapse defined in Equation 7. Every
individual input spike builds up a time-varying PSP,
which represents certain characteristics; [50] designs a dedicated
single-spiking MAC circuit to support TTFS.
To utilize the information embedded in spike timing,
neuron models with certain temporal dynamics, as discussed
in Section II-B, must be used. For example, [33],
[57]-[59] use SRM or its variants to learn spike timings.
However, these models are not readily supported by some
of the existing neuromorphic hardware. For example,
TrueNorth uses a simplified LIF model, where a neuron's
membrane potential is the accumulation of weighted input
spikes with a constant leakage. While this architecture
provides extremely high neuron/synapse density
and energy efficiency, it is not suitable to implement the
aforementioned models of temporal coding. A few other
neuromorphic systems have memory allocated to each
synapse to store the temporal dynamics. For example,
Loihi allows a synapse to have three different state variables,
which can be configured as traces. [34] reported an
FPGA based SNN where each neuron core has a dedicated
memory bank for the post-synaptic potential.
D. Network Topologies
Given the models of neurons and synapses, a spiking neural
network can be constructed. Based on the network
topology, we divide the SNNs into three categories, feedforward,
recurrent and bio-inspired networks. Feedforward
and recurrent networks are inspired by ANNs as
shown in Figure 5 whereas the bio-inspired networks
mimic the structure of various biological neural motifs.
1) ANN-Inspired
Feedforward network is the simplest form of neural networks
where the information moves in only one direction,
from the input layer, through the hidden layers and to the
output layer.
The recurrent SNN can be further divided into two
categories, recurrent with synchronization and recurrent
without synchronization. Recurrent structure is widely
used in ANNs to detect temporal patterns in input sequences
or generate temporally correlated outputs. In
these ANNs, the hidden state of the network induced by
previous input loops back to be processed with the current
input to generate new hidden states and outputs.
The synchronization between the hidden state of the previous
cycle and the input of the current cycle is difficult
IEEE CIRCUITS AND SYSTEMS MAGAZINE
11

IEEE Circuits and Systems Magazine - Q2 2022

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2022

IEEE Circuits and Systems Magazine - Q2 2022 - Cover1
IEEE Circuits and Systems Magazine - Q2 2022 - Cover2
IEEE Circuits and Systems Magazine - Q2 2022 - 1
IEEE Circuits and Systems Magazine - Q2 2022 - 2
IEEE Circuits and Systems Magazine - Q2 2022 - 3
IEEE Circuits and Systems Magazine - Q2 2022 - 4
IEEE Circuits and Systems Magazine - Q2 2022 - 5
IEEE Circuits and Systems Magazine - Q2 2022 - 6
IEEE Circuits and Systems Magazine - Q2 2022 - 7
IEEE Circuits and Systems Magazine - Q2 2022 - 8
IEEE Circuits and Systems Magazine - Q2 2022 - 9
IEEE Circuits and Systems Magazine - Q2 2022 - 10
IEEE Circuits and Systems Magazine - Q2 2022 - 11
IEEE Circuits and Systems Magazine - Q2 2022 - 12
IEEE Circuits and Systems Magazine - Q2 2022 - 13
IEEE Circuits and Systems Magazine - Q2 2022 - 14
IEEE Circuits and Systems Magazine - Q2 2022 - 15
IEEE Circuits and Systems Magazine - Q2 2022 - 16
IEEE Circuits and Systems Magazine - Q2 2022 - 17
IEEE Circuits and Systems Magazine - Q2 2022 - 18
IEEE Circuits and Systems Magazine - Q2 2022 - 19
IEEE Circuits and Systems Magazine - Q2 2022 - 20
IEEE Circuits and Systems Magazine - Q2 2022 - 21
IEEE Circuits and Systems Magazine - Q2 2022 - 22
IEEE Circuits and Systems Magazine - Q2 2022 - 23
IEEE Circuits and Systems Magazine - Q2 2022 - 24
IEEE Circuits and Systems Magazine - Q2 2022 - 25
IEEE Circuits and Systems Magazine - Q2 2022 - 26
IEEE Circuits and Systems Magazine - Q2 2022 - 27
IEEE Circuits and Systems Magazine - Q2 2022 - 28
IEEE Circuits and Systems Magazine - Q2 2022 - 29
IEEE Circuits and Systems Magazine - Q2 2022 - 30
IEEE Circuits and Systems Magazine - Q2 2022 - 31
IEEE Circuits and Systems Magazine - Q2 2022 - 32
IEEE Circuits and Systems Magazine - Q2 2022 - 33
IEEE Circuits and Systems Magazine - Q2 2022 - 34
IEEE Circuits and Systems Magazine - Q2 2022 - 35
IEEE Circuits and Systems Magazine - Q2 2022 - 36
IEEE Circuits and Systems Magazine - Q2 2022 - 37
IEEE Circuits and Systems Magazine - Q2 2022 - 38
IEEE Circuits and Systems Magazine - Q2 2022 - 39
IEEE Circuits and Systems Magazine - Q2 2022 - 40
IEEE Circuits and Systems Magazine - Q2 2022 - 41
IEEE Circuits and Systems Magazine - Q2 2022 - 42
IEEE Circuits and Systems Magazine - Q2 2022 - 43
IEEE Circuits and Systems Magazine - Q2 2022 - 44
IEEE Circuits and Systems Magazine - Q2 2022 - 45
IEEE Circuits and Systems Magazine - Q2 2022 - 46
IEEE Circuits and Systems Magazine - Q2 2022 - 47
IEEE Circuits and Systems Magazine - Q2 2022 - 48
IEEE Circuits and Systems Magazine - Q2 2022 - 49
IEEE Circuits and Systems Magazine - Q2 2022 - 50
IEEE Circuits and Systems Magazine - Q2 2022 - 51
IEEE Circuits and Systems Magazine - Q2 2022 - 52
IEEE Circuits and Systems Magazine - Q2 2022 - 53
IEEE Circuits and Systems Magazine - Q2 2022 - 54
IEEE Circuits and Systems Magazine - Q2 2022 - 55
IEEE Circuits and Systems Magazine - Q2 2022 - 56
IEEE Circuits and Systems Magazine - Q2 2022 - 57
IEEE Circuits and Systems Magazine - Q2 2022 - 58
IEEE Circuits and Systems Magazine - Q2 2022 - 59
IEEE Circuits and Systems Magazine - Q2 2022 - 60
IEEE Circuits and Systems Magazine - Q2 2022 - 61
IEEE Circuits and Systems Magazine - Q2 2022 - 62
IEEE Circuits and Systems Magazine - Q2 2022 - 63
IEEE Circuits and Systems Magazine - Q2 2022 - 64
IEEE Circuits and Systems Magazine - Q2 2022 - 65
IEEE Circuits and Systems Magazine - Q2 2022 - 66
IEEE Circuits and Systems Magazine - Q2 2022 - 67
IEEE Circuits and Systems Magazine - Q2 2022 - 68
IEEE Circuits and Systems Magazine - Q2 2022 - Cover3
IEEE Circuits and Systems Magazine - Q2 2022 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q1
https://www.nxtbookmedia.com