IEEE Consumer Electronics Magazine - July/August 2023 - 100

Special Section on Consumer Healthcare Technologies in Smart Cities
FIGURE 6. Circuit design of the self-attention module.
sequential computation in the LSTM network.24
The self-attentionmechanism can be described as
mapping a query and a set of key value pairs,
where the query (Q), keys (K), values (V), and output
are all vectors, as shown in the bottom left of
Figure 6. Notably, unlike the LSTM module, the
input of the self-attention module is voltage, while
the output of the self-attentionmodule is current.
From Figure 6, x2RNd is the input matrix,
parameter N and parameter d are the sequence
length and the hidden dimension, respectively.
Connection weights (Wq, Wk, Wv,and W) 2RN are
all learnable parameters. Following the weights
(Wq, Wk,and Wv), the input matrix x can be converted
into the attention query, key, and value
matrices Q, K,and V2RNd.Inthiswork, we compute
the additive attention between each query
vector and each key vector and apply a SoftMax
function to normalize the additive attention scores.
Then, we compute the dot product of the value
matrix Vand the corresponding normalized attention
score as the output.
Similarly, the self-attention mechanism can
also be characterized by two main phases, i.e., the
similarity computation and the dot-product operation.
Based on this, a self-attention circuit consists
of two subcircuits, i.e., the similarity operation
circuit and the dot-product operation circuit
are designed in Figure 6. To better illustrate the
circuit design scheme in Figure 6, the relevant circuit
analysis is provided as follows.
The similarity computation circuit: the similarity
computation circuit consists of three albumen protein
memristor crossbar arrays, a tanh circuit, and
a SoftMax circuit. The memristor crossbar arrays
are used to compute and store the parameters of
each row in the key matrix K, the query matrix Q,
and the matrix W, respectively. The tanh circuit
can directly use the above-mentioned activation
circuit, realizing the nonlinear activation function.
The SoftMax circuit is responsible for the normalization
operation.25
The dot-product operation circuit: the dotproduct
operation circuit consists of an albumen
protein memristor crossbar array and several
multiplying units. The albumen protein memristor
crossbar array is used to compute and store
the parameters of each row in the value matrix
V, and multiplying units compute the weighted
sum of rows in value matrix V and the corresponding
SoftMax circuit outputs, i.e., the final
outputs of the self-attention circuit.
100
IEEE Consumer Electronics Magazine

IEEE Consumer Electronics Magazine - July/August 2023

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - July/August 2023

Contents
IEEE Consumer Electronics Magazine - July/August 2023 - Cover1
IEEE Consumer Electronics Magazine - July/August 2023 - Cover2
IEEE Consumer Electronics Magazine - July/August 2023 - Contents
IEEE Consumer Electronics Magazine - July/August 2023 - 2
IEEE Consumer Electronics Magazine - July/August 2023 - 3
IEEE Consumer Electronics Magazine - July/August 2023 - 4
IEEE Consumer Electronics Magazine - July/August 2023 - 5
IEEE Consumer Electronics Magazine - July/August 2023 - 6
IEEE Consumer Electronics Magazine - July/August 2023 - 7
IEEE Consumer Electronics Magazine - July/August 2023 - 8
IEEE Consumer Electronics Magazine - July/August 2023 - 9
IEEE Consumer Electronics Magazine - July/August 2023 - 10
IEEE Consumer Electronics Magazine - July/August 2023 - 11
IEEE Consumer Electronics Magazine - July/August 2023 - 12
IEEE Consumer Electronics Magazine - July/August 2023 - 13
IEEE Consumer Electronics Magazine - July/August 2023 - 14
IEEE Consumer Electronics Magazine - July/August 2023 - 15
IEEE Consumer Electronics Magazine - July/August 2023 - 16
IEEE Consumer Electronics Magazine - July/August 2023 - 17
IEEE Consumer Electronics Magazine - July/August 2023 - 18
IEEE Consumer Electronics Magazine - July/August 2023 - 19
IEEE Consumer Electronics Magazine - July/August 2023 - 20
IEEE Consumer Electronics Magazine - July/August 2023 - 21
IEEE Consumer Electronics Magazine - July/August 2023 - 22
IEEE Consumer Electronics Magazine - July/August 2023 - 23
IEEE Consumer Electronics Magazine - July/August 2023 - 24
IEEE Consumer Electronics Magazine - July/August 2023 - 25
IEEE Consumer Electronics Magazine - July/August 2023 - 26
IEEE Consumer Electronics Magazine - July/August 2023 - 27
IEEE Consumer Electronics Magazine - July/August 2023 - 28
IEEE Consumer Electronics Magazine - July/August 2023 - 29
IEEE Consumer Electronics Magazine - July/August 2023 - 30
IEEE Consumer Electronics Magazine - July/August 2023 - 31
IEEE Consumer Electronics Magazine - July/August 2023 - 32
IEEE Consumer Electronics Magazine - July/August 2023 - 33
IEEE Consumer Electronics Magazine - July/August 2023 - 34
IEEE Consumer Electronics Magazine - July/August 2023 - 35
IEEE Consumer Electronics Magazine - July/August 2023 - 36
IEEE Consumer Electronics Magazine - July/August 2023 - 37
IEEE Consumer Electronics Magazine - July/August 2023 - 38
IEEE Consumer Electronics Magazine - July/August 2023 - 39
IEEE Consumer Electronics Magazine - July/August 2023 - 40
IEEE Consumer Electronics Magazine - July/August 2023 - 41
IEEE Consumer Electronics Magazine - July/August 2023 - 42
IEEE Consumer Electronics Magazine - July/August 2023 - 43
IEEE Consumer Electronics Magazine - July/August 2023 - 44
IEEE Consumer Electronics Magazine - July/August 2023 - 45
IEEE Consumer Electronics Magazine - July/August 2023 - 46
IEEE Consumer Electronics Magazine - July/August 2023 - 47
IEEE Consumer Electronics Magazine - July/August 2023 - 48
IEEE Consumer Electronics Magazine - July/August 2023 - 49
IEEE Consumer Electronics Magazine - July/August 2023 - 50
IEEE Consumer Electronics Magazine - July/August 2023 - 51
IEEE Consumer Electronics Magazine - July/August 2023 - 52
IEEE Consumer Electronics Magazine - July/August 2023 - 53
IEEE Consumer Electronics Magazine - July/August 2023 - 54
IEEE Consumer Electronics Magazine - July/August 2023 - 55
IEEE Consumer Electronics Magazine - July/August 2023 - 56
IEEE Consumer Electronics Magazine - July/August 2023 - 57
IEEE Consumer Electronics Magazine - July/August 2023 - 58
IEEE Consumer Electronics Magazine - July/August 2023 - 59
IEEE Consumer Electronics Magazine - July/August 2023 - 60
IEEE Consumer Electronics Magazine - July/August 2023 - 61
IEEE Consumer Electronics Magazine - July/August 2023 - 62
IEEE Consumer Electronics Magazine - July/August 2023 - 63
IEEE Consumer Electronics Magazine - July/August 2023 - 64
IEEE Consumer Electronics Magazine - July/August 2023 - 65
IEEE Consumer Electronics Magazine - July/August 2023 - 66
IEEE Consumer Electronics Magazine - July/August 2023 - 67
IEEE Consumer Electronics Magazine - July/August 2023 - 68
IEEE Consumer Electronics Magazine - July/August 2023 - 69
IEEE Consumer Electronics Magazine - July/August 2023 - 70
IEEE Consumer Electronics Magazine - July/August 2023 - 71
IEEE Consumer Electronics Magazine - July/August 2023 - 72
IEEE Consumer Electronics Magazine - July/August 2023 - 73
IEEE Consumer Electronics Magazine - July/August 2023 - 74
IEEE Consumer Electronics Magazine - July/August 2023 - 75
IEEE Consumer Electronics Magazine - July/August 2023 - 76
IEEE Consumer Electronics Magazine - July/August 2023 - 77
IEEE Consumer Electronics Magazine - July/August 2023 - 78
IEEE Consumer Electronics Magazine - July/August 2023 - 79
IEEE Consumer Electronics Magazine - July/August 2023 - 80
IEEE Consumer Electronics Magazine - July/August 2023 - 81
IEEE Consumer Electronics Magazine - July/August 2023 - 82
IEEE Consumer Electronics Magazine - July/August 2023 - 83
IEEE Consumer Electronics Magazine - July/August 2023 - 84
IEEE Consumer Electronics Magazine - July/August 2023 - 85
IEEE Consumer Electronics Magazine - July/August 2023 - 86
IEEE Consumer Electronics Magazine - July/August 2023 - 87
IEEE Consumer Electronics Magazine - July/August 2023 - 88
IEEE Consumer Electronics Magazine - July/August 2023 - 89
IEEE Consumer Electronics Magazine - July/August 2023 - 90
IEEE Consumer Electronics Magazine - July/August 2023 - 91
IEEE Consumer Electronics Magazine - July/August 2023 - 92
IEEE Consumer Electronics Magazine - July/August 2023 - 93
IEEE Consumer Electronics Magazine - July/August 2023 - 94
IEEE Consumer Electronics Magazine - July/August 2023 - 95
IEEE Consumer Electronics Magazine - July/August 2023 - 96
IEEE Consumer Electronics Magazine - July/August 2023 - 97
IEEE Consumer Electronics Magazine - July/August 2023 - 98
IEEE Consumer Electronics Magazine - July/August 2023 - 99
IEEE Consumer Electronics Magazine - July/August 2023 - 100
IEEE Consumer Electronics Magazine - July/August 2023 - 101
IEEE Consumer Electronics Magazine - July/August 2023 - 102
IEEE Consumer Electronics Magazine - July/August 2023 - 103
IEEE Consumer Electronics Magazine - July/August 2023 - 104
IEEE Consumer Electronics Magazine - July/August 2023 - 105
IEEE Consumer Electronics Magazine - July/August 2023 - 106
IEEE Consumer Electronics Magazine - July/August 2023 - 107
IEEE Consumer Electronics Magazine - July/August 2023 - 108
IEEE Consumer Electronics Magazine - July/August 2023 - Cover3
IEEE Consumer Electronics Magazine - July/August 2023 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20240102
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20231112
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230708
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230506
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230304
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230102
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20221112
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220708
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220506
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220304
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220102
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20211112
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210708
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210506
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210304
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202010
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202009
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202007
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202004
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202003
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202001
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201909
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201907
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201905
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201903
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201901
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201811
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201809
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201807
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201805
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201803
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_july2017
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_april2017
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_january2017
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_october2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_july2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_april2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_january2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_october2015
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_july2015
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_april2015
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_january2015
https://www.nxtbookmedia.com