sdadsas

1. Zhengzhong Guan
IoT Thrust
Information Hub
Neuromorphic Wireless Cognition: Event-Driven Semantic
Communications for Remote Inference
Jiechen Chen, Nicolas Skatchkovsky , Osvaldo Simeone

2. Motivation and Main Contributions
Zhengzhong Guan
IoT Thrust
Information Hub
Conventional deployment for remote inference
• Rely on digital frame-based sensing, processing, and computing blocks
• Drawbacks ： (i) Semantic-agnostic energy consumption
(ii) Latency quantization
Neuromorphic sensors, SNNs, and impulse radio blocks consume energy only
when spikes are produced which makes the technology are energy efficiency,
native event-driven processing of time-varying semantic sources
• Introduce NeuroComm, a novel architecture for wireless cognition
integrating neuromorphic sensing, processing, and communications.
• propose an end-to-end design of the NeuroComm architecture based
on supervised learning via surrogate gradient descent methods.
• In order to enable adaptation to channel conditions, the authors
introduce a hypernetwork-based approach.
• Extensive numerical results are provided that demonstrate the
advantages of the proposed architecture and design over
conventional frame-based digital solutions
Motivation Main Contributions
Neuromorphic sensing for remote inference
• Spikes are produced only when relevant changes occur in the signals
being sensed
• Innovations ： (i) Semantic-aware energy consumption
(ii) Enhanced time-to-efficiency

3. System Model
Zhengzhong Guan
IoT Thrust
Information Hub
A block diagram illustrating the main operations involved in the implementation of the NeuroComm system
• 1 receiver, K devices, each device equipped with devices.
• Discrete-time model: l= 1,2,….L
• Device output: 1 vector
u
D  k
l
u
Each binary entry of the vector represents the presence , or the absence
of a spike in the spiking signal at time .
B. Channel model
A. Setting
• General multi-path propagation model
• Continuous channel response:
1. is the total number of resolvable propagation paths
2. and represent the complex amplitude and delay of the th path
3. represents the matched filter response
4. Maximum delay spread of all channels is denoted as ,
5. A frame is assumed to be sufficiently long to encompass the presentation of
input signals of samples, ,
 
, , , , ,
1
( )
P
N
k k k
n m n m p n m p
p
h t g t
 

 


4. System Model
Zhengzhong Guan
IoT Thrust
Information Hub
D. Pilot Transmission
C. Transmission Model
,
1 1
( ) ( )* ( ) ( )
T
K N
k k
n m n m n
k m
y t s t h t z t
 
 

1. is the modulated signal emitted by the th antenna of the impulse radio
transmitter
2. and is assumed to have bandwidth ,
3. The overall duration of a transmission frame as , where integer is referred
to as the bandwidth expansion factor. Since is the number of samples
produced by the sensors per frame, when sampled at Nyquist rate , the
transmitted and received signals present samples for each sample of the
sensed signals.
4. is the additive white Gaussian noise with power spectral density
 
, , , ,
1 1
T
K N T
k k
i n n m i m i n
k m
y s z
 
 
 h
Sampling signal at rate
1. Effective discrete-time channels have taps
2. The channel between antenna of device and antenna of the receiver is
described by the vector
3. vector ,
4.Power constrain:
2
fr
2
, s
,
,
k k
k k
i m
E
s E


s
 
, , , ,
1
T
N T
k k k
i n n m i m i n
m
y s z

 
 h
1. Device transmits symbols at rate from each antenna.
2. Defined the vector , ,
and if
3. Power constrain
2
p p
k k
s E


5. Conventional Frame-based Digital Transmission
Zhengzhong Guan
IoT Thrust
Information Hub
A. Separate Source-Channel Coding
The samples produced by the neuromorphic sensor are first divided into subframes.
B. Inference
vector vector
ANN
C. Training
   
,
1
ˆ
, log
v
D
f j j f
j
L v v

 
v v

 
 
( , ) 1
min , .
F
f
f
L G
 
 
 
u v
u v
D
 
 
( , ) 1
,
F
f
f
L G
 
  
 
  
 
u v
u v
B
Assume access to a training data set containing input-output pairs , where
collects all the samples sensed by the devices and is the corresponding target
one-hot variable. The sensed signal is formatted into subframes , with .
The loss between the probability vector and the true target is measured by the
cross-entropy loss,
select the model parameter vector based on the training example by addressing the
empirical risk minimization (ERM) problem:
The model parameters can be updated based on stochastic gradient
descent (SGD)-based rule
´
𝒗 𝑓 =𝐺𝜙 (^
𝒖𝑓 )
ANN that takes as input the per-subframe
estimated sensed signals produced by the
source decoder for all to output an
estimate of the target vector. The
decisions produced for each subframe
are then combined to output a final
decision , e.g., via a majority rule.

6. NeuroComm System
Zhengzhong Guan
IoT Thrust
Information Hub
A. Neuromorphic Encoding
B. Impulse Radio Transmission
The impulse radio block takes as input the spiking signals
produced by the encoding SNN, and possibly by TH modulation,
and it produces the modulated signals
At time step for device :
vector vector
Encoding SNN
Two way to implement:
1) TH Modulation
Each sample is expanded into samples , for ,
by randomly adding a discrete-time offset to
the timing of each spike
2) LTH Modulation
Expand the sensed signal for each time instant
by adding zero samples before feeding the
sequence of samples to the encoding SNN

7. NeuroComm System
Zhengzhong Guan
IoT Thrust
Information Hub
C. Neuromorphic Decoding D. Spiking Neural Network Model
• Fully connected
• Encoding SNN: layers
• Decoding SNN: layers
• layer has neurons
• is the weight matrix between layer and layer
K
L
N
R
L
N
 
, , , , ,
* *
k l k j l j l l k l
j
o w b b
 

  



N
Membrane potential for neuron at time step
 
, ,
k l k l
b o 
 
Neuron outputs a spike at time step when its membrane potential passes
some fixed threshold ,
• is the element of matrix
• represents the synaptic response to a spike from the presynaptic
neurons to a postsynaptic neuron
• describes the synaptic response to the spike emitted by the neuron
itself
• " " is the convolution operator
• is the output of neuron at time step
At time step :
• Received signals at antenna:
• Since is generally complex, define the vector
Then vector
SNN decoder:
,
l n
y
𝒚𝑙 ,𝑛=[ℜ (𝒚𝑙, 𝑛)
𝑇
,𝔖 (𝒚𝑙, 𝑛)
𝑇
]
𝑇
,
,
l n
y
,1 ,
, , R
T
T T
l l l N
 
 
 
 
y y y
vector vector
SNN
…
1, ,
l l
 
 
 
V v v
Rate coding + softmax
 
softmax
l l
 
v V
 1

8. Enabling Adaptation via Hyper-NeuroComm
Zhengzhong Guan
IoT Thrust
Information Hub
received pilot signals into a vector and take the vector as input to the
hypernetwork.
Allow for the weights of the decoding SNN to adapt to the
current channel condition by leveraging the received pilots
vector
ANN
 
L
1 , , R
R R
N

w w
each vector defining the scaling of the weights applied by the
neurons in the th layer.
  
diag ,
R
R R
 
W W w

 
write for the collection of all matrices , and the overall mapping
implemented by the hypernetwork as
where are the weights of the hypernetwork.

9. Enabling Adaptation via Hyper-NeuroComm
Zhengzhong Guan
IoT Thrust
Information Hub
Training Problem
In order to compute the gradient, the authors follow the
surrogate gradient approach by replacing the Heaviside step
function with a differentiable surrogate function
Parameters need to be optimized:
• the model parameters of the encoding SNNs
• the weights of the decoding SNNs
• the weights of the hypernetwork
• pilot sequences
• write as
the training objective for Hyper-NeuroComm is defined as
 
( , )
min ,
L
L

 
 

θ
u v
v v

E
D
 
1
( ) 1 x
x e



 