BAM-SLDK: Biologically inspired Attention Mechanism with Spiking Learnable Delayed Kernel synapses
IOP Neuromorphic Computing and Engineering, 2025
Spiking Neural Networks (SNNs) are emerging as an alternative neural network model due to their biological plausibility, energy efficiency, and built-in ability to learn from temporal dynamics. However, in order to effectively process data with rich spatial and temporal dependencies, the usual static projections (feedforward and recurrent) among layers of spiking neurons fail to represent all the information needed. Inspired by how synaptic delays affect the learning process in biological neurons, in this paper, we propose a biologically inspired attention mechanism based on spiking convolutions with learnable delayed kernel synapses. The proposed model increases temporal learning ability, attending simultaneously to spatial and temporal dynamics with few parameters required.