Quick News Bit

Mesoscale neural plasticity helps in AI learning

0
Mesoscale neural plasticity helps in AI learning
Fig. 1. Introducing biological SBP into SNNs.(A) Schematic diagram depicting the BP of potentiation (“+”) or depression (“−”) from the synapses at the output layer to those at the hidden layer in a three-layer network. The propagated synaptic modification had the same sign, consistent with the biological discovery of SBP (27). Similar configurations of fixed gradient mapping between neighborhood layers exist in artificial feedback alignment (48) and direct target propagation (49). (B) For a three-layer SNN, the induction of potentiation (+) or depression (−) occurred at the synapse Wj, k on the output neuron by STDP, based on the timing of presynaptic spikes (in the hidden neuron) relative to the postsynaptic spikes in the output neuron, after updating by mean square error (MSE) of network-generated (Out2) and teaching spike trains (Teaching2). Wi, j and Wj, k represent synaptic weights of connections onto hidden and output neurons, respectively. The + and − signals created at synapses of hidden layer neuron (pink) onto an output neuron (blue) were allowed to spread to a percentage factor λp (λp∈ [10%,100%]) of input synapses with a fraction factor λf (λf∈ [0.1,1]) of the signals generated by the STDP (orange). Vj and Vk are membrane potentials at hidden and output layers, respectively. (C) The three-layer architecture of SNN, in which SBP and local plasticity (STP, STDP, and homeostatic V adjustment) were introduced at synapses at hidden and output layers, and the teaching spike train was given to the output LIF neurons. The diagram illustrates an output neuron inducing STDP (blue), a hidden neuron with the output synapse inducing STDP (pink), and input neurons with synapses receiving SBP (yellow). Credit: DOI: 10.1126/sciadv.abh0146

A joint research team led by Xu Bo from the Institute of Automation and Mu-Ming Poo from the Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, have discovered that self-backpropagation, a form of mesoscale synaptic plasticity rule in natural neural networks, can elevate the accuracy and reduce the computational cost of spiking neural networks (SNNs) and artificial neural networks (ANNs).

Their findings were published in Science Advances on Oct. 20.

Previous studies proved that self-backpropagation (SBP) is caused by the rapid retrograde axonal transport of molecular signals. It is considered to be the key for efficient and flexible learning of biological neural networks.

The backpropagation (BP) algorithm in artificial neural networks uses a global strategy for optimization, which can achieve excellent performance but, at the same time, take too much computational cost during learning.

The researchers introduced biologically plausible SBP into a three-layer SNN. They found an elevated accuracy of network performance in three standard benchmark tasks, MNIST, NETtalk, and DvsGesture.

“The computational cost in terms of the product of mean epochs and algorithmic complexity per epoch was markedly reduced,” said Xu. Similar results were obtained by further applying SBP on Restricted Boltzmann Machine.

According to the study, SBP is a special mesoscale biological plasticity mechanism, indicating a similar important role of SBP in SNNs compared to its counterpart BP in ANNs.

This will attract attention in the field of machine learning because training SNNs with pure biologically plausible algorithms (e.g., spike-timing-dependent plasticity) is difficult, in which the information is spatio-temporal and carried by discontinuous spikes.

The study has paved a way towards biologically plausible effective learning on both SNNs and ANNs, with high accuracy and low computational cost for learning different tasks.


Brain-on-a-chip would need little training


More information:
Tielin Zhang et al, Self-backpropagation of synaptic modifications elevates the efficiency of spiking and artificial neural networks, Science Advances (2021). DOI: 10.1126/sciadv.abh0146

Provided by
Chinese Academy of Sciences


Citation:
Mesoscale neural plasticity helps in AI learning (2021, October 22)
retrieved 22 October 2021
from https://techxplore.com/news/2021-10-mesoscale-neural-plasticity-ai.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.

For all the latest Technology News Click Here 

 For the latest news and updates, follow us on Google News

Read original article here

Denial of responsibility! NewsBit.us is an automatic aggregator around the global media. All the content are available free on Internet. We have just arranged it in one platform for educational purpose only. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials on our website, please contact us by email – [email protected]. The content will be deleted within 24 hours.

Leave a comment