Deep Learning (DL) performs classification tasks using a series of layers. To effectively execute these tasks, local decisions are performed progressively along the layers. But can we perform an all-encompassing decision by choosing the most influential path to the output rather than performing these decisions locally?
In an article just published in Scientific Reports (Enhancing the accuracies by performing pooling decisions adjacent to the output layer), researchers from Bar-Ilan University in Israel answer this question with a resounding “yes”. Pre-existing deep architectures have been improved by updating the most influential paths to the output.
“One can think of it as two children who wish to climb a mountain with many twists and turns. One of them chooses the fastest local route at every intersection while the other uses binoculars to see the entire path ahead and picks the shortest and most significant route, just like Google Maps or Waze. The first child might get a head start, but the second will end up winning,” said Prof. Ido Kanter, of Bar-Ilan’s Department of Physics and Gonda (Goldschmied) Multidisciplinary Brain Research Center, who led the research.
“This discovery can pave the way for better enhanced AI learning, by choosing the most significant route to the top,” added Yarden Tzach, a PhD student and one of the key contributors to this work.
This exploration of a deeper comprehension of AI systems by Prof. Kanter and his experimental research team, led by Dr. Roni Vardi, aims to bridge between the biological world and machine learning, thereby creating an improved, advanced AI system. To date they have discovered evidence for efficient dendritic adaptation using neuronal cultures, as well as how to implement those findings in machine learning, showing how shallow networks can compete with deep ones, and finding the mechanism underlying successful deep learning.
Enhancing existing architectures using global decisions can pave the way for improved AI, which can improve its classification tasks without the need for additional layers.
