Revolutionising image processing with photonics
The last few years have seen a staggering increase in the demand for computing power for cognitive image and video processing. To achieve improved performance in this area, scientists have focused on convolutional neural networks (CNNs), a type of network architecture for deep learning algorithms used for image recognition and pixel data processing tasks. However, while CNNs offer improved performance, they also consume much more power and memory, which is why researchers have turned to photonics as a way to enhance CNNs. A new study supported by the EU-funded NEoteRIC and PROMETHEUS projects now sheds some light on the rapidly changing landscape of integrated photonic neuromorphic architectures for the implementation of CNNs. The study was published in the journal ‘Intelligent Computing’. CNNs are designed to automatically learn hierarchical representations from input data, and the deeper they are made and the more trainable parameters they are given, the better they perform. However, as explained in a ‘EurekAlert!’ news release, this improvement comes at the cost of significantly greater power consumption and memory requirements. The attempt to solve this problem using multiple chip and parallel processing further increases energy usage. This “raises concerns both in terms of financial costs and ecological impact when scaling up the systems.” The solution lies in photonics, with its capacity to harness the properties of light for enhanced data transmission and processing. The study provides an overview of current integrated photonic CNNs that tackle the demanding field of ultrafast image processing. It analyses photonic cores operating as CNNs, “covering either the functionality of a conventional neural network or its spiking counterpart.”
Different perspectives
The review also presents two alternative photonic approaches that do not simply transfer neural network concepts directly into the optical domain but instead offer a different perspective in this rapidly changing field. These two approaches combine photonic, digital electronic and event-based bio-inspired processing, making the most of their respective advantages. “These approaches can offer beyond state-of-the-art performance while relying on realistic, scalable technology,” the study reports. The first approach is based on a photonic integrated platform and an optical spectrum–slicing technique. It does away with complicated circuits or image preprocessing, and uses special filters that separate the image into different parts based on their colours and patterns and then extract important features from the image. The news release explains: “By using this approach, the machine becomes scalable, meaning it can handle larger and more complex images. This method consumes very little power, as it only needs a small amount of energy for detecting the light and processing the signals. It also works instantly, without any delay, so it can process images in real-time.” The second approach follows a bio-isomorphic route combining miniaturised spiking laser neurons and unsupervised bio-inspired training in a deep architecture. “Laser neurons simulate the spiking behavior of biological neurons, providing robustness against noise. Unsupervised bioinspired training autonomously extracts meaningful features from data, enabling pattern recognition without explicit labels. Photonics-based information processing offers energy efficiency. By leveraging these technologies, the accelerator achieves noise resilience and reduced power consumption.” NEoteRIC (NEuromorphic Reconfigurable Integrated photonic Circuits as artificial image processor) ends in December 2023. PROMETHEUS (PROgraMmable integrated photonic nEuromorphic and quanTum networks for High-speed imaging, communications and sEcUrity applicationS) ends in August 2025. For more information, please see: NEoteRIC project website PROMETHEUS project website
Keywords
NEoteRIC, PROMETHEUS, photonics, processing, image processing, convolutional neural network
