Periodic Reporting for period 3 - EXIST (Extended Image Sensing Technologies)
Berichtszeitraum: 2017-05-01 bis 2018-12-31
• The Exist project addresses the need for improved and advanced image capturing techniques and further explores the potential of these. Applicable in multiple domains of the society like agriculture, healthcare, broadcast, security ...
• A first objective of the project is to increase the resolution of imaging systems. Higher resolution images give sharper detail in e.g. broadcast TV, in surveillance cameras, in industrial inspection cameras etc. Image sensors are built for 4K UHD imaging and a broadcast camera with this sensor was introduced on the market during this project. A large format 32Mpixel sensor has been built and experimented and a 14Mpixel sensor was built for industrial and security applications. All system components to support such high resolution were developed like high resolution zoom lenses, image processing SW for image improvement (de-noising, lens-aberration correction, demosaicing etc) and all HW to support the high-resolution image data streams.
• A second objective was to improve Time-of-flight imaging systems. A specific research domain is the development of an image capturing system that combines colour images and depth sensing to build compact systems (e.g. for VR and AR applications). SDS did build a dedicated image sensor with mosaic pixel layout. Commercially-available RGB filters were deposited onto the pixels with colour functionality and a NIR filter was deposited by IMEC, on the pixels with depth sensing functionality. Image data processing including novel demosaicing and colour correction was added. The resulting system has been successfully built and demonstrated in a VR game application.
• As a third objective, hyperspectral imaging was investigated and brought to a next level from technology to applications. Several application areas learned how to benefit of hyperspectral imaging by small systems based upon integrated image sensors. Hyperspectral image data capturing and processing were investigated in agriculture, medical, and security applications.
This research also indicated the limitations of hyperspectral imaging in terms of sensitivity in real application scenarios.
• As the last objective, a novel TDI image sensor has been built. This kind of imagers allows for image capturing at low light levels or at high speed and finds its typical application in e.g. space observation, inspection of e.g. pcb or flat panel A CCD-in-CMOS process is developed and improved. In this process a dedicated image sensor was designed to combine the benefits of CCD imaging with the on-chip read-out capabilities of CMOS to achieve a very fast TDI and even multispectral imager.
• A first objective of the project is to increase the resolution of imaging systems. Higher resolution images give sharper detail in e.g. broadcast TV, in surveillance cameras, in industrial inspection cameras etc. Image sensors are built for 4K UHD imaging and a broadcast camera with this sensor was introduced on the market during this project. A large format 32Mpixel sensor has been built and experimented and a 14Mpixel sensor was built for industrial and security applications. All system components to support such high resolution were developed like high resolution zoom lenses, image processing SW for image improvement (de-noising, lens-aberration correction, demosaicing etc) and all HW to support the high-resolution image data streams.
• A second objective was to improve Time-of-flight imaging systems. A specific research domain is the development of an image capturing system that combines colour images and depth sensing to build compact systems (e.g. for VR and AR applications). SDS did build a dedicated image sensor with mosaic pixel layout. Commercially-available RGB filters were deposited onto the pixels with colour functionality and a NIR filter was deposited by IMEC, on the pixels with depth sensing functionality. Image data processing including novel demosaicing and colour correction was added. The resulting system has been successfully built and demonstrated in a VR game application.
• As a third objective, hyperspectral imaging was investigated and brought to a next level from technology to applications. Several application areas learned how to benefit of hyperspectral imaging by small systems based upon integrated image sensors. Hyperspectral image data capturing and processing were investigated in agriculture, medical, and security applications.
This research also indicated the limitations of hyperspectral imaging in terms of sensitivity in real application scenarios.
• As the last objective, a novel TDI image sensor has been built. This kind of imagers allows for image capturing at low light levels or at high speed and finds its typical application in e.g. space observation, inspection of e.g. pcb or flat panel A CCD-in-CMOS process is developed and improved. In this process a dedicated image sensor was designed to combine the benefits of CCD imaging with the on-chip read-out capabilities of CMOS to achieve a very fast TDI and even multispectral imager.
• High resolution image sensors have been built by Grass Valley and a 4K imager has been designed into a broadcast camera that is available on the market now. High resolution zoom lenses were designed and built by Thales-Angenieux for high-end security and broadcast applications and will be industrialized hereafter. AMS designed and produced a next generation high resolution image sensor for industrial vision and presented this at Vision 2018. High resolution imaging applications are supported by demosaicing, denoising and chromatic aberration correction SW developed in this project by Imec and Le2i. Adimec developed cameras for 4K UHDTV in small outline with low power and improved image processing for security applications addressing the market in a few years from now.
• Sony Depth Sensing Solutions, in cooperation with Imec, developed a new generation VR system and an application in gaming.
• Adimec in cooperation with Imec did build a compact size hyperspectral camera that was used by e.d. Demcon to enrich their portfolio of SW to evaluate hyperspectral data for applications in forensic diagnostic tools and industrial sorting.
Quest-Innovations did build an RGB/VIS/NIR camera on a surgical imaging system for laparoscopy.
Based on this camera, a demonstration system was built by TNO that classifies veins on a person’s arm to demonstrate feasibility of live tissue classification.
Ex-vivo tissue was used to generate an annotated dataset to train a classification algorithm to demonstrate the camera’s performance for classification of various tissue types, such as fat, muscle, veins, arteries, and organ sections.
• Le2i and Silios did build a sensor and one-shot multispectral camera for control and security access that will come to exploitation in a few years.
• Imec optimized the CCD-in-CMOS technology and finalized the development of a fast 7 band TDI imager.
Imec did build a demonstration system to show the capabilities of this technology and this novel imager and showed this at Vision 2018 and at Photonics West 2019. Imec offers this imager as evaluation samples today and is open to accept similar TDI imager design requests.
• Sony Depth Sensing Solutions, in cooperation with Imec, developed a new generation VR system and an application in gaming.
• Adimec in cooperation with Imec did build a compact size hyperspectral camera that was used by e.d. Demcon to enrich their portfolio of SW to evaluate hyperspectral data for applications in forensic diagnostic tools and industrial sorting.
Quest-Innovations did build an RGB/VIS/NIR camera on a surgical imaging system for laparoscopy.
Based on this camera, a demonstration system was built by TNO that classifies veins on a person’s arm to demonstrate feasibility of live tissue classification.
Ex-vivo tissue was used to generate an annotated dataset to train a classification algorithm to demonstrate the camera’s performance for classification of various tissue types, such as fat, muscle, veins, arteries, and organ sections.
• Le2i and Silios did build a sensor and one-shot multispectral camera for control and security access that will come to exploitation in a few years.
• Imec optimized the CCD-in-CMOS technology and finalized the development of a fast 7 band TDI imager.
Imec did build a demonstration system to show the capabilities of this technology and this novel imager and showed this at Vision 2018 and at Photonics West 2019. Imec offers this imager as evaluation samples today and is open to accept similar TDI imager design requests.
• The results in high resolution imaging for ultra HD in broadcast represented for each of the building blocks a step beyond the state of the art and will soon be visible to the society in live television productions with improved resolution and image quality.
• The results in Si process development for CCD-in-CMOS and of the novel TDI image sensor are beyond the state of the art. This technology is replacing several traditional CCD components or will be used where traditional CCD components cannot reach the required speeds or complexity. The ideas for the process and imager are today taken over by at least 2 major players in the industry for applications in industrial inspection.
• The process technologies for multi spectral filters by colour shading or for hyperspectral filters on a BSI imager made significant process in this project. The project also enabled a number of partners to work in the new domain of compact hyperspectral imaging systems and to learn about the directions to be taken for future research work in the domain of medical care, forensic, agriculture and security applications.
• The results in Si process development for CCD-in-CMOS and of the novel TDI image sensor are beyond the state of the art. This technology is replacing several traditional CCD components or will be used where traditional CCD components cannot reach the required speeds or complexity. The ideas for the process and imager are today taken over by at least 2 major players in the industry for applications in industrial inspection.
• The process technologies for multi spectral filters by colour shading or for hyperspectral filters on a BSI imager made significant process in this project. The project also enabled a number of partners to work in the new domain of compact hyperspectral imaging systems and to learn about the directions to be taken for future research work in the domain of medical care, forensic, agriculture and security applications.