Final Report Summary - SKINSPECTION (MULTIMODAL SKIN INSPECTION WITH HYBRID ACOUSTIC & OPTICAL SPECTROSCOPIC IMAGING)
Executive Summary:
The SKINSPECTION consortium consists of six European partners from United Kingdom, Italy and Germany. The concept is based on a multimodal imaging approach combining multiphoton tomography, fluorescence lifetime imaging, optoacoustic imaging and high resolution ultrasound. The combination of theses modalities allows in-vivo non-invasive imaging of skin lesions both on the macroscopic and the microscopic level. Ultrasound and optoacoustic imaging allow a wide-field view of structures with special focus to subcutaneous microvasculature and morphology of lesions. While ultrasound is established in several clinical fields, optoacoustic imaging (OA) is a new technique based on the generation of broadband ultrasound signals by absorption of short laser pulses. OA therefore combines beneficious features of ultrasound imaging (low scattering, high image depth) with advantages of optics such as high contrast. These modalities provide images down to 3 mm in depth with a sub-100µm resolution. In addition, multiphoton tomography and FLIM (Fluorescence Lifetime Imaging Microscopy) give access to subcellular structures from the upper skin layers. Imperial College London has developed a FLIM module for integration in a multiphoton tomograph from the German company JenLab GmbH. The ultrasound based modalities are implemented by the German partners kibero GmbH and Fraunhofer IBMT. After 3 years of development and certification for clinical testing, the evaluation of the platform was performed in a bicentric clinical study at Hammersmith Hospital London and Universita di Modena.
While high resolution ultrasound is established in diagnosis of skin diseases, the optoacoustic imaging module developed by Fraunhofer IBMT and kibero has allowed the first evaluation of optoacoustic technology for investigation of skin lesions under clinical conditions. Further, the optical SKINSPECTION modalities give access to in-vivo 3D biopsies of skin with ultrahigh subcellular resolution. The multiphoton technology from Jenlab in combination with Imperials FLIM module have allowed to provide clinicians with a whole range of new diagnostic parameters, allowing to identify specific morphological or FLIM features from cells that can be associated to defined skin diseases.
The clinical evaluation phase has shown that the occurrence of defined cells features that can be extracted from the data acquired with the new platform can be correlated to specific diseases. In total, more than 700 lesions were imaged with FLIM and multiphoton tomography and a statistical analysis was performed with respect to the sensitivity and specificity of defined cell features for accurate diagnosis of skin diseases such as BCC or melanoma. For instance, it could be shown that FLIM provides excellent contrast between healthy skin and basal cell carcinoma. The characteristic FLIM signal further allows sensitive differentiation between lesion types such as BCC and melanoma. The developed US/OA platform has further shown to be ideally suitable for high resolution non-invasive imaging of subcutaneous vasculature, with sensitivity and contrast enhanced when compared to existing methods.
Project Context and Objectives:
The incidence of skin cancer in Europe, US, and Australia is rising rapidly. One in five will develop some form of skin cancer during the lifetime. A person has a 1:33 chance to develop melanoma, the most aggressive skin cancer. Melanoma is the second most common cancer in women aged 20-29, and the sixth most common cancer in men and women. In 2007, more than 1 million new cases will be diagnosed in the US alone. About 90% of skin cancers are caused by ultraviolet (UV) sun light. The World Health Organization estimates that 60,000 people will die this year from too much sun: 48,000 from melanoma and 12,000 from other skin cancer. Five sunburns double the risk of developing skin cancer. Actinic keratosis is a UV induced intraepidermal neoplasia which affects by now millions of Europeans. Nearly 10% of the cases transform into an aggressive invasive squamous cell carcinoma.
The incidence of melanoma has increased nearly 700% and the overall mortality rate by 165% from 1950 to 2001 (American Society of Dermatology: 2007 Skin Cancer Facts). The increase will be even more dramatic in the near future due to the man-made changes in climate and the increase of the UV hazard by ozone depletion, respectively, as well as by the aging population. Australia with the most drastic ozone problems reports on a significant increase of melanomas. The European Commission has identified the dangers of UV exposure from sunlight and tanning beds as a major health risk.
The five year survival rate for people with melanoma, basal cell carcinoma, and squamous cell carcinoma is higher than 95% if detected and treated early. However, most medical doctors are still using visual inspection as the only diagnostic method of skin cancer. If suspicious, biopsies are taken invasively from which histological 7 µm thick sections are prepared, then stained, and finally analyzed by pathologists with visual inspection under a light microscope. Several 10,000 biopsies are investigated annually in a typical hospital. The direct annual costs of diagnosis and treatment of skin cancer are several billion Euro.
A significant improvement of the current diagnostic tools of dermatologists is required in order to identify dermal disorders at a very early stage as well as to monitor directly the effects of treatment.
A very first step in this direction was done with the introduction of dermoscopy based on digital 2D images using CCD cameras. However, this technology does not provide 3D information and no sufficient resolution. Furthermore, it is very difficult to distinguish between some innocuous or premalignant conditions (such as dysplastic naevi or actinic keratosis) and malignant counterparts.
Non-invasive high-resolution imaging systems with the capability to gain additional information from inside the skin have significant potential to address these problems. Several types of 3D imaging have been proposed for skin diagnosis out of which optical methods and acoustic methods are the most promising ones.
The most recent development in high resolution in vivo 3D skin imaging is clinical two-photon tomography based on NIR femtosecond laser and flying-spot technology. A first commercial CE marked clinical multiphoton tomography is on the market and used e.g. by a clinical research group in Brisbane/Australia including Ian Frazer, the inventor of the first anticancer vaccine, to obtain information on intraskin cancer cells based on the detection of fluorescent endogenous biomolecules such as melanin. This first clinical femtosecond laser diagnostic tool was developed by the SME JenLab GmbH, a member of this consortium. The novel tomography however has the limitations of being able to inspect only small sub-mm3 skin lesions and of having a pure qualitative signal. Further wide-field imaging tools in combination with this novel multiphoton technology are required to get a full 3D view of the skin lesion and to enable also a closer high-resolution look into particular suspicious areas. In addition, photon statistics is needed to provide quantitative information.
Nowadays, the most often used clinical wide-field 3D imaging tool is based on ultrasound. However, clinical devices lack sufficient resolution to monitor intratissue cancer cell clusters.
Also optoacoustical imaging based on localized near infrared tissue excitation and the generation of ultrasound waves is a promising novel technology which can, in principle, provide information on vascularization and melanin content. However, no commercial clinical optoacoustical system exists.
So far, all single image modalities are more descriptive than quantitative, lack certain information or are limited in their field of view, the required resolution, and the clinical use.
In order to solve this problem, the SKINSPECTION consortium has developed a non-invasive multimodal hybrid imaging system with the capability to perform non-invasive high resolution three-dimensional clinical (i) two-photon imaging with time-correlated single photon detection, (ii) autofluorescence lifetime imaging, (iii) high-frequency acoustical imaging, and (iv) optoacoustical imaging using ultrashort near infrared (NIR) laser pulses. This novel multimodal approach will provide a wide-field acoustic/optoacoustic view with depth information of the dermatological lesion as well as a close optical look into particular intratissue compartments with subcellular resolution.
The goal of the project was to develop an imaging platform allowing the user to easily acquired both high resolution and low resolution images under clinical conditions. In addition, fluorescence lifetime imaging has been implemented to obtain a further method of contrast enhancement, to distinguish between different fluorescent biomolecules and more importantly, to provide a method of signal quantification by time-correlated single photon counting with picosecond temporal resolution.
Furthermore, functional imaging will be possible through highly selective imaging of vasculature by means of the newly developed optoacoustic imaging device.
The objectives of the SKINSPECTION project were therefore the development of a novel multimodal hybrid technology and the construction of a sophisticated non-invasive clinical image platform which combines ultrasound and optical features as well as quantitative technologies to visualize, to image and to detect cancerous tissue and intratissue cancer cells in skin.
The consortium consists of pioneers in the field of clinical multiphoton tomography, fluorescence lifetime imaging, high-frequency ultrasound and optoacoustic imaging, melanoma research, and evaluation of skin cancer data based on image processing.
Project Results:
In the context of the SKINSPECTION project, different technologies were developed, implemented into imaging systems, certified and tested in a clinical evaluation phase. The different technologies as well as the main clinical achievements are presented in the text that follows.
DermaInspect System Overview
The combined FLIM/MPT imaging device is based on the existing DermaInspect system from JenLab GmbH. This multiphoton base system is a femtosecond-laser scanning system which allows performing non-invasive in vivo optical biopsies of skin with ultrahigh subcellular resolution. The optical biopsies are based on measurements of two-photon autofluorescence and second harmonic generation (SHG). The nonlinearly induced autofluorescence originates from naturally existent endogenous fluorophores and protein structures of the skin such as NAD(P)H, flavins, elastin, porphyrins, and melanin. SHG which occurs in certain non-symmetric molecules can be used to detect collagen structures and to distinguish them from the previously mentioned fluorophores. The multiphoton base system can be equipped with detectors to perform fluorescence lifetime imaging (FLIM) and enables then 4-D-imaging (x,y,z,?) with subcellular spatial resolution. The system provides detailed information about cell and tissue structure to a depth of up to 200 ?m inside human skin. A major application of the multiphoton system is skin cancer diagnosis. Further applications include tissue engineering and in situ drug monitoring. The base system is equipped with interfaces for several FLIM detectors which can be used to measure signals at different spectral ranges simultaneously. In order to be able to include several photo multiplier tubes (PMTs) inside the scan-detector module a second level plate is currently designed and will be installed. Also, an interface to connect to state-of-the-art dermatoscopy systems (Fotofinder) is being developed.
The system integrates a light source used for the multiphoton base system (titanium:sapphire laser (MaiTai XF-1)) which generates sub 100-fs pulses in the red and near infrared wavelength region with a repetition rate of 80 MHz. The center wavelength of the pulses laser is tunable in the range of 710 nm and 920 nm.
The laser beam is coupled into the scan-detector module using special mirrors suited for NIR fs pulses. Within the scan-detector module the laser light is attenuated by a step-motor driven polarizer which can be controlled by the JENLAB software. Any unintended laser output is prevented by a safety shutter which opens during scanning operation only. After attenuation the laser pulses are directed onto a two-dimensional scanning module which consists of two temperature-controlled galvo-scanners. The scanning module deflects the laser beam in the x- and y-direction and directs the laser beam into the beam expander and the microscope objective. The scan-detector module contains a three-position-beam-splitter slider which allows holding any combination of beam splitters, mirrors or empty positions. Typically, the system allows excitation in the range between 700 nm and 1000 nm and detection in the blue and in the green/yellow spectral region. The microscope objective is connected to a piezo-driven module which allows to move the focusing optics over a range of 400 ?m with submicron precision. The typical imaging field is 250×250 ?m2.
The focused laser beam excites autofluorescence and induces SHG by certain skin components (collagen). This laser-induced radiation is collected and directed by the focusing optics through the beam splitter onto highly sensitive detectors.
Ultrasound and Optoacoustic system overview
The SKINSPECTION ultrasound and optoacoustic imaging system consists in a new development of the partners KIBERO and FRAUNHOFER. The specifications of the imaging platform were defined at the beginning of the SKINSPECTION project. To meet the requirements of clinical imaging of skin lesions, the system shall have the following properties:
- High frequency ultrasound signals
- 3D imaging
- Lateral scanning range > 6,4 mm x 6,4 mm
- Switchable between optoacoustic and ultrasound imaging
For allowing 3D optoacoustic and ultrasound imaging, different concepts were discussed. 3D imaging can be achieved by used of array technology or by using single element transducers combined with a mechanical scanning.
An array structure allows to move the ultrasound focus without moving the complete transducer. Like this, movement artefacts caused by the scanning of the complete transducer can be eliminated. The penetration depth into human skin and the lateral resolution are functions of the frequency. With increasing frequency the penetration depth decreases and the resolution increases. At 50 MHz the penetration depth is 4 mm and at 100 MHz the penetration depth is 2 mm. For defining the best-suited array geometry, different transducer soundfield simulations were run.
Opto acoustic applications need a transducer with a large acoustical bandwidth of maximal 75%. The bandwidth can be adjusted with a defined damping layer on the backside of the piezoelectric material, called backing. The magnitude of the photoacoustic waves is proportional to the local optical energy deposition. This mechanism fundamentally diverges from both the pure optical mechanism of signal generation (optical reflection, fluorescence) and pure acoustic mechanism (acoustic reflection, damping). The sample is irradiated with short laser pulses in the range of a few nanoseconds which corresponds to an ultrasound frequency response up to a cut-off frequency.
In order to have a transducer meeting the specifications and being usable both for ultrasound and optoacoustic imaging, different hardware strategies were investigated in the development phase of the project. 2D ultrasound arrays based on PZT, ZnO or PVDF were set-up and characterized as well as single focused PZT and PVDF transducers.
The preliminary tests suggested that a single focused transducer is better suited for the tasks of in-vivo combined optoacoustic and ultrasound imaging for the following reasons:
- Focused single element transducers have a higher sensitivity than arrays with comparably smaller aperture, which is especially of importance for optoacoustic imaging where signals are generated by low energy laser pulses
- Single element focused transducers are better suited for allowing an adequate light delivery (needed for optoacoustic signal generation)
For this reason, it was decided to build a single element transducer. Two versions have initially been developed:
a 30 MHz PZT transducer with a focal length of 250 µm and a focal width of 50 µm and a 70 MHz PVDF transducer with a focal length of 150 µm and a focal width of 30 µm. Both with a focal distance of 5.5 mm. The transducers are mounted in a scan unit allowing displacement in 3 dimensions in order to adjust the distance to the sample and to acquire threedimensional data sets. Both single element transducers are available for the integration into the acoustic detection unit. The acoustic performance of the PZT material is excellent, the acoustic performance of the PVDF transducer is acceptable. Due to the different acoustic performance of the active materials, it was decided to define the 30 MHZ PZT transducer as the primary system transducer with an option to use the 60 MHz PVDF transducer if the higher resolution is needed.
Optoacoustic hardware components:
In photoacoustic imaging, the detected acoustic signal correlates directly to the amount of absorbed energy. According to that the laser light should get mainly absorbed by the interesting structures and not by the main layers of the skin. Understanding how light interacts with skin assists in designing an optoacoustic imaging module. In medical physics, the optical window is the portion of the visible and infrared spectrum where living tissue absorbs only a small fraction of the light. This optical window approximately ranges from 650 nm to 1200 nm. At shorter wavelengths light is strongly absorbed by the hemoglobin in blood, while at longer wavelengths water strongly absorbs infrared light. Two different laser types are available working at 1064 nm and 532 nm wavelength. Imaging of microvasculature or the amount of blood in a lesion can be performed with a frequency-doubled Nd:YAG laser working at 532 nm. However, the penetration depth is limited. Working at 1064 nm the penetration depth is much higher due to the low absorption in tissue. Due to the availability we decided to use a 1064 Nd:YAG laser.
Photoacoustic probing of confined volumes can be achieved by several instrumental configurations. Using a multichannel photoacoustic system, signals are acquired by an unfocussed transducer array. A linear array needs a homogeneous diffuse illumination directly underneath the active area. In this case the entire observation volume is illuminated. For this scenario simulations of the light propagation in human skin based on a Monte Carlo model were made to evaluate possible designs of the light wave guide.
The depth of field is a function of the incident angle. In order to image structures near to the surface of the skin, the best performance was achieved at 45°. Nevertheless 70% of the energy gets absorbed within the first 2 millimeters underneath the surface of the skin. With the results of the simulation the specifications of the light fibre were deduced and a first prototype was designed. According to these results a first prototype for diffuse illumination using a transducer array as a detector was produced.
After simulating and building the prototype of the optical fibre illumination device, experiments for signal generation were performed in cooperation with the University of Modena. Unfortunately no photoacoustic signal could be detected with the used setup because the energy density was insufficient for optoacoustic signal generation. Higher intensity requires a solid state laser. However, flashlamp-pumped Q-switched lasers limit the data acquisition rate because of their fixed low pulse repetition rate (e.g. 10 Hz). Diode pumped solid state lasers based on the SLAB-concept are not subject to this limitation, but the size of such a laser limits the functional integrity.
An alternative to the wide field illumination is to focus the laser light. Using a focused single element transducer, the illuminated area was reduced in space to increase the light intensity and to avoid unnecessary illumination. The outcoupling angle of the light had to be modified in order to match the optical focus spot and the acoustic focus spot as accurately as possible.
Once the optical, mechanical and ultrasound components were defined, a first prototype of a handheld probe was developed and tested on phantoms at FRAUNHOFER. These tests were followed by a technical redesign which lead to a second version of the combined ultrasound/optoacoustic handheld probe.
Ultrasound electronics:
The electronics system has four distinctive parts: Computer, Ultrasound electronics, the laser module including the drive electronics, the transducers including mechanical scanner for 3-dimensional imaging. All components will be set up in one single housing.
Software for acquisition and display of acoustic and optoacoustic data:
The software for the data acquisition and the file export of the ultrasound and opto acoustical imaging modality is called skinspector 2010 and includes the hardware control skinterface. The tasks of the software include controlling the hardware, reading and handling the measured data including band pass filtering and storage of ultrasound RF-data. The user interface was programmed and is able to control the first simple region of interests in the B-Scan-Mode. Furthermore incoming data from the hardware system is converted and passed to the scan-conversion which includes envelope detection, requantization and the scan conversion itself. The scan conversion optimizes the images for better display quality. In this way B-Scan slices through the skin from the transducer into the depth can be imaged as greyscale image. The scan conversion of C-Scans using the measured data of a complete scan is shown in the application and displays slices through the skin with constant depth to the transducer. The spectrum of measured data and the line based data itself is displayed as graphs on the bottom and currently shown data can be selected using the mouse on the C-Scan Image.
The C-Scan display shows a slice perpendicular to the B-Scan on the left and can be used to visualize different slices in different depth parallel to the skin surface or to display a projection through the skin in direction of the sound propagation. The user can select different display modes (maximum intensity projection, average intensity projection, slice only visualization. The software further allows export of the signals. DICOM data will be exported as image stacks for 3d visualization in an external DICOM viewer and RF-data is stored for further signal processing and advanced imaging possibilities.
Software for ultrasound and optoacoustic image reconstruction
In the chosen concept, Data acquisition is based on a single element focusing ultrasound transducer which is scanned in x- and y- direction over the investigated sample. For this reason, the acoustic and optoacoustic data are acquired as a-scan signals with adjustable sample rate and ascan length. Depending on the selected scanning preset, the number of ascans per bscan can further be varied. For storage of the data, ascans are combined to bscans and are saved as such in the binary .grb format. All relevant scan parameters such as number of ascans or stepsize are also saved in the header of the grb file. For easy data exchange, a dicom converter has also been integrated in the Skinspector software platform.
The acoustic image reconstruction of scanned slices (B-Scans) is done by scan conversion of received acoustic signals after additional signal processing (i.e. bandpass filtering). After optional SAFT reconstruction, the signals are envelope filtered and converted into grayscale images. For increase of the lateral resolution and the effective SNR, the data can be processed with adequate beamforming algorithms based on SAFT (synthetic aperture focusing technique) prior to scan conversion. In this reconstruction algorithm, the geometry of the used focusing lens is taken in account and allows to enhance the SNR of the data. This reconstruction further allows to reduce the lateral point-spread-function of the system.
After the development phase of the project, the ultrasound and optoacoustic imaging platform was tested and successfully certified for being usable in a clinical study. The SKINSPECTION Ultrasound/Optoacoustic imaging platform is the first system being used clinically in the context of dermatology. First preliminary test on healthy probands have shown the excellent ability of the device for imaging subcutaneous microvasculature with high contrast.
Multiphoton Imaging/FLIM hardware:
The DermaInspect instrument described above was modified to enhance its ability to record the spectroscopic autofluorescence signals from skin. The final design of the multispectral FLIM detector is based on the following: fluorescence from the sample is deflected by a mirror into a cascaded beam splitter arrangement that divides the signal into four spectral channels. The signal in each channel is detected using a high quantum efficiency PMT. In order to maximize the collection efficiency, the light path from the microscope objective through to the detector was modeled using a ray tracing algorithm. These calculations were repeated as the light source was translated across the field of view and this enabled the collection efficiency as a function of field of view to be calculated. The aim was to design an optical system that provided the highest possible collection efficiency over as great a field of view as possible while working within the physical constraints of the existing DermaInspect system. The final design achieved >85% collection efficiency over 2/3 of the field of view of the objective. Maximizing the field of view is particularly important to ensure that as many scattered photons as possible are collected when imaging at depth in the specimen.
In order to accurately fit a multiexponential decay model to a fluorescence lifetime decay, it is necessary to have an accurate measurement of the instrument response function (IRF), i.e. the temporal response of the FLIM detector to a pulse of light much shorter than the temporal resolution of the detector. Normally, such pulses of light are provided by second harmonic generation (SHG) in a suitable sample, e.g. urea crystals. The SHG light is generated at half the wavelength of the infrared laser and can normally be detected by the FLIM PMT. However, when performing multispectral FLIM detection, only one spectral channel will detect light from a SHG sample. It was therefore necessary to implement a calibration procedure that could measure the IRF for all four channels and this then enables an accurate analysis of decay data.
In summary, IMPERIAL/JENLAB have designed, constructed, installed and tested a new multispectral FLIM detector on the base DermaInspect system. This instrument fits within the existing instrument case and Jenlab have confirmed that this instrument meets the certification requirements for in vivo use.
FLIM analysis software:
IMPERIAL has developed a custom software package to perform the analysis of FLIM data and has implemented the technique of convolution with a reference decay fluorophore. This technique overcomes a negative issue inherent with the spectral selection of the detector that inhibits the acquisition of an IRF with scattered light or the second harmonic in each channel. The technique also compensates for the spectrally dependent temporal shift of the PMTs that each detect a different portion of the spectrum. The fitting software can fit a decay curve to every pixel of the image to produce FLIM maps, which can then be saved as .tiff files. Processing times when fitting double exponential decays to typical data acquired with the four channel detector is less than 2 minutes per channel. Alternatively, for statistical analysis of the data, the software can fit a decay curve to a user defined ROI, or produce a histogram of any fitted parameter within any ROI and it can produce correlation plots of any two fitted parameters.
Ex-vivo/in-vivo evaluation:
Both systems, the FLIM/MPT and the US/OA imaging system have been successfully certified for use in clinical trials. The systems have been evaluated on ex-vivo samples and in-vivo as reported in the following.
In total, IMPERIAL used the 4-channel multispectral FLIM detector developed to acquire more than 200 multispectral FLIM images from 100 lesions (normal skin, BCC, melanoma, SCC ). The acquired data was analyzed with respect to its lifetime information and to the morphological information of the single cells.
The additional lifetime information provided by the FLIM module allows to differentiate a defined location within a cell with respect to its fluorophore content. For example, the fluorescence lifetime information allows NAD(P)H autofluorescence, dominant in the green spectral channel to be easily distinguished from melanin (short lifetime, red on false colour scale), which is dominant in the red spectral channel. The yellow spectral channel includes fluorescence from flavins, NAD(P)H and melanin. The multiple FLIM spectral channels also allow SHG from collagen (with an instantaneous decay) seen in the blue channel and elastin (with a long fluorescence lifetime, seen in the green/yellow channels to be distinguished from the same spatial location within the dermal papilla
The results of the morphological analysis show that the key features observed in BCCs were monomorphous cells, cells with large nuclei and elongated nuclei and cytoplasm. In addition, IMPERIAL repeatedly saw a pattern of monomorphous cells, often heterogeneous in size, with large nuclear/cytoplasmic ratios, poorly defined cell margins and appearing to overlap, which we referred to as overlapping cells. In melanoma, the most frequent feature observed was that of large intercellular distance.
By means of the analysis of large amounts of cells, the sensitivity/specificity of the visual morphological analysis for the diagnosis of BCC could be quantified to 79%/93%.
At University of Modena, 525 images corresponding to the stacks of 35 BCC samples were analysed. 3 epidermal descriptors and 7 tumour descriptors for BCC could thereby be identified. It was found that, over the BCC, the epidermis may be ulcerated, thinned or thickened, compared to normal skin. When the epidermis is present, epidermal layers are not recognizable. Compared to normal epidermal cells, those overlying the BCC exhibit irregular cellular contours, have lost the normal cohesion and are disposed in a random order; moreover, intercellular spaces are larger and irregular. It was further found out that BCC cells are monomorphous and show elongated shape and nuclei; they are aligned along the same direction and tightly packed together; sometimes a double alignment is observable with divergent sheets of cells. At the edge of the nodule a palisading phenomenon is visible in 52% of the cases. Further, by an excitation wavelength of 760 nm, basaloid nodules are observable as cell aggregates surrounded by fibers.
Further findings show that BCC cells exhibit a longer fluorescence decay time (blue) and fibers exhibit a short decay (red), which is likely to be due to SHG from collagen fibers. In some cases clear nests are not observable, but only sheets of cells intermingled with fibers. When shifting the excitation wavelength to 800-820 nm, to explore the extracellular matrix, BCC cells disappear and only fibers with short fluorescence lifetimes (red) surrounding dark spaces are visible (phantom islands). Fine elastic fibers appear to have a long fluorescence lifetime (which appears blue) when inserting the filter excluding SHG signals.
The likelihood of different descriptors was further investigated by University of Modena. For example, in terms of epidermal descriptors, "detached cells with enlargement of intercellular spaces" were found in 60.32% of BCCs and in 10.61% of other skin lesions; "cells with irregular contours" and "random arrangement" were observed in 31.75% of BCCs and in 12.12% and 7.57% of miscellaneous, respectively. BCC features were never observed in healthy skin samples. The frequency of BCC descriptors ranged between 25.39% and 61.90% in BCC and between 1.51% and 7.57% in miscellaneous. Cells with a longer fluorescence lifetime (blue cells) were observed in each BCC sample, whereas they were present in only one third of miscellaneous cells. The mean number of BCC descriptors/lesion was significantly higher in BCC (3.86 + 1.45) with respect to miscellaneous lesions (0.54 + 0.86) and the presence of at least one BCC descriptor was observed in all BCCs but only in 36.36% of other skin samples. This means that by choosing adequate descriptors, BCC can be differentiated from other samples by analyzing FLIM/MPT data.
The classification of a lesion as BCC was made based on the presence of 2 or more BCC descriptors per lesion. This approach correctly identified 96.8% of the cases of BCC and produced no false positives in other skin diseases or healthy skin.
US/OA investigation of samples:
In addition to FLIM/MPT imaging, ultrasound and optoacoustic data was acquired from freshly excised tissue samples. The subcutaneous morphological structure of the imaged lesion correlates with the appearance in the ultrasound data. Areas which are identified as lesions appear as dark structures with poor backscattering amplitude in the ultrasound data sets. The depth of the lesion could not exactly be correlated because of uncertainties in the relative positions of the histological slice and the ultrasound image slice. However, the comparison of the available US and histology slices suggests that there is a good correlation between tissue areas identified as lesion in the histology (darker subcutaneous areas in histological slices) and the tissue areas with low backscattering amplitude (appearing black in the ultrasound image).
Assessment of Tumor margins:
In order to assess tumour margins, IMPERIAL developed a motorized stage that allowed mosaics consisting of multiple multispectral FLIM fields of view to be acquired. The resulting image sets were then coregistered in post processing, corrected for variations in illumination intensity across the field of view and blended together to produce a large field of view. Although significant effort was exerted to solve the problems faced, it ultimately was not possible to obtain sufficiently accurate registration with the histology gold standard in order to be able to assess the accuracy of either the diagnostic algorithm or visual morphology approaches.
This work is the first demonstration of multiphoton FLIM imaging of large (mm) fields of view with subcellular resolution and the first time that it has been coupled to a sophisticated automatic diagnostic algorithm. This illustrates the potential of this technique for clinical ex vivo margin assessment, e.g. during Mohs procedures, and provides a potential means to identify small nests of BCCs in normal skin, such as those seen with infiltrative BCCs, whose margins are notoriously difficult to define.
Summary of ex-vivo/in vivo examinations:
In total over 130 lesions were imaged during the study at IMPERIAL. Multispectral FLIM has been shown to provide good contrast between normal skin and BCC (AUC 0.83) which was assessed between normal skin imaged mainly in vivo and BCCs imaged ex vivo. This approached was adopted in order to make maximum use of patients/samples in the clinical time available.
The US/OA system was used to assess BCCs both in vivo and ex vivo, and 5/9 lesions showed features indicating the depth of the lesion. This US lesion images were compared with the histological gold standard in order to assess the ability of the non-invasive system to provide information about the lesion depth. A good correlation could be found between areas associated with lesions in the histology and dark (= with poor acoustic backscattering) areas in ultrasound images. OA were not suited for assessing the depth of the lesion, however they gave an insight into the morphology of the subcutaneous vasculature.
University of Modena has performed in total over 726 data acquisitions from tissue samples (ex-vivo and in-vivo) and cell cultures. The final assessment of the diagnostic accuracy of the MPT- FLIM DermaInspect system used the combined results of ex vivo and in vivo acquisitions. MPT-FLIM Dermainspect has shown to provide good contrast between BCC and other skin lesion as well as between BCC and normal skin.
In summary, the SKINSPECTION consortium has developed and certified two imaging systems based on US/OA on one hand and MPT/FLIM on the other hand to be used as a combined multimodal and multiscale imaging platform. The clinical evaluation has show the suitability of the new approach to differentiate skin lesions by analysis of morphological or/and FLIM features on the single cell level. In addition, the macroscopic US/OA system has shown its extremely high sensitivity for imaging of subcutaneous microvasculature and gave an non-invasive insight in the subcutaneous structure of skin lesions.
Potential Impact:
This project has developed SKINSPECTION, a unique non-invasive multimodal imaging platform, which provides non-invasive high resolution three-dimensional clinical images to aid the detection, diagnosis and treatment of skin cancer. The SKINSPECTION platform combines acoustic, optical and opto-acoustic imaging techniques. This novel multimodal approach provides a wide-field acoustic/optoacoustic view with depth information of the dermatological lesion as well as a close optical look into particular intratissue compartments with subcellular resolution. The most significant outputs of this project are the development and testing of the SKINSPECTION system and an in vivo clinical trial to establish the usefulness and efficacy of the system for the diagnosis and characterization as well as treatment control of skin cancer.
The project is the first step towards an enhanced diagnosis of skin cancer, which is an increasing problem in Europe (populated in majority by Caucasians with skin type I and II). Further, the outcome of this project is of European interest due to the increasing aging population in most EC countries and the enhanced damages due to UV light exposure, which contribute to the dramatic increase of skin cancer.
Impact on science:
So far, the only CE marked clinical diagnostic tool based on femtosecond laser technology, namely the multiphoton tomograph DermaInspect, is produced and distributed by the consortium member and SME JenLab. There is no similar product in the United States available. The consortium is therefore on the leading edge of skin diagnostics with ultrashort laser pulses. Further, the project has for the first time allowed the use of optoacoustic techniques in the clinical context of dermatology. The developed optoacoustic system is furthermore the first one to allow high resolution 3D optoacoustic imaging based on a handheld probe.
Impact on industry
This proposal has a significant impact in a new generation of dermatological imaging systems which will not only benefit skin cancer patients but would have wider applicability in non-neoplastic skins disorders and reconstructive surgery. The ability to check for premalignant skin conditions and provide preventative care will enable diagnostics and anti-aging products to have a major share of a multi billion euro market globally.
In addition there is a vast market for skin cancer and ageing prevention topical preparations and diagnostics, estimated to be in excess of Euro 10 bn per annum.
The Fraunhofer Society is the leading research organization for applied research in Europe and the strong partner of the European industry. The SME JenLab as the manufacturer and distributer is a spin-off of the university and member of the Fraunhofer venture group. The project will help to strength the pioneering role of this European HighTech company as well as of providers of special compartments, such as ZEISS, the SME Becker&Hickl GmbH and others.
JenLab owns patents in the field of multiphoton technology and is open to sign contracts with potential large European distributors. The Fraunhofer IBMT filed patents in the area of high frequency (GHz) and miniaturized multi-element element acoustical detector systems. As a non-profit organization the Fraunhofer Society is interested in the exploitation of their results by the European industry.
In order to present the project results to a wider audience than the scientific community, a large number of dissemination activities were conducted within SKINSPECTION.
First of all, a project website (http://www.skinspection-fp7.eu/) was launched. The project website informs about the objectives of SKINSPECTION, presents the consortium, provides links to project-related topics and summarizes the scientific publications generated within SKINSPECTION. Further, a project leaflet was generated which gives an overview on the technical developments of the different project partners.
Finally, the SKINSPECTION results and the developed hardware was showcased at different events (conferences, faires) open to industry (MEDICA), the scientific community (EADV, FLIM 2011) and the general public.
List of Websites:
SKINSPECTION public website: www.skinspection-fp7.eu
Contact details of the SKINSPECTION coordinator:
Dr. Marc Fournelle
Fraunhofer Institute for Biomedical Technology (IBMT)
- Biomedical Ultrasound Research -
Ensheimer Straße 48
D-66386 St. Ingbert
Tel.: +49 (0)6894 980 220
Fax: +49 (0)6894 980 234
email: marc.fournelle@ibmt.fraunhofer.de
The SKINSPECTION consortium consists of six European partners from United Kingdom, Italy and Germany. The concept is based on a multimodal imaging approach combining multiphoton tomography, fluorescence lifetime imaging, optoacoustic imaging and high resolution ultrasound. The combination of theses modalities allows in-vivo non-invasive imaging of skin lesions both on the macroscopic and the microscopic level. Ultrasound and optoacoustic imaging allow a wide-field view of structures with special focus to subcutaneous microvasculature and morphology of lesions. While ultrasound is established in several clinical fields, optoacoustic imaging (OA) is a new technique based on the generation of broadband ultrasound signals by absorption of short laser pulses. OA therefore combines beneficious features of ultrasound imaging (low scattering, high image depth) with advantages of optics such as high contrast. These modalities provide images down to 3 mm in depth with a sub-100µm resolution. In addition, multiphoton tomography and FLIM (Fluorescence Lifetime Imaging Microscopy) give access to subcellular structures from the upper skin layers. Imperial College London has developed a FLIM module for integration in a multiphoton tomograph from the German company JenLab GmbH. The ultrasound based modalities are implemented by the German partners kibero GmbH and Fraunhofer IBMT. After 3 years of development and certification for clinical testing, the evaluation of the platform was performed in a bicentric clinical study at Hammersmith Hospital London and Universita di Modena.
While high resolution ultrasound is established in diagnosis of skin diseases, the optoacoustic imaging module developed by Fraunhofer IBMT and kibero has allowed the first evaluation of optoacoustic technology for investigation of skin lesions under clinical conditions. Further, the optical SKINSPECTION modalities give access to in-vivo 3D biopsies of skin with ultrahigh subcellular resolution. The multiphoton technology from Jenlab in combination with Imperials FLIM module have allowed to provide clinicians with a whole range of new diagnostic parameters, allowing to identify specific morphological or FLIM features from cells that can be associated to defined skin diseases.
The clinical evaluation phase has shown that the occurrence of defined cells features that can be extracted from the data acquired with the new platform can be correlated to specific diseases. In total, more than 700 lesions were imaged with FLIM and multiphoton tomography and a statistical analysis was performed with respect to the sensitivity and specificity of defined cell features for accurate diagnosis of skin diseases such as BCC or melanoma. For instance, it could be shown that FLIM provides excellent contrast between healthy skin and basal cell carcinoma. The characteristic FLIM signal further allows sensitive differentiation between lesion types such as BCC and melanoma. The developed US/OA platform has further shown to be ideally suitable for high resolution non-invasive imaging of subcutaneous vasculature, with sensitivity and contrast enhanced when compared to existing methods.
Project Context and Objectives:
The incidence of skin cancer in Europe, US, and Australia is rising rapidly. One in five will develop some form of skin cancer during the lifetime. A person has a 1:33 chance to develop melanoma, the most aggressive skin cancer. Melanoma is the second most common cancer in women aged 20-29, and the sixth most common cancer in men and women. In 2007, more than 1 million new cases will be diagnosed in the US alone. About 90% of skin cancers are caused by ultraviolet (UV) sun light. The World Health Organization estimates that 60,000 people will die this year from too much sun: 48,000 from melanoma and 12,000 from other skin cancer. Five sunburns double the risk of developing skin cancer. Actinic keratosis is a UV induced intraepidermal neoplasia which affects by now millions of Europeans. Nearly 10% of the cases transform into an aggressive invasive squamous cell carcinoma.
The incidence of melanoma has increased nearly 700% and the overall mortality rate by 165% from 1950 to 2001 (American Society of Dermatology: 2007 Skin Cancer Facts). The increase will be even more dramatic in the near future due to the man-made changes in climate and the increase of the UV hazard by ozone depletion, respectively, as well as by the aging population. Australia with the most drastic ozone problems reports on a significant increase of melanomas. The European Commission has identified the dangers of UV exposure from sunlight and tanning beds as a major health risk.
The five year survival rate for people with melanoma, basal cell carcinoma, and squamous cell carcinoma is higher than 95% if detected and treated early. However, most medical doctors are still using visual inspection as the only diagnostic method of skin cancer. If suspicious, biopsies are taken invasively from which histological 7 µm thick sections are prepared, then stained, and finally analyzed by pathologists with visual inspection under a light microscope. Several 10,000 biopsies are investigated annually in a typical hospital. The direct annual costs of diagnosis and treatment of skin cancer are several billion Euro.
A significant improvement of the current diagnostic tools of dermatologists is required in order to identify dermal disorders at a very early stage as well as to monitor directly the effects of treatment.
A very first step in this direction was done with the introduction of dermoscopy based on digital 2D images using CCD cameras. However, this technology does not provide 3D information and no sufficient resolution. Furthermore, it is very difficult to distinguish between some innocuous or premalignant conditions (such as dysplastic naevi or actinic keratosis) and malignant counterparts.
Non-invasive high-resolution imaging systems with the capability to gain additional information from inside the skin have significant potential to address these problems. Several types of 3D imaging have been proposed for skin diagnosis out of which optical methods and acoustic methods are the most promising ones.
The most recent development in high resolution in vivo 3D skin imaging is clinical two-photon tomography based on NIR femtosecond laser and flying-spot technology. A first commercial CE marked clinical multiphoton tomography is on the market and used e.g. by a clinical research group in Brisbane/Australia including Ian Frazer, the inventor of the first anticancer vaccine, to obtain information on intraskin cancer cells based on the detection of fluorescent endogenous biomolecules such as melanin. This first clinical femtosecond laser diagnostic tool was developed by the SME JenLab GmbH, a member of this consortium. The novel tomography however has the limitations of being able to inspect only small sub-mm3 skin lesions and of having a pure qualitative signal. Further wide-field imaging tools in combination with this novel multiphoton technology are required to get a full 3D view of the skin lesion and to enable also a closer high-resolution look into particular suspicious areas. In addition, photon statistics is needed to provide quantitative information.
Nowadays, the most often used clinical wide-field 3D imaging tool is based on ultrasound. However, clinical devices lack sufficient resolution to monitor intratissue cancer cell clusters.
Also optoacoustical imaging based on localized near infrared tissue excitation and the generation of ultrasound waves is a promising novel technology which can, in principle, provide information on vascularization and melanin content. However, no commercial clinical optoacoustical system exists.
So far, all single image modalities are more descriptive than quantitative, lack certain information or are limited in their field of view, the required resolution, and the clinical use.
In order to solve this problem, the SKINSPECTION consortium has developed a non-invasive multimodal hybrid imaging system with the capability to perform non-invasive high resolution three-dimensional clinical (i) two-photon imaging with time-correlated single photon detection, (ii) autofluorescence lifetime imaging, (iii) high-frequency acoustical imaging, and (iv) optoacoustical imaging using ultrashort near infrared (NIR) laser pulses. This novel multimodal approach will provide a wide-field acoustic/optoacoustic view with depth information of the dermatological lesion as well as a close optical look into particular intratissue compartments with subcellular resolution.
The goal of the project was to develop an imaging platform allowing the user to easily acquired both high resolution and low resolution images under clinical conditions. In addition, fluorescence lifetime imaging has been implemented to obtain a further method of contrast enhancement, to distinguish between different fluorescent biomolecules and more importantly, to provide a method of signal quantification by time-correlated single photon counting with picosecond temporal resolution.
Furthermore, functional imaging will be possible through highly selective imaging of vasculature by means of the newly developed optoacoustic imaging device.
The objectives of the SKINSPECTION project were therefore the development of a novel multimodal hybrid technology and the construction of a sophisticated non-invasive clinical image platform which combines ultrasound and optical features as well as quantitative technologies to visualize, to image and to detect cancerous tissue and intratissue cancer cells in skin.
The consortium consists of pioneers in the field of clinical multiphoton tomography, fluorescence lifetime imaging, high-frequency ultrasound and optoacoustic imaging, melanoma research, and evaluation of skin cancer data based on image processing.
Project Results:
In the context of the SKINSPECTION project, different technologies were developed, implemented into imaging systems, certified and tested in a clinical evaluation phase. The different technologies as well as the main clinical achievements are presented in the text that follows.
DermaInspect System Overview
The combined FLIM/MPT imaging device is based on the existing DermaInspect system from JenLab GmbH. This multiphoton base system is a femtosecond-laser scanning system which allows performing non-invasive in vivo optical biopsies of skin with ultrahigh subcellular resolution. The optical biopsies are based on measurements of two-photon autofluorescence and second harmonic generation (SHG). The nonlinearly induced autofluorescence originates from naturally existent endogenous fluorophores and protein structures of the skin such as NAD(P)H, flavins, elastin, porphyrins, and melanin. SHG which occurs in certain non-symmetric molecules can be used to detect collagen structures and to distinguish them from the previously mentioned fluorophores. The multiphoton base system can be equipped with detectors to perform fluorescence lifetime imaging (FLIM) and enables then 4-D-imaging (x,y,z,?) with subcellular spatial resolution. The system provides detailed information about cell and tissue structure to a depth of up to 200 ?m inside human skin. A major application of the multiphoton system is skin cancer diagnosis. Further applications include tissue engineering and in situ drug monitoring. The base system is equipped with interfaces for several FLIM detectors which can be used to measure signals at different spectral ranges simultaneously. In order to be able to include several photo multiplier tubes (PMTs) inside the scan-detector module a second level plate is currently designed and will be installed. Also, an interface to connect to state-of-the-art dermatoscopy systems (Fotofinder) is being developed.
The system integrates a light source used for the multiphoton base system (titanium:sapphire laser (MaiTai XF-1)) which generates sub 100-fs pulses in the red and near infrared wavelength region with a repetition rate of 80 MHz. The center wavelength of the pulses laser is tunable in the range of 710 nm and 920 nm.
The laser beam is coupled into the scan-detector module using special mirrors suited for NIR fs pulses. Within the scan-detector module the laser light is attenuated by a step-motor driven polarizer which can be controlled by the JENLAB software. Any unintended laser output is prevented by a safety shutter which opens during scanning operation only. After attenuation the laser pulses are directed onto a two-dimensional scanning module which consists of two temperature-controlled galvo-scanners. The scanning module deflects the laser beam in the x- and y-direction and directs the laser beam into the beam expander and the microscope objective. The scan-detector module contains a three-position-beam-splitter slider which allows holding any combination of beam splitters, mirrors or empty positions. Typically, the system allows excitation in the range between 700 nm and 1000 nm and detection in the blue and in the green/yellow spectral region. The microscope objective is connected to a piezo-driven module which allows to move the focusing optics over a range of 400 ?m with submicron precision. The typical imaging field is 250×250 ?m2.
The focused laser beam excites autofluorescence and induces SHG by certain skin components (collagen). This laser-induced radiation is collected and directed by the focusing optics through the beam splitter onto highly sensitive detectors.
Ultrasound and Optoacoustic system overview
The SKINSPECTION ultrasound and optoacoustic imaging system consists in a new development of the partners KIBERO and FRAUNHOFER. The specifications of the imaging platform were defined at the beginning of the SKINSPECTION project. To meet the requirements of clinical imaging of skin lesions, the system shall have the following properties:
- High frequency ultrasound signals
- 3D imaging
- Lateral scanning range > 6,4 mm x 6,4 mm
- Switchable between optoacoustic and ultrasound imaging
For allowing 3D optoacoustic and ultrasound imaging, different concepts were discussed. 3D imaging can be achieved by used of array technology or by using single element transducers combined with a mechanical scanning.
An array structure allows to move the ultrasound focus without moving the complete transducer. Like this, movement artefacts caused by the scanning of the complete transducer can be eliminated. The penetration depth into human skin and the lateral resolution are functions of the frequency. With increasing frequency the penetration depth decreases and the resolution increases. At 50 MHz the penetration depth is 4 mm and at 100 MHz the penetration depth is 2 mm. For defining the best-suited array geometry, different transducer soundfield simulations were run.
Opto acoustic applications need a transducer with a large acoustical bandwidth of maximal 75%. The bandwidth can be adjusted with a defined damping layer on the backside of the piezoelectric material, called backing. The magnitude of the photoacoustic waves is proportional to the local optical energy deposition. This mechanism fundamentally diverges from both the pure optical mechanism of signal generation (optical reflection, fluorescence) and pure acoustic mechanism (acoustic reflection, damping). The sample is irradiated with short laser pulses in the range of a few nanoseconds which corresponds to an ultrasound frequency response up to a cut-off frequency.
In order to have a transducer meeting the specifications and being usable both for ultrasound and optoacoustic imaging, different hardware strategies were investigated in the development phase of the project. 2D ultrasound arrays based on PZT, ZnO or PVDF were set-up and characterized as well as single focused PZT and PVDF transducers.
The preliminary tests suggested that a single focused transducer is better suited for the tasks of in-vivo combined optoacoustic and ultrasound imaging for the following reasons:
- Focused single element transducers have a higher sensitivity than arrays with comparably smaller aperture, which is especially of importance for optoacoustic imaging where signals are generated by low energy laser pulses
- Single element focused transducers are better suited for allowing an adequate light delivery (needed for optoacoustic signal generation)
For this reason, it was decided to build a single element transducer. Two versions have initially been developed:
a 30 MHz PZT transducer with a focal length of 250 µm and a focal width of 50 µm and a 70 MHz PVDF transducer with a focal length of 150 µm and a focal width of 30 µm. Both with a focal distance of 5.5 mm. The transducers are mounted in a scan unit allowing displacement in 3 dimensions in order to adjust the distance to the sample and to acquire threedimensional data sets. Both single element transducers are available for the integration into the acoustic detection unit. The acoustic performance of the PZT material is excellent, the acoustic performance of the PVDF transducer is acceptable. Due to the different acoustic performance of the active materials, it was decided to define the 30 MHZ PZT transducer as the primary system transducer with an option to use the 60 MHz PVDF transducer if the higher resolution is needed.
Optoacoustic hardware components:
In photoacoustic imaging, the detected acoustic signal correlates directly to the amount of absorbed energy. According to that the laser light should get mainly absorbed by the interesting structures and not by the main layers of the skin. Understanding how light interacts with skin assists in designing an optoacoustic imaging module. In medical physics, the optical window is the portion of the visible and infrared spectrum where living tissue absorbs only a small fraction of the light. This optical window approximately ranges from 650 nm to 1200 nm. At shorter wavelengths light is strongly absorbed by the hemoglobin in blood, while at longer wavelengths water strongly absorbs infrared light. Two different laser types are available working at 1064 nm and 532 nm wavelength. Imaging of microvasculature or the amount of blood in a lesion can be performed with a frequency-doubled Nd:YAG laser working at 532 nm. However, the penetration depth is limited. Working at 1064 nm the penetration depth is much higher due to the low absorption in tissue. Due to the availability we decided to use a 1064 Nd:YAG laser.
Photoacoustic probing of confined volumes can be achieved by several instrumental configurations. Using a multichannel photoacoustic system, signals are acquired by an unfocussed transducer array. A linear array needs a homogeneous diffuse illumination directly underneath the active area. In this case the entire observation volume is illuminated. For this scenario simulations of the light propagation in human skin based on a Monte Carlo model were made to evaluate possible designs of the light wave guide.
The depth of field is a function of the incident angle. In order to image structures near to the surface of the skin, the best performance was achieved at 45°. Nevertheless 70% of the energy gets absorbed within the first 2 millimeters underneath the surface of the skin. With the results of the simulation the specifications of the light fibre were deduced and a first prototype was designed. According to these results a first prototype for diffuse illumination using a transducer array as a detector was produced.
After simulating and building the prototype of the optical fibre illumination device, experiments for signal generation were performed in cooperation with the University of Modena. Unfortunately no photoacoustic signal could be detected with the used setup because the energy density was insufficient for optoacoustic signal generation. Higher intensity requires a solid state laser. However, flashlamp-pumped Q-switched lasers limit the data acquisition rate because of their fixed low pulse repetition rate (e.g. 10 Hz). Diode pumped solid state lasers based on the SLAB-concept are not subject to this limitation, but the size of such a laser limits the functional integrity.
An alternative to the wide field illumination is to focus the laser light. Using a focused single element transducer, the illuminated area was reduced in space to increase the light intensity and to avoid unnecessary illumination. The outcoupling angle of the light had to be modified in order to match the optical focus spot and the acoustic focus spot as accurately as possible.
Once the optical, mechanical and ultrasound components were defined, a first prototype of a handheld probe was developed and tested on phantoms at FRAUNHOFER. These tests were followed by a technical redesign which lead to a second version of the combined ultrasound/optoacoustic handheld probe.
Ultrasound electronics:
The electronics system has four distinctive parts: Computer, Ultrasound electronics, the laser module including the drive electronics, the transducers including mechanical scanner for 3-dimensional imaging. All components will be set up in one single housing.
Software for acquisition and display of acoustic and optoacoustic data:
The software for the data acquisition and the file export of the ultrasound and opto acoustical imaging modality is called skinspector 2010 and includes the hardware control skinterface. The tasks of the software include controlling the hardware, reading and handling the measured data including band pass filtering and storage of ultrasound RF-data. The user interface was programmed and is able to control the first simple region of interests in the B-Scan-Mode. Furthermore incoming data from the hardware system is converted and passed to the scan-conversion which includes envelope detection, requantization and the scan conversion itself. The scan conversion optimizes the images for better display quality. In this way B-Scan slices through the skin from the transducer into the depth can be imaged as greyscale image. The scan conversion of C-Scans using the measured data of a complete scan is shown in the application and displays slices through the skin with constant depth to the transducer. The spectrum of measured data and the line based data itself is displayed as graphs on the bottom and currently shown data can be selected using the mouse on the C-Scan Image.
The C-Scan display shows a slice perpendicular to the B-Scan on the left and can be used to visualize different slices in different depth parallel to the skin surface or to display a projection through the skin in direction of the sound propagation. The user can select different display modes (maximum intensity projection, average intensity projection, slice only visualization. The software further allows export of the signals. DICOM data will be exported as image stacks for 3d visualization in an external DICOM viewer and RF-data is stored for further signal processing and advanced imaging possibilities.
Software for ultrasound and optoacoustic image reconstruction
In the chosen concept, Data acquisition is based on a single element focusing ultrasound transducer which is scanned in x- and y- direction over the investigated sample. For this reason, the acoustic and optoacoustic data are acquired as a-scan signals with adjustable sample rate and ascan length. Depending on the selected scanning preset, the number of ascans per bscan can further be varied. For storage of the data, ascans are combined to bscans and are saved as such in the binary .grb format. All relevant scan parameters such as number of ascans or stepsize are also saved in the header of the grb file. For easy data exchange, a dicom converter has also been integrated in the Skinspector software platform.
The acoustic image reconstruction of scanned slices (B-Scans) is done by scan conversion of received acoustic signals after additional signal processing (i.e. bandpass filtering). After optional SAFT reconstruction, the signals are envelope filtered and converted into grayscale images. For increase of the lateral resolution and the effective SNR, the data can be processed with adequate beamforming algorithms based on SAFT (synthetic aperture focusing technique) prior to scan conversion. In this reconstruction algorithm, the geometry of the used focusing lens is taken in account and allows to enhance the SNR of the data. This reconstruction further allows to reduce the lateral point-spread-function of the system.
After the development phase of the project, the ultrasound and optoacoustic imaging platform was tested and successfully certified for being usable in a clinical study. The SKINSPECTION Ultrasound/Optoacoustic imaging platform is the first system being used clinically in the context of dermatology. First preliminary test on healthy probands have shown the excellent ability of the device for imaging subcutaneous microvasculature with high contrast.
Multiphoton Imaging/FLIM hardware:
The DermaInspect instrument described above was modified to enhance its ability to record the spectroscopic autofluorescence signals from skin. The final design of the multispectral FLIM detector is based on the following: fluorescence from the sample is deflected by a mirror into a cascaded beam splitter arrangement that divides the signal into four spectral channels. The signal in each channel is detected using a high quantum efficiency PMT. In order to maximize the collection efficiency, the light path from the microscope objective through to the detector was modeled using a ray tracing algorithm. These calculations were repeated as the light source was translated across the field of view and this enabled the collection efficiency as a function of field of view to be calculated. The aim was to design an optical system that provided the highest possible collection efficiency over as great a field of view as possible while working within the physical constraints of the existing DermaInspect system. The final design achieved >85% collection efficiency over 2/3 of the field of view of the objective. Maximizing the field of view is particularly important to ensure that as many scattered photons as possible are collected when imaging at depth in the specimen.
In order to accurately fit a multiexponential decay model to a fluorescence lifetime decay, it is necessary to have an accurate measurement of the instrument response function (IRF), i.e. the temporal response of the FLIM detector to a pulse of light much shorter than the temporal resolution of the detector. Normally, such pulses of light are provided by second harmonic generation (SHG) in a suitable sample, e.g. urea crystals. The SHG light is generated at half the wavelength of the infrared laser and can normally be detected by the FLIM PMT. However, when performing multispectral FLIM detection, only one spectral channel will detect light from a SHG sample. It was therefore necessary to implement a calibration procedure that could measure the IRF for all four channels and this then enables an accurate analysis of decay data.
In summary, IMPERIAL/JENLAB have designed, constructed, installed and tested a new multispectral FLIM detector on the base DermaInspect system. This instrument fits within the existing instrument case and Jenlab have confirmed that this instrument meets the certification requirements for in vivo use.
FLIM analysis software:
IMPERIAL has developed a custom software package to perform the analysis of FLIM data and has implemented the technique of convolution with a reference decay fluorophore. This technique overcomes a negative issue inherent with the spectral selection of the detector that inhibits the acquisition of an IRF with scattered light or the second harmonic in each channel. The technique also compensates for the spectrally dependent temporal shift of the PMTs that each detect a different portion of the spectrum. The fitting software can fit a decay curve to every pixel of the image to produce FLIM maps, which can then be saved as .tiff files. Processing times when fitting double exponential decays to typical data acquired with the four channel detector is less than 2 minutes per channel. Alternatively, for statistical analysis of the data, the software can fit a decay curve to a user defined ROI, or produce a histogram of any fitted parameter within any ROI and it can produce correlation plots of any two fitted parameters.
Ex-vivo/in-vivo evaluation:
Both systems, the FLIM/MPT and the US/OA imaging system have been successfully certified for use in clinical trials. The systems have been evaluated on ex-vivo samples and in-vivo as reported in the following.
In total, IMPERIAL used the 4-channel multispectral FLIM detector developed to acquire more than 200 multispectral FLIM images from 100 lesions (normal skin, BCC, melanoma, SCC ). The acquired data was analyzed with respect to its lifetime information and to the morphological information of the single cells.
The additional lifetime information provided by the FLIM module allows to differentiate a defined location within a cell with respect to its fluorophore content. For example, the fluorescence lifetime information allows NAD(P)H autofluorescence, dominant in the green spectral channel to be easily distinguished from melanin (short lifetime, red on false colour scale), which is dominant in the red spectral channel. The yellow spectral channel includes fluorescence from flavins, NAD(P)H and melanin. The multiple FLIM spectral channels also allow SHG from collagen (with an instantaneous decay) seen in the blue channel and elastin (with a long fluorescence lifetime, seen in the green/yellow channels to be distinguished from the same spatial location within the dermal papilla
The results of the morphological analysis show that the key features observed in BCCs were monomorphous cells, cells with large nuclei and elongated nuclei and cytoplasm. In addition, IMPERIAL repeatedly saw a pattern of monomorphous cells, often heterogeneous in size, with large nuclear/cytoplasmic ratios, poorly defined cell margins and appearing to overlap, which we referred to as overlapping cells. In melanoma, the most frequent feature observed was that of large intercellular distance.
By means of the analysis of large amounts of cells, the sensitivity/specificity of the visual morphological analysis for the diagnosis of BCC could be quantified to 79%/93%.
At University of Modena, 525 images corresponding to the stacks of 35 BCC samples were analysed. 3 epidermal descriptors and 7 tumour descriptors for BCC could thereby be identified. It was found that, over the BCC, the epidermis may be ulcerated, thinned or thickened, compared to normal skin. When the epidermis is present, epidermal layers are not recognizable. Compared to normal epidermal cells, those overlying the BCC exhibit irregular cellular contours, have lost the normal cohesion and are disposed in a random order; moreover, intercellular spaces are larger and irregular. It was further found out that BCC cells are monomorphous and show elongated shape and nuclei; they are aligned along the same direction and tightly packed together; sometimes a double alignment is observable with divergent sheets of cells. At the edge of the nodule a palisading phenomenon is visible in 52% of the cases. Further, by an excitation wavelength of 760 nm, basaloid nodules are observable as cell aggregates surrounded by fibers.
Further findings show that BCC cells exhibit a longer fluorescence decay time (blue) and fibers exhibit a short decay (red), which is likely to be due to SHG from collagen fibers. In some cases clear nests are not observable, but only sheets of cells intermingled with fibers. When shifting the excitation wavelength to 800-820 nm, to explore the extracellular matrix, BCC cells disappear and only fibers with short fluorescence lifetimes (red) surrounding dark spaces are visible (phantom islands). Fine elastic fibers appear to have a long fluorescence lifetime (which appears blue) when inserting the filter excluding SHG signals.
The likelihood of different descriptors was further investigated by University of Modena. For example, in terms of epidermal descriptors, "detached cells with enlargement of intercellular spaces" were found in 60.32% of BCCs and in 10.61% of other skin lesions; "cells with irregular contours" and "random arrangement" were observed in 31.75% of BCCs and in 12.12% and 7.57% of miscellaneous, respectively. BCC features were never observed in healthy skin samples. The frequency of BCC descriptors ranged between 25.39% and 61.90% in BCC and between 1.51% and 7.57% in miscellaneous. Cells with a longer fluorescence lifetime (blue cells) were observed in each BCC sample, whereas they were present in only one third of miscellaneous cells. The mean number of BCC descriptors/lesion was significantly higher in BCC (3.86 + 1.45) with respect to miscellaneous lesions (0.54 + 0.86) and the presence of at least one BCC descriptor was observed in all BCCs but only in 36.36% of other skin samples. This means that by choosing adequate descriptors, BCC can be differentiated from other samples by analyzing FLIM/MPT data.
The classification of a lesion as BCC was made based on the presence of 2 or more BCC descriptors per lesion. This approach correctly identified 96.8% of the cases of BCC and produced no false positives in other skin diseases or healthy skin.
US/OA investigation of samples:
In addition to FLIM/MPT imaging, ultrasound and optoacoustic data was acquired from freshly excised tissue samples. The subcutaneous morphological structure of the imaged lesion correlates with the appearance in the ultrasound data. Areas which are identified as lesions appear as dark structures with poor backscattering amplitude in the ultrasound data sets. The depth of the lesion could not exactly be correlated because of uncertainties in the relative positions of the histological slice and the ultrasound image slice. However, the comparison of the available US and histology slices suggests that there is a good correlation between tissue areas identified as lesion in the histology (darker subcutaneous areas in histological slices) and the tissue areas with low backscattering amplitude (appearing black in the ultrasound image).
Assessment of Tumor margins:
In order to assess tumour margins, IMPERIAL developed a motorized stage that allowed mosaics consisting of multiple multispectral FLIM fields of view to be acquired. The resulting image sets were then coregistered in post processing, corrected for variations in illumination intensity across the field of view and blended together to produce a large field of view. Although significant effort was exerted to solve the problems faced, it ultimately was not possible to obtain sufficiently accurate registration with the histology gold standard in order to be able to assess the accuracy of either the diagnostic algorithm or visual morphology approaches.
This work is the first demonstration of multiphoton FLIM imaging of large (mm) fields of view with subcellular resolution and the first time that it has been coupled to a sophisticated automatic diagnostic algorithm. This illustrates the potential of this technique for clinical ex vivo margin assessment, e.g. during Mohs procedures, and provides a potential means to identify small nests of BCCs in normal skin, such as those seen with infiltrative BCCs, whose margins are notoriously difficult to define.
Summary of ex-vivo/in vivo examinations:
In total over 130 lesions were imaged during the study at IMPERIAL. Multispectral FLIM has been shown to provide good contrast between normal skin and BCC (AUC 0.83) which was assessed between normal skin imaged mainly in vivo and BCCs imaged ex vivo. This approached was adopted in order to make maximum use of patients/samples in the clinical time available.
The US/OA system was used to assess BCCs both in vivo and ex vivo, and 5/9 lesions showed features indicating the depth of the lesion. This US lesion images were compared with the histological gold standard in order to assess the ability of the non-invasive system to provide information about the lesion depth. A good correlation could be found between areas associated with lesions in the histology and dark (= with poor acoustic backscattering) areas in ultrasound images. OA were not suited for assessing the depth of the lesion, however they gave an insight into the morphology of the subcutaneous vasculature.
University of Modena has performed in total over 726 data acquisitions from tissue samples (ex-vivo and in-vivo) and cell cultures. The final assessment of the diagnostic accuracy of the MPT- FLIM DermaInspect system used the combined results of ex vivo and in vivo acquisitions. MPT-FLIM Dermainspect has shown to provide good contrast between BCC and other skin lesion as well as between BCC and normal skin.
In summary, the SKINSPECTION consortium has developed and certified two imaging systems based on US/OA on one hand and MPT/FLIM on the other hand to be used as a combined multimodal and multiscale imaging platform. The clinical evaluation has show the suitability of the new approach to differentiate skin lesions by analysis of morphological or/and FLIM features on the single cell level. In addition, the macroscopic US/OA system has shown its extremely high sensitivity for imaging of subcutaneous microvasculature and gave an non-invasive insight in the subcutaneous structure of skin lesions.
Potential Impact:
This project has developed SKINSPECTION, a unique non-invasive multimodal imaging platform, which provides non-invasive high resolution three-dimensional clinical images to aid the detection, diagnosis and treatment of skin cancer. The SKINSPECTION platform combines acoustic, optical and opto-acoustic imaging techniques. This novel multimodal approach provides a wide-field acoustic/optoacoustic view with depth information of the dermatological lesion as well as a close optical look into particular intratissue compartments with subcellular resolution. The most significant outputs of this project are the development and testing of the SKINSPECTION system and an in vivo clinical trial to establish the usefulness and efficacy of the system for the diagnosis and characterization as well as treatment control of skin cancer.
The project is the first step towards an enhanced diagnosis of skin cancer, which is an increasing problem in Europe (populated in majority by Caucasians with skin type I and II). Further, the outcome of this project is of European interest due to the increasing aging population in most EC countries and the enhanced damages due to UV light exposure, which contribute to the dramatic increase of skin cancer.
Impact on science:
So far, the only CE marked clinical diagnostic tool based on femtosecond laser technology, namely the multiphoton tomograph DermaInspect, is produced and distributed by the consortium member and SME JenLab. There is no similar product in the United States available. The consortium is therefore on the leading edge of skin diagnostics with ultrashort laser pulses. Further, the project has for the first time allowed the use of optoacoustic techniques in the clinical context of dermatology. The developed optoacoustic system is furthermore the first one to allow high resolution 3D optoacoustic imaging based on a handheld probe.
Impact on industry
This proposal has a significant impact in a new generation of dermatological imaging systems which will not only benefit skin cancer patients but would have wider applicability in non-neoplastic skins disorders and reconstructive surgery. The ability to check for premalignant skin conditions and provide preventative care will enable diagnostics and anti-aging products to have a major share of a multi billion euro market globally.
In addition there is a vast market for skin cancer and ageing prevention topical preparations and diagnostics, estimated to be in excess of Euro 10 bn per annum.
The Fraunhofer Society is the leading research organization for applied research in Europe and the strong partner of the European industry. The SME JenLab as the manufacturer and distributer is a spin-off of the university and member of the Fraunhofer venture group. The project will help to strength the pioneering role of this European HighTech company as well as of providers of special compartments, such as ZEISS, the SME Becker&Hickl GmbH and others.
JenLab owns patents in the field of multiphoton technology and is open to sign contracts with potential large European distributors. The Fraunhofer IBMT filed patents in the area of high frequency (GHz) and miniaturized multi-element element acoustical detector systems. As a non-profit organization the Fraunhofer Society is interested in the exploitation of their results by the European industry.
In order to present the project results to a wider audience than the scientific community, a large number of dissemination activities were conducted within SKINSPECTION.
First of all, a project website (http://www.skinspection-fp7.eu/) was launched. The project website informs about the objectives of SKINSPECTION, presents the consortium, provides links to project-related topics and summarizes the scientific publications generated within SKINSPECTION. Further, a project leaflet was generated which gives an overview on the technical developments of the different project partners.
Finally, the SKINSPECTION results and the developed hardware was showcased at different events (conferences, faires) open to industry (MEDICA), the scientific community (EADV, FLIM 2011) and the general public.
List of Websites:
SKINSPECTION public website: www.skinspection-fp7.eu
Contact details of the SKINSPECTION coordinator:
Dr. Marc Fournelle
Fraunhofer Institute for Biomedical Technology (IBMT)
- Biomedical Ultrasound Research -
Ensheimer Straße 48
D-66386 St. Ingbert
Tel.: +49 (0)6894 980 220
Fax: +49 (0)6894 980 234
email: marc.fournelle@ibmt.fraunhofer.de