Final Report Summary - BIO-DISTANCE (BIOMETRICS AT A DISTANCE)
** Objectives, work performed and main results achieved **
Despite the practical importance and the advantages of biometric solutions to the task of verifying personal identity, their adoption has proved to be slower than predicted. Biometrics “on the move” is a hottest research topic aimed to acquire biometric data “at a distance” as a person walks by detection equipment. This drastically reduces the need of user’s cooperation, achieving low intrusiveness and thus, high acceptance and transparency.
With these ideas in mind, the main objective of this proposal is to investigate a number of activities aimed to make biometric technologies applicable to data acquired at a distance and on the move. We propose the use of face and iris as the reference modalities, being the two traits that are attracting more efforts thanks to the possibility of their simultaneous acquisition. This project is an integrated approach that covers the whole structure of a biometric system, including basic research and algorithm development for the different stages of the system, as well as practical results through case studies implementation and evaluation. To accomplish the overall objectives, a number of challenges need to be overcome, which constitute the specific research objectives. These conform the different workpackages or sub-objectives:
1. Face and iris quality measures
2. Face and iris detection and segmentation algorithms
3. Feature extraction and matching algorithms
4. Robust multibiometric fusion
5. Case studies of practical interest
The following is a summary of the work performed in each of these sub-objectives, with the main results achieved.
1. Face and iris quality measures
Several quality measures of the iris region have been implemented and evaluated, both locally (iris boundaries) and globally (whole eye region), including the proposal of novel algorithms. We have aimed to quantify image properties reported in the literature as having the greatest influence in iris recognition accuracy, in support of the standard ISO/IEC 29794-6 Biometric Sample Quality (part 6: Iris image). The algorithms implemented include measurements of: defocus blur (1 algorithm), motion blur and image interlace (1 algorithm), contrast of iris boundaries (2 algorithms), circularity of iris boundaries (1 algorithm), gray scale spread (2 algorithms), and occlusion (1 algorithm).
2. Face and iris detection and segmentation algorithms
A complete system that allows the segmentation of iris boundaries and the detection of occluding eyelids has been developed. Using manual segmentation as benchmark for our experiments, our system has shown a considerable improvement over traditional methods. The results obtained show the validity of our proposed approach and demonstrate that it constitutes an alternative to classic iris segmentation approaches.
We have also analyzed the impact of image defocus and motion blur in the segmentation. Reported results shows the superior resilience of our segmentation system compared with the popular segmentation methods indicated. The impact of the quality measures implemented in the previous sub-objective on the segmentation accuracy has also been evaluated. It has been found that local quality metrics are better predictors of the segmentation accuracy than global metrics, despite the obvious limitation of requiring segmentation. Some measures also behave differently when they are computed locally or globally.
Accurate segmentation of iris boundaries demands the use of good quality images acquired in more or less controlled conditions. With the purpose of going towards a less-controlled and distant acquisition, we have developed a system for detection of the complete eye region. The target is to make detection of human faces possible over a wide range of distances, even when the iris texture cannot be reliably obtained (low resolution, off-angle, etc.) or under partial face occlusion (close distances, viewing angle, etc.). This is a necessary step, for example, to zoom into the iris texture and segment the iris boundaries, or to allow tracking of human faces when the whole face is not available. Existing face detection algorithms model faces in a holistic manner, so they fail in case of face occlusion or sometimes simply with arbitrary (non-frontal) poses. On the other hand, the iris texture can be very difficult to obtain in distant acquisition. The task proposed is an intermediate step, i.e. detection of the eye region, not relying on having neither the full face available nor a well-defined iris texture. Our system has been evaluated with images acquired with a webcam from a portable tablet PC, showing strong detection capabilities of eye regions.
3. Feature extraction and matching algorithms
We have also evaluated the impact of quality components (using the measures developed above) in the performance of two iris matchers. We observe that the matchers are also sensitive to quality variations, but not necessarily in the same way as the segmentation algorithm. Also, one of the matchers is observed to be more resilient to segmentation inaccuracies. In this sense, errors in the segmentation may be hidden by the matcher, pointing out the importance of evaluating also the precision of iris segmentation, as proposed in this project, rather than focusing on recognition accuracy only.
It is relevant to note that the success of iris segmentation is crucial for the good performance of iris recognition systems. Previous research works treat iris recognition systems as a black box, evaluating the impact of quality factors at the output of the system (i.e. by looking at the recognition performance). The same procedure is traditionally followed to evaluate new developments in the segmentation stage. This project has brought innovative aspects in the sense that it has focused on evaluating directly the precision of the segmentation. The observed results, with segmentator and matcher not necessary behaving in the same way, support the proposed evaluation framework.
Experiments also show that quality measures are not necessarily correlated. Quality is intrinsically multi-dimensional and it is affected by factors of very different nature. Research done in this direction includes fusing the estimated quality measures to obtain a single measure with higher prediction capability of the segmentation and matching accuracy. On-going research also includes exploiting the different sensitivity observed in the two matchers, so by using adaptive quality fusion schemes, we are seeking to obtain better performance over a wide range of qualities.
Periocular recognition is a direction that has gained attention recently in biometrics. Periocular refers to the face region in the vicinity of the eye, including the eye, eyelids, lashes and eyebrows. This region can be easily obtained with existing setups for face and iris, and the requirement of user cooperation can be relaxed. An evident advantage of the periocular region is its availability over a wide range of acquisition distances even when the iris texture cannot be reliably obtained (low resolution, off-angle, etc.) or under partial face occlusion (close distances). Most face recognition systems use a holistic approach, requiring a full face image, so the performance is affected in case of occlusion. In addition, current commercial iris systems have constrained image acquisition conditions, prompting the user to position the eye in front of the sensor (normally at ~ 20-40 cm). In the current context of ubiquitous access to information, with the proliferation of portable devices and mobility requirements, the relaxation of acquisition constraints is a factor that have great (if not the greatest) impact in mass acceptance levels of these technologies. Other application fields that are receiving much attention are forensics and surveillance, where the requirement of controlled acquisition is simply impossible. These considerations of a more flexible and less intrusive acquisition are central for the objectives of this project, and therefore periocular work is of high relevance.
Work in this direction has consisted in the development of a new periocular recognition system. Our system achieves competitive verification rates compared with existing periocular recognition approaches. We have also carried out experiments which demonstrate the validity of our recognition system in less-controlled images. For example, it does not need an accurate detection of the iris region, but it is enough with having the center of the eye. This makes this system useful in a wide range of applications where the iris texture cannot be obtained, and provides the way to integrate it with the detection system of eye regions developed in the previous sub-objective.
4. Multibiometric recognition
The periocular system has been also compared with a dedicated iris expert based on a popular public software. The dedicated iris expert works much better than the proposed periocular approach, something that is to be expected since iris images contain more texture. However, this encourages us to continue the improvement of our system. The fusion of iris and periocular systems also shows a relative improvement in some of our experiments. This points out the potential complementarity of the iris texture and the periocular region, as reported in some other works in the literature.
Existing face recognition systems are based on a global model of the face, thus failing if the whole face is not available. We are also working in the fusion of periocular information with other local face regions, where the identity model of the person is built from the available regions. This is the approach being followed in the literature to cope with incomplete faces.
The fellow has participated in ICIR2013, the First ICB Competition on Iris Recognition, organized by the Institute of Automation, Chinese Academy of Sciences (CASIA), in conjunction with the 6th International Conference on Biometrics (ICB-2013). The objective was to evaluate iris recognition algorithms on data including motion blur, non-linear deformation, eyeglasses and specular reflections. The fellow participated with developments of this project. The submission obtained excellent results, ranked in second position out of 13 participating systems.
5. Case studies of practical interest
Embedded in the previous activities, the following case studies have been evaluated in this project:
-Evaluation of developments of this project with controlled acquisition (iris close-up data) and relaxed acquisition data (webcam from a portable tablet PC)
-Participation with developments of this project in ICIR2013, the First ICB Competition on Iris Recognition
Other case studies have also been evaluated during this research:
-Evaluation of several public face detection algorithms on animal faces (dogs, cats, elks). This is of interest for several industrial partners of the host, comprising applications such as home robots or in-car safety systems to avoid e.g. collisions with animals. This task has been done in the framework of a supervised Master thesis, and activities have also comprised building a database of 3000 animal faces with images from Internet.
-Collaboration with PhD student of the former host in Spain to evaluate some developments of this project with very low resolution images from surveillance cameras
-Collaboration with researchers from Salzburg University in a survey of iris segmentation methods, contributing with data and algorithms developed in this project.
** Potential impact and use (including socio-economic impact and societal implications) **
While technological aspects will still play an important role, demands for more consumer convenience will drive innovation in biometrics, with user-centered research actions focused on its transparent use, facilitating the ease of interaction with the system. With face and iris as the reference modalities, the major limitation of current face and iris commercial systems is the degree of control and cooperation required during the acquisition. This project will contribute to increase consumer convenience in the use of biometric systems by reducing the intrusiveness and the required level of cooperation, which have been identified as the key aspect for gaining mass acceptance levels and further widespread adoption of biometric technologies. Applications of the research followed in this project are very diverse, enhancing the customer experience where large number of people moves through a bottleneck (security checkpoints in airports, access control). It will also contribute to make a vast amount of surveillance cameras in non-cooperative or uncontrolled environments suitable for recognition purposes (e.g. placed in hallways or outdoors).
The research of this project concerning eye region detection in less-controlled conditions will be the source of future research of the fellow. Reliable detection of human (and even not human) faces in less-controlled and distant acquisition is the key for many applications, not only personal recognition, bur for example: human-computer interaction, in-car applications (pedestrian collision avoidance, driver drowsiness detection, execution of commands by lip-reading, etc.) or home and industrial robots with autonomous capabilities, including human/pets presence detection. These applications are currently in the agenda of many industrial partners of the host. The relevance of this research is demonstrated by a Project Research Grant for young researchers recently obtained by the fellow (by the Swedish Research Council) to enable further work in the detection of faces in uncontrolled conditions.
In order to overcome the barriers to adoption and effective deployment of biometric systems, this is a user-centered proposal, where novel technological aspects are aimed to increase consumer convenience by reducing the need of user’s cooperation in using biometric systems. Therefore, this proposal will work towards increasing usability and universal access to information systems (facilitating the ease of interaction), user friendliness (reducing the need of cooperation and the intrusiveness) and multiple interfaces (nearly every camera is of potential use here). From an industrial perspective, this project has intersectorial applicability in the sense that research findings can be used in a wide range of applications. Outputs of this project will contribute towards making a vast amount of cameras placed outdoor suitable for recognition purposes, so any application making use of a camera would be virtually usable: commercial environments (e-commerce, e-banking, network login, physical access...) governmental (ID card, e-health, border control, security chekpoints in airports...) forensic (criminal investigation using images from unconstrained enviroments) and surveillance (security cameras). This intersectorial aspect have been already exploited with the analysis of different case-studies of practical interest, comprising iris close-up data, webcam images from Tablet PCs, animal faces, or very low resolution images.
Despite the practical importance and the advantages of biometric solutions to the task of verifying personal identity, their adoption has proved to be slower than predicted. Biometrics “on the move” is a hottest research topic aimed to acquire biometric data “at a distance” as a person walks by detection equipment. This drastically reduces the need of user’s cooperation, achieving low intrusiveness and thus, high acceptance and transparency.
With these ideas in mind, the main objective of this proposal is to investigate a number of activities aimed to make biometric technologies applicable to data acquired at a distance and on the move. We propose the use of face and iris as the reference modalities, being the two traits that are attracting more efforts thanks to the possibility of their simultaneous acquisition. This project is an integrated approach that covers the whole structure of a biometric system, including basic research and algorithm development for the different stages of the system, as well as practical results through case studies implementation and evaluation. To accomplish the overall objectives, a number of challenges need to be overcome, which constitute the specific research objectives. These conform the different workpackages or sub-objectives:
1. Face and iris quality measures
2. Face and iris detection and segmentation algorithms
3. Feature extraction and matching algorithms
4. Robust multibiometric fusion
5. Case studies of practical interest
The following is a summary of the work performed in each of these sub-objectives, with the main results achieved.
1. Face and iris quality measures
Several quality measures of the iris region have been implemented and evaluated, both locally (iris boundaries) and globally (whole eye region), including the proposal of novel algorithms. We have aimed to quantify image properties reported in the literature as having the greatest influence in iris recognition accuracy, in support of the standard ISO/IEC 29794-6 Biometric Sample Quality (part 6: Iris image). The algorithms implemented include measurements of: defocus blur (1 algorithm), motion blur and image interlace (1 algorithm), contrast of iris boundaries (2 algorithms), circularity of iris boundaries (1 algorithm), gray scale spread (2 algorithms), and occlusion (1 algorithm).
2. Face and iris detection and segmentation algorithms
A complete system that allows the segmentation of iris boundaries and the detection of occluding eyelids has been developed. Using manual segmentation as benchmark for our experiments, our system has shown a considerable improvement over traditional methods. The results obtained show the validity of our proposed approach and demonstrate that it constitutes an alternative to classic iris segmentation approaches.
We have also analyzed the impact of image defocus and motion blur in the segmentation. Reported results shows the superior resilience of our segmentation system compared with the popular segmentation methods indicated. The impact of the quality measures implemented in the previous sub-objective on the segmentation accuracy has also been evaluated. It has been found that local quality metrics are better predictors of the segmentation accuracy than global metrics, despite the obvious limitation of requiring segmentation. Some measures also behave differently when they are computed locally or globally.
Accurate segmentation of iris boundaries demands the use of good quality images acquired in more or less controlled conditions. With the purpose of going towards a less-controlled and distant acquisition, we have developed a system for detection of the complete eye region. The target is to make detection of human faces possible over a wide range of distances, even when the iris texture cannot be reliably obtained (low resolution, off-angle, etc.) or under partial face occlusion (close distances, viewing angle, etc.). This is a necessary step, for example, to zoom into the iris texture and segment the iris boundaries, or to allow tracking of human faces when the whole face is not available. Existing face detection algorithms model faces in a holistic manner, so they fail in case of face occlusion or sometimes simply with arbitrary (non-frontal) poses. On the other hand, the iris texture can be very difficult to obtain in distant acquisition. The task proposed is an intermediate step, i.e. detection of the eye region, not relying on having neither the full face available nor a well-defined iris texture. Our system has been evaluated with images acquired with a webcam from a portable tablet PC, showing strong detection capabilities of eye regions.
3. Feature extraction and matching algorithms
We have also evaluated the impact of quality components (using the measures developed above) in the performance of two iris matchers. We observe that the matchers are also sensitive to quality variations, but not necessarily in the same way as the segmentation algorithm. Also, one of the matchers is observed to be more resilient to segmentation inaccuracies. In this sense, errors in the segmentation may be hidden by the matcher, pointing out the importance of evaluating also the precision of iris segmentation, as proposed in this project, rather than focusing on recognition accuracy only.
It is relevant to note that the success of iris segmentation is crucial for the good performance of iris recognition systems. Previous research works treat iris recognition systems as a black box, evaluating the impact of quality factors at the output of the system (i.e. by looking at the recognition performance). The same procedure is traditionally followed to evaluate new developments in the segmentation stage. This project has brought innovative aspects in the sense that it has focused on evaluating directly the precision of the segmentation. The observed results, with segmentator and matcher not necessary behaving in the same way, support the proposed evaluation framework.
Experiments also show that quality measures are not necessarily correlated. Quality is intrinsically multi-dimensional and it is affected by factors of very different nature. Research done in this direction includes fusing the estimated quality measures to obtain a single measure with higher prediction capability of the segmentation and matching accuracy. On-going research also includes exploiting the different sensitivity observed in the two matchers, so by using adaptive quality fusion schemes, we are seeking to obtain better performance over a wide range of qualities.
Periocular recognition is a direction that has gained attention recently in biometrics. Periocular refers to the face region in the vicinity of the eye, including the eye, eyelids, lashes and eyebrows. This region can be easily obtained with existing setups for face and iris, and the requirement of user cooperation can be relaxed. An evident advantage of the periocular region is its availability over a wide range of acquisition distances even when the iris texture cannot be reliably obtained (low resolution, off-angle, etc.) or under partial face occlusion (close distances). Most face recognition systems use a holistic approach, requiring a full face image, so the performance is affected in case of occlusion. In addition, current commercial iris systems have constrained image acquisition conditions, prompting the user to position the eye in front of the sensor (normally at ~ 20-40 cm). In the current context of ubiquitous access to information, with the proliferation of portable devices and mobility requirements, the relaxation of acquisition constraints is a factor that have great (if not the greatest) impact in mass acceptance levels of these technologies. Other application fields that are receiving much attention are forensics and surveillance, where the requirement of controlled acquisition is simply impossible. These considerations of a more flexible and less intrusive acquisition are central for the objectives of this project, and therefore periocular work is of high relevance.
Work in this direction has consisted in the development of a new periocular recognition system. Our system achieves competitive verification rates compared with existing periocular recognition approaches. We have also carried out experiments which demonstrate the validity of our recognition system in less-controlled images. For example, it does not need an accurate detection of the iris region, but it is enough with having the center of the eye. This makes this system useful in a wide range of applications where the iris texture cannot be obtained, and provides the way to integrate it with the detection system of eye regions developed in the previous sub-objective.
4. Multibiometric recognition
The periocular system has been also compared with a dedicated iris expert based on a popular public software. The dedicated iris expert works much better than the proposed periocular approach, something that is to be expected since iris images contain more texture. However, this encourages us to continue the improvement of our system. The fusion of iris and periocular systems also shows a relative improvement in some of our experiments. This points out the potential complementarity of the iris texture and the periocular region, as reported in some other works in the literature.
Existing face recognition systems are based on a global model of the face, thus failing if the whole face is not available. We are also working in the fusion of periocular information with other local face regions, where the identity model of the person is built from the available regions. This is the approach being followed in the literature to cope with incomplete faces.
The fellow has participated in ICIR2013, the First ICB Competition on Iris Recognition, organized by the Institute of Automation, Chinese Academy of Sciences (CASIA), in conjunction with the 6th International Conference on Biometrics (ICB-2013). The objective was to evaluate iris recognition algorithms on data including motion blur, non-linear deformation, eyeglasses and specular reflections. The fellow participated with developments of this project. The submission obtained excellent results, ranked in second position out of 13 participating systems.
5. Case studies of practical interest
Embedded in the previous activities, the following case studies have been evaluated in this project:
-Evaluation of developments of this project with controlled acquisition (iris close-up data) and relaxed acquisition data (webcam from a portable tablet PC)
-Participation with developments of this project in ICIR2013, the First ICB Competition on Iris Recognition
Other case studies have also been evaluated during this research:
-Evaluation of several public face detection algorithms on animal faces (dogs, cats, elks). This is of interest for several industrial partners of the host, comprising applications such as home robots or in-car safety systems to avoid e.g. collisions with animals. This task has been done in the framework of a supervised Master thesis, and activities have also comprised building a database of 3000 animal faces with images from Internet.
-Collaboration with PhD student of the former host in Spain to evaluate some developments of this project with very low resolution images from surveillance cameras
-Collaboration with researchers from Salzburg University in a survey of iris segmentation methods, contributing with data and algorithms developed in this project.
** Potential impact and use (including socio-economic impact and societal implications) **
While technological aspects will still play an important role, demands for more consumer convenience will drive innovation in biometrics, with user-centered research actions focused on its transparent use, facilitating the ease of interaction with the system. With face and iris as the reference modalities, the major limitation of current face and iris commercial systems is the degree of control and cooperation required during the acquisition. This project will contribute to increase consumer convenience in the use of biometric systems by reducing the intrusiveness and the required level of cooperation, which have been identified as the key aspect for gaining mass acceptance levels and further widespread adoption of biometric technologies. Applications of the research followed in this project are very diverse, enhancing the customer experience where large number of people moves through a bottleneck (security checkpoints in airports, access control). It will also contribute to make a vast amount of surveillance cameras in non-cooperative or uncontrolled environments suitable for recognition purposes (e.g. placed in hallways or outdoors).
The research of this project concerning eye region detection in less-controlled conditions will be the source of future research of the fellow. Reliable detection of human (and even not human) faces in less-controlled and distant acquisition is the key for many applications, not only personal recognition, bur for example: human-computer interaction, in-car applications (pedestrian collision avoidance, driver drowsiness detection, execution of commands by lip-reading, etc.) or home and industrial robots with autonomous capabilities, including human/pets presence detection. These applications are currently in the agenda of many industrial partners of the host. The relevance of this research is demonstrated by a Project Research Grant for young researchers recently obtained by the fellow (by the Swedish Research Council) to enable further work in the detection of faces in uncontrolled conditions.
In order to overcome the barriers to adoption and effective deployment of biometric systems, this is a user-centered proposal, where novel technological aspects are aimed to increase consumer convenience by reducing the need of user’s cooperation in using biometric systems. Therefore, this proposal will work towards increasing usability and universal access to information systems (facilitating the ease of interaction), user friendliness (reducing the need of cooperation and the intrusiveness) and multiple interfaces (nearly every camera is of potential use here). From an industrial perspective, this project has intersectorial applicability in the sense that research findings can be used in a wide range of applications. Outputs of this project will contribute towards making a vast amount of cameras placed outdoor suitable for recognition purposes, so any application making use of a camera would be virtually usable: commercial environments (e-commerce, e-banking, network login, physical access...) governmental (ID card, e-health, border control, security chekpoints in airports...) forensic (criminal investigation using images from unconstrained enviroments) and surveillance (security cameras). This intersectorial aspect have been already exploited with the analysis of different case-studies of practical interest, comprising iris close-up data, webcam images from Tablet PCs, animal faces, or very low resolution images.
