Periodic Reporting for period 2 - OPTIMAL (OPtimal Transport for Identifying Marauder Activities on LiDAR)
Berichtszeitraum: 2023-01-16 bis 2024-01-15
Illegal excavation of archaeological sites to collect historical material culture ("looting") is a pressing problem on a global scale with strong consequences on security, economics, and society. Looting is the main source of income for terroristic groups and organised crime undermining the security and development of the affected countries. The monitoring of looting (past and ongoing) thus plays a crucial role in the protection of cultural heritage by strengthening the ability of Law Enforcement Agencies to promptly react to criminal activities. Due to the spread of the phenomenon and the impossibility of physically inspecting unreachable areas (e.g. forests covered by thick and closed canopies) or hazardous zones, surveillance via remote sensing is the most efficient approach to monitor looting activities. The OPTIMAL (OPtimal Transport for Identifying Marauder Activities on LiDAR) project aims to undermine the illegal excavation of cultural heritage sites by developing an efficient and principled machine learning approach, based on optimal transport, to automatically detect past and present looting directly on airborne Light Detection And Ranging (LiDAR) point cloud time-series.
Moving from this overall objective, OPTIMAL specifically intended to:
O1. create the first multi-temporal LiDAR dataset to train change detection methods for the identification of looting activities;
O2. implement a novel change detection method, based on optimal transport, for the automatic identification and monitoring of cultural heritage looted sites directly on LiDAR point cloud time series;
O3. achieve a looting detection accuracy of 85% on two user-case scenarios relying on the ground-truthing data already collected and on the collaboration with landscape archaeologists.
The project outcomes will concur to increase Europe's research profile in the current dominant discourse over the heritage safeguard by offering a powerful machine learning tool for archaeologists and stakeholders involved in the fight against marauder activities which represent a major source of income for criminal groups.
Moving from this overall objective, OPTIMAL specifically intended to:
O1. create the first multi-temporal LiDAR dataset to train change detection methods for the identification of looting activities;
O2. implement a novel change detection method, based on optimal transport, for the automatic identification and monitoring of cultural heritage looted sites directly on LiDAR point cloud time series;
O3. achieve a looting detection accuracy of 85% on two user-case scenarios relying on the ground-truthing data already collected and on the collaboration with landscape archaeologists.
The project outcomes will concur to increase Europe's research profile in the current dominant discourse over the heritage safeguard by offering a powerful machine learning tool for archaeologists and stakeholders involved in the fight against marauder activities which represent a major source of income for criminal groups.
The project set a benchmark in the use of optimal transport for identifying looting activities directly on a bi-temporal pair of airborne LiDAR point clouds collected over the same geographical area.
The characterisation and selection of looted sites were performed in the three months of secondment at the Centre for Cultural Heritage Technology (CCHT) of the Italian Institute of Technology (the European host institution). The main output of this secondment was the collection of multi-temporal LiDAR point clouds to identify traces of looting activities provided by archaeologists who collaborated on this project.
In the outgoing phase at Kyoto University, the fellow designed and implemented an unbalanced optimal transport-based pipeline to identify changes related to looting activities on point clouds. The efficacy of this approach was demonstrated both on a test-bench dataset for change detection in an urban environment and on the created dataset for the identification of looting activities. The use of optimal transport for processing point clouds elucidated the depth of the detected looting pits, which is crucial information to understand the state of degradation of the pillaged sites.
In the incoming phase at the CCHT, the fellow transferred the knowledge acquired in optimal transport, optimization, and LiDAR processing to CCHT's PhD students and junior postdocs through supervision activities. The primary outcome of these supervision activities was the submission of two papers to the International Geoscience and Remote Sensing Symposium (IGARSS). In collaboration with Kyoto University, the fellow developed another novel change detection approach based on implicit neural representation that was presented at the Winter Conference on Applications of Computer Vision 2024 (WACV).
A dedicated website was created to serve as a hub providing access to content suitable for both academic and general audiences. Communication activities entailed the participation of the fellow in initiatives such as visits to primary schools as MC Ambassador and training of young students through hands-on workshops. Scientific dissemination was pursued through the presentation of two accepted papers at IGARSS 2023 in Pasadena, California, and a paper at WACV in Hawaii, one of the four top-tier conferences in computer vision. Another two papers were submitted to IGARSS 2024. The fellow disseminates the OPTIMAL project through seven invited talks given at various institutions around the world. Additionally, the fellow organized an international workshop that delved into the combined expertise in the fields of Machine Learning, Remote Sensing, and Landscape Archaeology.
The characterisation and selection of looted sites were performed in the three months of secondment at the Centre for Cultural Heritage Technology (CCHT) of the Italian Institute of Technology (the European host institution). The main output of this secondment was the collection of multi-temporal LiDAR point clouds to identify traces of looting activities provided by archaeologists who collaborated on this project.
In the outgoing phase at Kyoto University, the fellow designed and implemented an unbalanced optimal transport-based pipeline to identify changes related to looting activities on point clouds. The efficacy of this approach was demonstrated both on a test-bench dataset for change detection in an urban environment and on the created dataset for the identification of looting activities. The use of optimal transport for processing point clouds elucidated the depth of the detected looting pits, which is crucial information to understand the state of degradation of the pillaged sites.
In the incoming phase at the CCHT, the fellow transferred the knowledge acquired in optimal transport, optimization, and LiDAR processing to CCHT's PhD students and junior postdocs through supervision activities. The primary outcome of these supervision activities was the submission of two papers to the International Geoscience and Remote Sensing Symposium (IGARSS). In collaboration with Kyoto University, the fellow developed another novel change detection approach based on implicit neural representation that was presented at the Winter Conference on Applications of Computer Vision 2024 (WACV).
A dedicated website was created to serve as a hub providing access to content suitable for both academic and general audiences. Communication activities entailed the participation of the fellow in initiatives such as visits to primary schools as MC Ambassador and training of young students through hands-on workshops. Scientific dissemination was pursued through the presentation of two accepted papers at IGARSS 2023 in Pasadena, California, and a paper at WACV in Hawaii, one of the four top-tier conferences in computer vision. Another two papers were submitted to IGARSS 2024. The fellow disseminates the OPTIMAL project through seven invited talks given at various institutions around the world. Additionally, the fellow organized an international workshop that delved into the combined expertise in the fields of Machine Learning, Remote Sensing, and Landscape Archaeology.
The outcomes of the project provide a new study basis for future research relative to the use of machine learning and remote sensing for the automatic identification and monitoring of cultural heritage looting activities.
(i) Previous scholars have not taken into consideration the direct use of LiDAR point clouds to detect looting -> OPTIMAL proposed for the first time the use of airborne LiDAR for monitoring and detecting looting activities by constructing the first multi-temporal LiDAR dataset with a machine learning baseline for illegal activities’ identification. This outcome provides insights into the capability of point clouds to identify the 3-D shapes of looting pits, thus fostering the development of novel methods to directly process LiDAR point clouds in archaeological research.
(ii) Literature rarely focuses on the application of optimal transport to detect changes occurring in multi-temporal remote sensing data -> Results of the project show the potential of optimal transport theory to develop a sound computational framework to effectively deal with the change detection problem. Specifically, the developed change detection method based on unbalanced optimal transport showed superior performance over the state-of-the-art on (1) the only publicly available airborne LiDAR test-bench dataset for building change detection; and (2) the dataset created by the fellow for looting identification.
(iii) Machine learning literature showed the effectiveness of neural implicit learning in scene understanding and computer graphics -> In collaboration with the CCHT and Kyoto University, the fellow proposed the use of implicit neural representation as a novel change detection method for detecting looting activities directly on LIDAR point clouds, captured at two successive acquisitions over the same geographical location equally spaced in time.
The OPTIMAL project has already had a crucial impact on the fellow’s career, setting the stage for him to be elevated to the role of ‘Researcher’, a senior position in research within the Italian Institute of Technology.
The outcomes of this project will also have an impact on the current dominant debate on heritage safeguarding by offering a powerful machine learning tool to archaeologists and stakeholders involved in the fight against marauder activities that represent a loss of invaluable properties as well as obliterate modern society's roots.
(i) Previous scholars have not taken into consideration the direct use of LiDAR point clouds to detect looting -> OPTIMAL proposed for the first time the use of airborne LiDAR for monitoring and detecting looting activities by constructing the first multi-temporal LiDAR dataset with a machine learning baseline for illegal activities’ identification. This outcome provides insights into the capability of point clouds to identify the 3-D shapes of looting pits, thus fostering the development of novel methods to directly process LiDAR point clouds in archaeological research.
(ii) Literature rarely focuses on the application of optimal transport to detect changes occurring in multi-temporal remote sensing data -> Results of the project show the potential of optimal transport theory to develop a sound computational framework to effectively deal with the change detection problem. Specifically, the developed change detection method based on unbalanced optimal transport showed superior performance over the state-of-the-art on (1) the only publicly available airborne LiDAR test-bench dataset for building change detection; and (2) the dataset created by the fellow for looting identification.
(iii) Machine learning literature showed the effectiveness of neural implicit learning in scene understanding and computer graphics -> In collaboration with the CCHT and Kyoto University, the fellow proposed the use of implicit neural representation as a novel change detection method for detecting looting activities directly on LIDAR point clouds, captured at two successive acquisitions over the same geographical location equally spaced in time.
The OPTIMAL project has already had a crucial impact on the fellow’s career, setting the stage for him to be elevated to the role of ‘Researcher’, a senior position in research within the Italian Institute of Technology.
The outcomes of this project will also have an impact on the current dominant debate on heritage safeguarding by offering a powerful machine learning tool to archaeologists and stakeholders involved in the fight against marauder activities that represent a loss of invaluable properties as well as obliterate modern society's roots.