Periodic Reporting for period 2 - OPTIMAL (OPtimal Transport for Identifying Marauder Activities on LiDAR)
Período documentado: 2023-01-16 hasta 2024-01-15
Moving from this overall objective, OPTIMAL specifically intended to:
O1. create the first multi-temporal LiDAR dataset to train change detection methods for the identification of looting activities;
O2. implement a novel change detection method, based on optimal transport, for the automatic identification and monitoring of cultural heritage looted sites directly on LiDAR point cloud time series;
O3. achieve a looting detection accuracy of 85% on two user-case scenarios relying on the ground-truthing data already collected and on the collaboration with landscape archaeologists.
The project outcomes will concur to increase Europe's research profile in the current dominant discourse over the heritage safeguard by offering a powerful machine learning tool for archaeologists and stakeholders involved in the fight against marauder activities which represent a major source of income for criminal groups.
The characterisation and selection of looted sites were performed in the three months of secondment at the Centre for Cultural Heritage Technology (CCHT) of the Italian Institute of Technology (the European host institution). The main output of this secondment was the collection of multi-temporal LiDAR point clouds to identify traces of looting activities provided by archaeologists who collaborated on this project.
In the outgoing phase at Kyoto University, the fellow designed and implemented an unbalanced optimal transport-based pipeline to identify changes related to looting activities on point clouds. The efficacy of this approach was demonstrated both on a test-bench dataset for change detection in an urban environment and on the created dataset for the identification of looting activities. The use of optimal transport for processing point clouds elucidated the depth of the detected looting pits, which is crucial information to understand the state of degradation of the pillaged sites.
In the incoming phase at the CCHT, the fellow transferred the knowledge acquired in optimal transport, optimization, and LiDAR processing to CCHT's PhD students and junior postdocs through supervision activities. The primary outcome of these supervision activities was the submission of two papers to the International Geoscience and Remote Sensing Symposium (IGARSS). In collaboration with Kyoto University, the fellow developed another novel change detection approach based on implicit neural representation that was presented at the Winter Conference on Applications of Computer Vision 2024 (WACV).
A dedicated website was created to serve as a hub providing access to content suitable for both academic and general audiences. Communication activities entailed the participation of the fellow in initiatives such as visits to primary schools as MC Ambassador and training of young students through hands-on workshops. Scientific dissemination was pursued through the presentation of two accepted papers at IGARSS 2023 in Pasadena, California, and a paper at WACV in Hawaii, one of the four top-tier conferences in computer vision. Another two papers were submitted to IGARSS 2024. The fellow disseminates the OPTIMAL project through seven invited talks given at various institutions around the world. Additionally, the fellow organized an international workshop that delved into the combined expertise in the fields of Machine Learning, Remote Sensing, and Landscape Archaeology.
(i) Previous scholars have not taken into consideration the direct use of LiDAR point clouds to detect looting -> OPTIMAL proposed for the first time the use of airborne LiDAR for monitoring and detecting looting activities by constructing the first multi-temporal LiDAR dataset with a machine learning baseline for illegal activities’ identification. This outcome provides insights into the capability of point clouds to identify the 3-D shapes of looting pits, thus fostering the development of novel methods to directly process LiDAR point clouds in archaeological research.
(ii) Literature rarely focuses on the application of optimal transport to detect changes occurring in multi-temporal remote sensing data -> Results of the project show the potential of optimal transport theory to develop a sound computational framework to effectively deal with the change detection problem. Specifically, the developed change detection method based on unbalanced optimal transport showed superior performance over the state-of-the-art on (1) the only publicly available airborne LiDAR test-bench dataset for building change detection; and (2) the dataset created by the fellow for looting identification.
(iii) Machine learning literature showed the effectiveness of neural implicit learning in scene understanding and computer graphics -> In collaboration with the CCHT and Kyoto University, the fellow proposed the use of implicit neural representation as a novel change detection method for detecting looting activities directly on LIDAR point clouds, captured at two successive acquisitions over the same geographical location equally spaced in time.
The OPTIMAL project has already had a crucial impact on the fellow’s career, setting the stage for him to be elevated to the role of ‘Researcher’, a senior position in research within the Italian Institute of Technology.
The outcomes of this project will also have an impact on the current dominant debate on heritage safeguarding by offering a powerful machine learning tool to archaeologists and stakeholders involved in the fight against marauder activities that represent a loss of invaluable properties as well as obliterate modern society's roots.