Periodic Reporting for period 2 - DigiArt (The Internet Of Historical Things And Building New 3D Cultural Worlds)
Okres sprawozdawczy: 2016-06-01 do 2018-11-30
"DigiArt seeks to provide a new, cost efficient solution to the capture, processing and display of cultural artefacts. It offers innovative 3D capture systems and methodologies, including aerial capture via drones, automatic registration and modelling techniques to speed up post-processing, semantic image analysis to extract features from 3D representations, a ""story telling engine"" offering a pathway to a deeper understanding of art, and also augmented/virtual reality technologies offering advanced abilities for viewing, or interacting with the 3D models. Semantic analysis by automatic feature extraction is used to form hyper-links between artefacts. These links are employed to connect the artefacts in what the project terms ""the internet of historical things"", available anywhere, at any time, on any web-enabled device. The contextual view of art is very much enhanced by the ""story telling engine"" that is developed within the project. The major output of this project is the toolset that is used by the museums to create such a revolutionary way of viewing and experiencing artefacts. These tools leverage the interdisciplinary sill set of the partners to cover the complete process, namely data capture, data processing, story building, 3D visualization and 3D interaction, offering new pathways to deeper understanding of European culture. Via its three demonstration activities, the project establishes the viability of the approach in thee different museum settings, offering a range of artefacts posing different challenges to the system.
The objectives of this research are
Obj.1 Develop highly efficient methods for gathering 3D data of high quality from large and distributed sites by automated means
Obj.2 Make available to workers in cultural heritage a new state-of-the-art Open Source 3D scanner system built from commercially available, low cost, hardware.
Obj.3 Create a suite of software that will allow the analysis of highly redundant point-cloud data from scanners
Obj.4 Develop new methods of generating semantic meaning from 3D solid models and through this meaning drive the connectivity which will realise the internet of historical things
Obj.5 Use the generated 3D content as the enabling landscape for setting-up and offering radically new experiences to remote and on-site visitors
Obj.6 Adoption of our technologies by an expanding community of cultural heritage workers
"
The objectives of this research are
Obj.1 Develop highly efficient methods for gathering 3D data of high quality from large and distributed sites by automated means
Obj.2 Make available to workers in cultural heritage a new state-of-the-art Open Source 3D scanner system built from commercially available, low cost, hardware.
Obj.3 Create a suite of software that will allow the analysis of highly redundant point-cloud data from scanners
Obj.4 Develop new methods of generating semantic meaning from 3D solid models and through this meaning drive the connectivity which will realise the internet of historical things
Obj.5 Use the generated 3D content as the enabling landscape for setting-up and offering radically new experiences to remote and on-site visitors
Obj.6 Adoption of our technologies by an expanding community of cultural heritage workers
"
DigiArt throughout the second and third year of the project, concentrated on completion deliverable assigned for the reporting period. All deliverables have been achieved in all required areas. The outcome of the main results are presented below:
1. The full 3D capture of different cultural heritage were completed namely, the prehistoric cave at Scladina, Belgium; Palace of Aigiai, Greece; Bomarzo Mostri Park, Italy; Daniel Adamson, United Kingdom; Halton Castle, United Kingdom; Williamson Tunnels, United Kingdom and the Royal Tombs of Philip II, Greece
2. The project has successfully developed a low cost, open source, desktop scanner. At present this is at prototype-test level but is already successfully 3D scanning various smaller artefacts. The open-source software and guidance is available on DigiArt website.
3. Project has successfully achieved the synthesis of SfM data with LIDAR data from the cave.
4. The project has develop exciting and interesting “story boards” that are placed within the virtual worlds created by the project’s innovative story engine. These story scenarios centre around Scladina and Aigai and have been developed by the specialist staff at those centres in dialogue through the users group with the technical teams in DigiArt.
5. DigiArt consortium has engaged with different types of museums, archaeological sites and large scale industrial heritage artefacts to promote the DigiArt vision.
1. The full 3D capture of different cultural heritage were completed namely, the prehistoric cave at Scladina, Belgium; Palace of Aigiai, Greece; Bomarzo Mostri Park, Italy; Daniel Adamson, United Kingdom; Halton Castle, United Kingdom; Williamson Tunnels, United Kingdom and the Royal Tombs of Philip II, Greece
2. The project has successfully developed a low cost, open source, desktop scanner. At present this is at prototype-test level but is already successfully 3D scanning various smaller artefacts. The open-source software and guidance is available on DigiArt website.
3. Project has successfully achieved the synthesis of SfM data with LIDAR data from the cave.
4. The project has develop exciting and interesting “story boards” that are placed within the virtual worlds created by the project’s innovative story engine. These story scenarios centre around Scladina and Aigai and have been developed by the specialist staff at those centres in dialogue through the users group with the technical teams in DigiArt.
5. DigiArt consortium has engaged with different types of museums, archaeological sites and large scale industrial heritage artefacts to promote the DigiArt vision.
DigiArt project has demonstrated the progress beyond the state-of-the-art in some research areas. These are briefly outlined below:
Comparative UAV and Ground-based 3D Scanning.
Ground-based scanning, achieved by mounting cameras in an array of positions mounted on tripods and taking a series of images, is a well-established technology. As such it acts as baseline standard. Airborne 3D scanning using a derivative of photogrammetry.
New Flight Path Design methodologies to Optimise SfM.
The airborne scan of the cave using UAVs yielded some excellent data. The resultant 3D model is of very high quality. But the processing time required to produce it was excessive. Investigations were carried out to determine why the processing was taking so much time. This revealed that the principal problem was not the large number of mathematical operations required, but the constant swapping in and out of memory of images in the sequence separated by considerable temporal distance and only small physical distances.
GPS Denied Enclosed Space 3D Scanning.
We have shown via our UAV scan of the Scladina cave, that it is possible to gather comprehensive 3D data on the boundaries of a completely enclosed space which is GPS denied.
The Desktop Open Source Scanner.
The realisation of the new desktop, static, scanner offers the prospect of ultra-low-cost 3D scanning capability for every museum in Europe.
Using Deep Learning Architectures for 3D Data Analysis.
Recent advancements in 3D sensing technology and the appearance of low-cost devices such as Microsoft Kinect have made the collection of 3D data more feasible and affordable than ever. Based on the scanning device employed for capturing the 3D scene or object of interest, raw data are collected in different forms. UAV scanners get range images from different camera viewpoints. Then, these images are typically combined through a registration process or Structure-from-Motion techniques (SfM) in order to discard noisy data, establish correspondences between them and ultimately generate a unified 3D point cloud for further processing. The increasing abundance of 3D data encouraged the research community to exploit this richer content for addressing several computer vision problems related to understanding 3D scenes, e.g. 3D Object Classification, 3D Object Recognition and 3D Shape Retrieval.
Story Telling Engine in a Flexible Architecture.
In designing the architecture of our story-telling engine, we had to address a number of requirements including expandability, generalizability, user-friendliness, immersiveness, compatibility to several platforms, and the ability for the final product to be commercialised. In the past several attempts to make VR/AR tours were discontinued due to the lack of an appropriate architecture that will be self-sustained and commercially active. The architecture that has been proposed as part of WP6 activities focuses on reducing, as much as possible, the interference of a programmer or a game designer so that the main target group, i.e. the curators, can easily make their own VR/AR tours and maintain it. This was achieved by proposing an architecture that interconnects a web-based interface for 3D content management with the powerful game making engine of Unity. In this way, we were able to abstract the process of setting the 3D scene and defining triggers on the 3D objects, without sacrificing the quality of the resulting game.
Comparative UAV and Ground-based 3D Scanning.
Ground-based scanning, achieved by mounting cameras in an array of positions mounted on tripods and taking a series of images, is a well-established technology. As such it acts as baseline standard. Airborne 3D scanning using a derivative of photogrammetry.
New Flight Path Design methodologies to Optimise SfM.
The airborne scan of the cave using UAVs yielded some excellent data. The resultant 3D model is of very high quality. But the processing time required to produce it was excessive. Investigations were carried out to determine why the processing was taking so much time. This revealed that the principal problem was not the large number of mathematical operations required, but the constant swapping in and out of memory of images in the sequence separated by considerable temporal distance and only small physical distances.
GPS Denied Enclosed Space 3D Scanning.
We have shown via our UAV scan of the Scladina cave, that it is possible to gather comprehensive 3D data on the boundaries of a completely enclosed space which is GPS denied.
The Desktop Open Source Scanner.
The realisation of the new desktop, static, scanner offers the prospect of ultra-low-cost 3D scanning capability for every museum in Europe.
Using Deep Learning Architectures for 3D Data Analysis.
Recent advancements in 3D sensing technology and the appearance of low-cost devices such as Microsoft Kinect have made the collection of 3D data more feasible and affordable than ever. Based on the scanning device employed for capturing the 3D scene or object of interest, raw data are collected in different forms. UAV scanners get range images from different camera viewpoints. Then, these images are typically combined through a registration process or Structure-from-Motion techniques (SfM) in order to discard noisy data, establish correspondences between them and ultimately generate a unified 3D point cloud for further processing. The increasing abundance of 3D data encouraged the research community to exploit this richer content for addressing several computer vision problems related to understanding 3D scenes, e.g. 3D Object Classification, 3D Object Recognition and 3D Shape Retrieval.
Story Telling Engine in a Flexible Architecture.
In designing the architecture of our story-telling engine, we had to address a number of requirements including expandability, generalizability, user-friendliness, immersiveness, compatibility to several platforms, and the ability for the final product to be commercialised. In the past several attempts to make VR/AR tours were discontinued due to the lack of an appropriate architecture that will be self-sustained and commercially active. The architecture that has been proposed as part of WP6 activities focuses on reducing, as much as possible, the interference of a programmer or a game designer so that the main target group, i.e. the curators, can easily make their own VR/AR tours and maintain it. This was achieved by proposing an architecture that interconnects a web-based interface for 3D content management with the powerful game making engine of Unity. In this way, we were able to abstract the process of setting the 3D scene and defining triggers on the 3D objects, without sacrificing the quality of the resulting game.