Archived data gets a second life with a new application for architects and virtual reality game designers
Our societies almost literally bathe in data. Some 2.5 quintillion bytes are produced by humans every day. In the last 2 years alone, we generated 90 % of the world’s currently available data. For some of the most creative of us, say, virtual reality (VR) game designers and architects, this amount of information is a blessing. But its heterogeneous nature calls for innovative means to reuse it by means of 3D reconstructions and models. This is precisely what V4Design (Visual and textual content re-purposing FOR(4) architecture, Design and video virtual reality games) is about, as Stefanos Vrochidis, senior researcher at the Centre for Research and Technology Hellas (CERTH) and coordinator of the project, explains. “V4Design provides the ability to reuse and repurpose existing visual and textual content from content providers and public web resources. We integrated and combined state-of-the-art technologies in computer vision, 3D generation, text analysis and generation, as well as semantic integration and interlinking. With these, we provide architects, VR game developers and designers with innovative tools to reuse and repurpose heterogeneous multimedia content.” Imagine you’re a game designer and would like to draw inspiration from archive footage and documentaries. Of course, these are great sources of inspiration, but there are currently no means to repurpose them. V4Design fills this gap by performing automatic content analysis and seamless transformation into exploitable 3D reconstructions. It’s a considerable time and, by extension, money saver. “Architects and video game designers currently use conventional analogue prototypes. These include scale models and physical demo environments such as rooms and apartments. Building such prototypes is much more expensive and time-consuming than using existing digital content. Additionally, these prototypes are static,” Vrochidis points out. With V4Design, producing prototypes is faster, cheaper and more efficient, and allows for easy modifications on top of 3D dynamic models.
A world of opportunities
V4Design encompasses two authoring tools: One for VR game designers and one for architects. Both apply innovative 3D model reconstruction techniques to the visual content (videos and images) in order to extract 3D assets of interest like buildings and objects. From there, computer vision and text analysis solutions process the content, extract annotations and dynamically enrich the generated 3D models with information. By the end of the workflow, all available information is semantically interlinked in rich knowledge graphs, offering advanced indexing and retrieval capabilities. Meanwhile, text generation is used to create multilingual summaries of the assets and assist end users in using the information. The V4Design project is due to be completed in March 2021, however the two solutions are already operational. The one for architects is presented as a plugin built on top of the well-established Rhinoceros 3D application, while the second one for game designers can be launched either through the Unity game engine or directly via the VR environment. Architects can use the application for architectural design related to existing or historical buildings and sites. Meanwhile, VR game designers can, for instance, create a time-travel VR experience. But Georgios Meditskos, postdoctoral research associate at ITI-CERTH and also the technical manager of the project, doesn’t exclude other potential applications. “Examples include first response in a disaster event, where the creation of a 3D, semantically enriched environment could increase the situational awareness of every person involved and improve decision-making,” he outlines. “In healthcare, we could create digital twins of patients and integrate them inside a VR environment for high-value monitoring and treatment solutions. We could also think of automotive design and cultural heritage.” The project is now in its final phase and the team are busy preparing the final version of the platform. This work includes rigorous testing and resolving of any emerging issues to guarantee the high quality and reliability of the platform. “With the last stages of the project now in full swing, we are organising the final evaluation phase to validate and assess the solution in real world architecture and VR game design use cases,” Meditskos concludes.
Keywords
V4Design, architecture, virtual reality, archive, visual content, textual information, 3D reconstruction