Final Report Summary - INFRASTRUCTUREMODELS (AUTOMATED AS-BUILT MODELLING OF THE BUILT INFRASTRUCTURE)
The research objective of this project is to test whether a novel framework proposed by the fellow can reasonably detect and classify common building objects from 3D point clouds of infrastructure, for the purpose of significantly reducing the time it takes to create a Building Information Model (BIM) of an existing facility. In practice, such models are manually created by professional modellers who spend over half of their time in modelling frequent, common objects, such as structural components, piping and valves in industrial facilities, and HVAC ducts in residential or office buildings. The scope of this project is to automate the detection of those objects that are distinguishable and frequently encountered, for as-built modelling purposes.
Work packages 1, 2 and 3 have been completed for bridge structures. The first two work packages have been completed for buildings as well. In WP1 step one, Dr Brilakis and his team have selected the representative civil infrastructure element types and identified their visually distinctive characteristics for three cases: bridges, buildings, and industrial facilities. The focus was purely on frequently encountered bulk element types such as structural members, and pipes. In this step, visual and shape characteristics related to the colour, texture, structure, and geometric properties of each element type, as well as its transparency information, were investigated to reveal the aspects that make them most distinctive, from a civil engineering perspective. This was successful, as most infrastructure elements are solid objects that have a) Uniform texture and colour patterns, b) Symmetrical and/or “simple” structure (elements can be easily decomposed to basic shapes, such as parallelograms, polygons and ellipses), and c) few contoured edges (straight lines and sharp angles).
Dr Brilakis’ team then proceeded to step 2, to identify image features suitable for numerically representing the distinctive characteristics. This step was easier than previously thought, as the low level features most suitable for the chosen elements were common across many element categories (e.g. planes, lines, etc. The team investigated the use of multiple machine learning classifiers and statistical structure trainers to “learn” the representation accuracy of each feature and its optimal parameters. Additional tests took place with feature selection mechanisms for expressing characteristics with statistics. Machine learning tools such as AdaBoost and its variants were used for selecting features using cascades of weak classifiers. SVM was used to increment the discriminative power by increasing the margins between classes in the feature space.
Dr Brilakis’ team then proceeded to step 3, which involves representing relative topology of element characteristics and correlating them across views. The initial hypothesis proved false, so this led the research team to include point cloud data in the learning process. The team inverted the detection process by starting from the point cloud data first and then enhancing the results with the visual appearance. This eliminated the need for WP2 entirely, as the existence of point clouds with all visual data registered on top solved the posed problem. However it inserted a number of new problems that stem from the data quality and occlusions typically common in point cloud data, and the computational efficiency of learning objects in combined 3D and 2D datasets. These challenges are being addressed with a follow up project sponsored by Trimble.
WP3 was addressed as described in the proposal. The mapping of elements into labelled point clusters is a straightforward geometry problem in most cases. The research uncovered another problem not entirely expected at the beginning; the polymorphy of object types. The initial assumption of this project was that most infrastructure objects can be represented with some form of cuboid, rhomboid, or spherical representation. The imperfections of the geometry could then be accounted for with the addition of parametric skews in all three dimensions. In reality, we encountered many objects that did not fit the assumption above, and would have to be further dissected into a combination of multiple cuboids, rhomboids, etc. This would hamper subsequent efforts for geometry processing (mapping textures, analysing structural stability, etc.). Eventually, we opted for a patch-based representation of the geometry to ensure that we retain the real world imperfections. This would then allow future software to convert the patches into any other geometric form that would better fit its needs.
Overall, the project achieved its initial goals. It delivered a robust methodology for modelling existing facilities with applications in bridges and buildings. The potential impact of the work stems from the fact that only very few constructed facilities today can afford to have a complete record of as-built information due to inadequate current technology; thus, as-is models are not produced for the vast majority of new construction and retrofit projects. This leads to productivity loss through rework and design changes that cost up to 10% of the installed costs and produces unnecessary material waste. Any efforts towards automating as-is modelling will increase the percentage of infrastructure projects being modelled and/or their design BIMs updated. As-is models would then enable the next IT wave, by facilitating data from smart infrastructure, enabling learning through big data, and addressing data security and resiliency concerns.
Work packages 1, 2 and 3 have been completed for bridge structures. The first two work packages have been completed for buildings as well. In WP1 step one, Dr Brilakis and his team have selected the representative civil infrastructure element types and identified their visually distinctive characteristics for three cases: bridges, buildings, and industrial facilities. The focus was purely on frequently encountered bulk element types such as structural members, and pipes. In this step, visual and shape characteristics related to the colour, texture, structure, and geometric properties of each element type, as well as its transparency information, were investigated to reveal the aspects that make them most distinctive, from a civil engineering perspective. This was successful, as most infrastructure elements are solid objects that have a) Uniform texture and colour patterns, b) Symmetrical and/or “simple” structure (elements can be easily decomposed to basic shapes, such as parallelograms, polygons and ellipses), and c) few contoured edges (straight lines and sharp angles).
Dr Brilakis’ team then proceeded to step 2, to identify image features suitable for numerically representing the distinctive characteristics. This step was easier than previously thought, as the low level features most suitable for the chosen elements were common across many element categories (e.g. planes, lines, etc. The team investigated the use of multiple machine learning classifiers and statistical structure trainers to “learn” the representation accuracy of each feature and its optimal parameters. Additional tests took place with feature selection mechanisms for expressing characteristics with statistics. Machine learning tools such as AdaBoost and its variants were used for selecting features using cascades of weak classifiers. SVM was used to increment the discriminative power by increasing the margins between classes in the feature space.
Dr Brilakis’ team then proceeded to step 3, which involves representing relative topology of element characteristics and correlating them across views. The initial hypothesis proved false, so this led the research team to include point cloud data in the learning process. The team inverted the detection process by starting from the point cloud data first and then enhancing the results with the visual appearance. This eliminated the need for WP2 entirely, as the existence of point clouds with all visual data registered on top solved the posed problem. However it inserted a number of new problems that stem from the data quality and occlusions typically common in point cloud data, and the computational efficiency of learning objects in combined 3D and 2D datasets. These challenges are being addressed with a follow up project sponsored by Trimble.
WP3 was addressed as described in the proposal. The mapping of elements into labelled point clusters is a straightforward geometry problem in most cases. The research uncovered another problem not entirely expected at the beginning; the polymorphy of object types. The initial assumption of this project was that most infrastructure objects can be represented with some form of cuboid, rhomboid, or spherical representation. The imperfections of the geometry could then be accounted for with the addition of parametric skews in all three dimensions. In reality, we encountered many objects that did not fit the assumption above, and would have to be further dissected into a combination of multiple cuboids, rhomboids, etc. This would hamper subsequent efforts for geometry processing (mapping textures, analysing structural stability, etc.). Eventually, we opted for a patch-based representation of the geometry to ensure that we retain the real world imperfections. This would then allow future software to convert the patches into any other geometric form that would better fit its needs.
Overall, the project achieved its initial goals. It delivered a robust methodology for modelling existing facilities with applications in bridges and buildings. The potential impact of the work stems from the fact that only very few constructed facilities today can afford to have a complete record of as-built information due to inadequate current technology; thus, as-is models are not produced for the vast majority of new construction and retrofit projects. This leads to productivity loss through rework and design changes that cost up to 10% of the installed costs and produces unnecessary material waste. Any efforts towards automating as-is modelling will increase the percentage of infrastructure projects being modelled and/or their design BIMs updated. As-is models would then enable the next IT wave, by facilitating data from smart infrastructure, enabling learning through big data, and addressing data security and resiliency concerns.