Marine INspection rObotic Assistant System

Final Report Summary - MINOAS (Marine INspection rObotic Assistant System)

Executive Summary:
The increasing competitiveness in marine operations creates a need for new system concepts that introduce high technology added value products, facilitate the processes involved and minimize the downtimes. The need, thus, lies not only in incorporating the technological means so far available, but in changing the way the corresponding authorities stand against the challenges at hand.
MINOAS project proposed the utilization of a fleet of robots (with underlying infrastructure) which move onboard the ship and provide information (visual and measurements) about multiple points across the vessel at the same time, without the need for the human inspector to climb up scaffolding or use cherry pickers. The gathered measurements (e.g. data, images, videos) can be post-processed and kept for future reference.
MINOAS aimed at reengineering the overall vessel-inspection methodology, by introducing this innovative system concept that incorporates state of the art technologies, but at the same time formulates a new standardization of the overall inspection process. Through a holistic approach, the project proposed the development of a new infrastructure able to support the human personnel by high locomotion enabled robots, which "teleport" the human inspector from a control room to any vessel hold. The human's perceptual abilities are enhanced through the utilization of high resolution tools (e.g. sensors) and are augmented through the parallel processing property provided by MINOAS. Control techniques and algorithms provide a semi-autonomous nature to the operation of the robot fleet, while through graphical interfaces, the human inspector is provided with tools for planning the robotized inspection, and storing and processing the harvested data, supplying the inspector with an effective Decision Support System.
By means of MINOAS, human personnel can be withdrawn from the hazardous environment of a vessel under inspection, and the employment for temporary staging in large compartments can be significantly reduced. Surveyors are "teleported" onboard using virtual reality and stereographic vision, which allows them to review and extract results without the psychological pressure of actually working within a non-friendly environment.
MINOAS aimed at facilitating the inspection process by:
a) Providing ease of access on over & under water regions of a vessel through the incorporation of a multi-disciplinary robot fleet
b) Commutating the gathered data to a central collection point
c) Providing additional software tools for the on-line elaboration of the assembled information
d) Recording of the overall process, hence providing the opportunity of future reference (this also potentially allows to point out deficiencies in the process followed)

Expected outcomes in implementing the MINOAS system with regards to the traditional inspection procedures are:
1. Minimization of the downtime related to the preparation of the inspection procedures due to the parallel deployment of the robotic platforms utilized.
2. A more expedite inspection procedure by the utilization of robots to prepare the ground and obtain measurements in the form of images, video and thickness data.
3. Withdrawal of human personnel from hazardous areas.
4. A more systematic inspection methodology that leads to minimization of the inspection time with enhanced quality.
5. Removal of the need of extensive staging and other temporary arrangements traditionally required through the incorporation of advanced technological means, thus minimizing the costs of the overall inspection procedure.
6. Reduction of the overall inspection cost for a specific vessel due to the modularity of the platform proposed by MINOAS, as it provides a repeatability property to the inspection.
7. Increase of the inspection quality, which increases the safe operating conditions of the vessels and prolong their life-cycle.
8. Minimization of trade costs due to the vessels increased lifecycle and operational time
9. Increase of the environmental protection through the elevation of the inspection quality.

Project Context and Objectives:
The increasing competitiveness in marine operations creates a need for new system concepts that introduce high technology added value products, facilitate the processes involved and minimize the downtimes. The need, thus, lies not only in incorporating the technological means so far available, but in changing the way the corresponding authorities stand against the challenges at hand.
MINOAS project proposed the utilization of a fleet of robots (with underlying infrastructure) which move onboard the ship and provide information (visual and measurements) about multiple points across the vessel at the same time, without the need for the human inspector to climb up scaffolding or use cherry pickers. The gathered measurements (e.g. data, images, videos) can be post-processed and kept for future reference.
MINOAS aimed at reengineering the overall vessel-inspection methodology, by introducing this innovative system concept that incorporates state of the art technologies, but at the same time formulates a new standardization of the overall inspection process. Through a holistic approach, the project proposed the development of a new infrastructure able to support the human personnel by high locomotion enabled robots, which "teleport" the human inspector from a control room to any vessel hold. The human's perceptual abilities are enhanced through the utilization of high resolution tools (e.g. sensors) and are augmented through the parallel processing property provided by MINOAS. Control techniques and algorithms provide a semi-autonomous nature to the operation of the robot fleet, while through graphical interfaces, the human inspector is provided with tools for planning the robotized inspection, and storing and processing the harvested data, supplying the inspector with an effective Decision Support System.
By means of MINOAS, human personnel can be withdrawn from the hazardous environment of a vessel under inspection, and the employment for temporary staging in large compartments can be significantly reduced. Surveyors are "teleported" onboard using virtual reality and stereographic vision, which allows them to review and extract results without the psychological pressure of actually working within a non-friendly environment.
MINOAS aimed at facilitating the inspection process by:
a) Providing ease of access on over & under water regions of a vessel through the incorporation of a multi-disciplinary robot fleet
b) Commutating the gathered data to a central collection point
c) Providing additional software tools for the on-line elaboration of the assembled information
d) Recording of the overall process, hence providing the opportunity of future reference (this also potentially allows to point out deficiencies in the process followed)

Expected outcomes in implementing the MINOAS system to the traditional inspection procedures are:
1. Minimization of the downtime related to the preparation of the inspection procedures due to the parallel deployment of the robotic platforms utilized.
2. A more expedite inspection procedure by the utilization of robots to prepare the ground and obtain measurements in the form of images, video and thickness data.
3. Withdrawal of human personnel from hazardous areas.
4. A more systematic inspection methodology that leads to minimization of the inspection time with enhanced quality.
5. Removal of the need of extensive staging and other temporary arrangements traditionally required through the incorporation of advanced technological means, thus minimizing the costs of the overall inspection procedure.
6. Reduction of the overall inspection cost for a specific vessel due to the modularity of the platform proposed by MINOAS, as it provides a repeatability property to the inspection.
7. Increase of the inspection quality, which increases the safe operating conditions of the vessels and prolong their life-cycle.
8. Minimization of trade costs due to the vessels increased lifecycle and operational time
9. Increase of the environmental protection through the elevation of the inspection quality.

The MINOAS project vision was realised through the integration of state of the art technologies in the fields of robotics, visual perception and man-machine interface. However, during the project it was found that some specific requirements were not met by the already existing technologies; in these cases, new platforms and tools have been developed.

The MINOAS system relies on six vehicles that are actually made up over four basic platforms:
a micro-aerial vehicle, which is expected to provide with an overall view of the state of the vessel and to detect potential critical areas
a lightweight magnetic crawler, which can be easily deployed onboard and provides close-up visual feedbacks
a heavyweight magnetic crawler, which is fitted with a dedicated robotic arm able to prepare the surface and perform ultrasonic thickness measurements
an underwater vehicle, able to perform underwater visual inspections and thickness measurements, particularly in tanks

Both the lightweight and the heavyweight crawlers can be employed also for marking areas where defects are found.

Originally, the main focus of MINOAS was the vessels hull inspection in the context of vessels surveys by classification societies. Hull inspection during class surveys is mandatory to verify compliance with the statutory requirements of the international conventions governing trading vessels. It engulfs the majority (if not all) of the challenges that a robotic platform for vessels hull inspection must address. Nevertheless, classification societies are not the only stakeholder in ensuring the good condition of a vessel. Owners, charterers and prospective buyers have direct interest to ensure that the vessel is currently at a good condition and maintained properly.
Taking advantage of the modular approach adopted in the design of the system, three operating scenarios have been identified and analysed, including also cases where hull inspection is not performed in conjunction with class surveys, but are still of high interest for safety and/or commercial reasons.

Beyond the outcomes expected while introducing the MINOAS concept in the maritime industry, some results emerging from the evolution and the realization of the MINOAS vision are immediately useable in various applications within and beyond the maritime industry. That is, the MINOAS idea is based on the interconnection of different layers of technological tools that have a market value even as stand-alone applications.

The main applications are listed below:
- Self-localisation technique and software
- External localization tools and software
- Algorithm and software for robot task allocation
- Algorithm for path planning
- Networking architecture in harsh environments
- Integrated application for image processing and pattern recognition in maritime environments
- Man Machine Interface and GUI prototype for the MINOAS controller

Finally, tentative draft procedures have been developed for the employment of the MINOAS system in ship inspections.

Project Results:
The aerial scout:
Among the different kinds of helicopter designs that have been proposed so far, multi-rotor configurations present several advantages over comparably scale helicopters. Within these configurations, the four-rotor, or quadrotor, is emerging as the most popular design. The MINOAS aerial scout belongs to this class of vehicle. Being more precise, it is based on the well-known Pelican platform from Ascending Technologies. This is a 50 cm-diameter platform with 10-inch propellers, able to carry a payload of 500g, and equipped with a barometric pressure sensor for height estimation, a GPS receiver and an inertial measuring unit (IMU), comprising a 3-axis gyroscope, a 3-axis accelerometer, and a 3-axis magnetometer. Attitude stabilization control loops making use of those sensors run over an ARM7 microcontroller as part of the platform firmware; the manufacturer leaves almost free an additional secondary ARM7 microcontroller, so that higher-level control loops (e.g. a position controller) can also be run onboard.
The UAV has been equipped with a Hokuyo URG-UTM-30LX laser, which is able to detect a range of approximately 30 meters. This sensor is used for obstacle avoidance and to estimate the platform motion by matching consecutive laser scans. The device is also used, by deflection of lateral beams using mirrors, to estimate distance to the floor as well as to the ceiling. This method has been found more adequate for the application instead of using the barometric pressure sensor or the GPS, which tend to show large variations around the true height, making height stabilization difficult when the platform navigates indoors or relatively close to other objects. The vehicle also features a set of uEye 1226-LE-C cameras in order to provide visual information during the flight: one fixed bottom-looking device allows obtaining pictures from the ground, and two additional units can be attached to a carbon fiber tube mounted at the top of the platform. This last set of cameras can adopt a number of configurations depending on the inspection mission to be performed: two front-looking cameras forming a stereo vision system, one camera looking to the front and the other looking to the ceiling, or, to save weight, a single camera looking to the front. An analog videocamera operating at 5.8GHz has also been attached to provide, at the base station, real-time information about the state of the vessel. Finally, the vehicle carries an additional processing board (CoreExpress Intel Atom-based) which avoids sending sensor data to a base station, but process it onboard avoiding communications latency inside critical control loops.
Communications result to be critical for controlling the UAV. In the current configuration, a cluster of laptops is used to perform all the offboard operations, so that information exchange between laptops is performed by wire and the wireless datalink is left only for the communications with the quadrotor. Moreover, only one laptop talks directly with the MAV in order to reduce multiple point-to-point communications for the same data, but they are republished by this laptop to provide the rest of computers of the cluster with the information. This configuration permits adding new computers to the cluster as needed, ensuring there is no extra wireless communications with the vehicle.
Sending images over a wireless network turns out to be expensive in bandwidth and time terms. For this reason, pictures are stored in the MAV during an inspection mission. In a first approach, visual odometry techniques were employed to estimate the robot motion. These algorithms required a lot of computation and needed to be performed o?board, which implied the corresponding delay produced by wireless communications. On this account, they were discarded and the laser-based solution was adopted. The wireless device attached to the vehicle is connected to the atom board using a dedicated PCI Express port, avoiding wireless communications from sharing USB bandwidth.
The control software architecture comprises at least two physically separated agents: the MAV itself and a base/ground station. More specifically, the different computational resources of the MAV run the control algorithms as detailed next (either as firmware or as software): (1) the main ARM7 controller runs the low-level software taking care of attitude stabilization and direct motor control; (2) the secondary ARM7 controller runs a position controller; and (3) the high-level processor executes, on top of the Robot Operating System running over Linux Ubuntu, ROS nodes providing platform motion estimation as well as platform safety, interaction with the onboard platform controllers and WiFi communication with the base station. Apart from this, the base station supporting the MAV also runs ROS over Linux Ubuntu. For this configuration, ROS becomes particularly relevant as it supplies the middleware functionality for transparent messages exchange between processes. Those that can tolerate latency in the communications are executed on the base station, while critical control loops run onboard the vehicle in order to ensure minimum delay. The control software has been designed following modularity and software reutilization principles. In this way, adapting the platform for different missions involving different payloads, or the selective activation of software modules, can be performed in a fast and reliable way. In this regard, ROS has also proved to be specially useful and, in fact, has guided the control software modularization.
The control architecture here described allows up to three modes of operation for the platform: manual, semi-autonomous and autonomous. While in the latter mode the system just requires a description of the mission to accomplish and it takes care of the mission execution without the intervention of any operator, in semi-autonomous operation, an operator is expected to send velocity commands in x, y and z using the sticks of the R/C transmitter, while the vehicle provides hovering and height control functionalities using onboard sensors and controllers. In manual mode, the operator has total control of the platform and can send roll, pitch, yaw and thrust commands.

Lightweight magnetic inspection robot:
A lightweight inspection robot was developed for close up visual survey. This system consists of two actuated wheels, which are augmented with neodymium magnets. On the tail section a neodymium ring is attached serving as a passive rear wheel. The system is very lightweight and weights below 1kg. The magnetic crawler is currently controlled remotely via a 2.4GHz remote control. Attached on the system is a camera, providing a wireless video transmission using 5.8 GHz band.
Another option is the attachment of an onboard camera with an integrated SD card. This video option is suitable to provide undisturbed video images in HD-Quality. The problem with the wireless video transmission using 5.8GHz band is that due to signal interference the videos are not always of a high quality. This was especially true on the ship trials, where the robot was tested within a vessels cargo hold. The image quality was always good enough to provide the ship surveyor with enough data to assess the damage or the corrosion of the ship. For an automatic image processing algorithm, the SD-card camera solution provides a better solution.
The main effort during the design, implementation and deployment phase of the crawler was put into the design of the actuated main wheels of the crawler. Several designs have been implemented and tested afterwards under real conditions as well as under laboratory conditions. The design went from a legged wheel approach, where several feet where attached to the wheel to a simple design with a round wheel shape and simple magnets attached. In the final design of the wheels a round wheel shape was selected, where each of the 60 neodymium magnets are attached in a way that they can adapt slightly to the metal surface. This is especially important if the magnetic crawler has to overcome small obstacles or must overcome convex surfaces.
The system was tested to crawl on vertical walls as well as on overhead parts of the ship. The important design question was always the ratio between magnet strength, motor power and wheel design. It was found out, that magnets working perfectly on a clean, bare metal surface were not suitable to climb on a painted, corroded ship wall with the same performance. The other direction was also true, which means that the strong magnets, which worked good on a ship wall, were too strong for a clean perfect metal wall. In this case the magnetic crawler was not able to move at all because the magnets on the actuated wheels held the system which such a strong force, that the motors where too weak to drive the system.
At the end of the project during the ship trials in Varna, a good configuration of wheels, motors and magnets was found. Still, a safety net was always needed, especially if the magnetic crawler had to climb on dirty surfaces with blistering, corrosion and paint.
The system was able to climb on a vertical wall with a speed of around 50cm per second. During the trials, the video images of the crawler where submitted to the operator, who was wearing wireless video goggles, using also the 5.8GHz band.
The position of the magnetic crawler was tracked using a 3D tracking unit. In order to be traceable, a bright power LED is attached to the back of the crawler. This bright light source can be tracked by the trackers camera, using a blob detection algorithm. The crawler system provides also an internal light source, illuminating the area in front of the crawler. This is especially needed if the robot has to work in dark and not well lit environments.
The lightweight crawler serves two purposes within the project. One purpose is the close up visual survey during the second phase of the inspection project (the first stage is the rough visual inspection using a helicopter, the third stage is the NDT phase using the heavy weight crawler). For the close up inspection, only visual information and the position where the image was taken is mandatory. This visual information is provided by the lightweight crawler using the aforementioned wireless image transmission. During the inspection process the surveyor can guide the robot operator to the spots of the vessel, where he needs a close up survey. If a history of images is needed, the data can be stored in a spatial content management system, also developed within the MINOAS project.
The second application of the lightweight crawler is the marking of defects. During a inspection task, the survey should be able to optically mark the spot, where a defect or corrosion was found and where the repair team should have a look. For this purpose, the lightweight crawler has a spray unit as an optional payload. This payload consists of an actuated pump and a tank, containing a special varnish for marking. The pump is driven by an electric motor which can be triggered by the remote control. If the surveyor wants to mark a defect, he can simply press the corresponding button on the remote and the magnetic crawler will spray the varnish on the spot beneath it. For the marking test, several tests were performed, which type of varnish, ink or paint is suitable to stick on metal. The best results were achieved using acrylic varnish with thinner.
The magnetic crawler is energy autonomous by using a 12V lithium polymer battery pack. The systems energy autonomy lasts for about 30 minutes, if the system is driven constantly and all subsystems are constantly turned on (i.e. the bright tracking LED, the LED light source of the camera, the camera and the video transmission). The magnetic crawler was designed, that a battery pack can be changed within a few minutes. Providing enough battery packs, the system can be used during a full inspection. The system is light weight and does therefore not need any safety precautions such as a safety robes.
The use of a safety robes (as well as the use of a power cord for higher energy autonomy) would significantly reduce the high maneuverability of the system. Because there is always a risk of the system falling down from the vessels wall, it is still advisable to use a small, man portable safety net. Such a safety net was used during all experiments and ship trials. Even if the system dropped down from a high greater than 10 meter, it was always possible to catch the system because of the small size and because of the light weight of below one kg.

Heavyweight magnetic inspection robot:
The problem of developing a robotic platform able to operate in harsh, hostile and dirty environments as ship holds, is to ensure the robustness of the system itself guaranteeing at the same time the operational capabilities required by the specifics. To this aim, the choice of a track-based vehicle as platform for autonomous observation and inspection operations is straightforward and sustained by similar studies and applications both in research and commercial areas.
According to the MINOAS case study introduced in the technical annex and subsequent deliverables of the project, basic functional and system requirements of a heavy climbing robot, able to carry on marking and thickness measure devices, including small electrical arms, has been specified in order to implement the hybrid climbing marker and inspection robot. Basically, the heavy climbing robot has to move inside ship hold floor, side slopes and web-frames. In particular, the need of maneuvering between these latter structures constrains the size of the vehicle.
A target footprint of the vehicle equivalent to a square of about 0.4 m of side has been established for a corresponding weight of about 50 kg. Discontinuities in the slope of adjacent planar surfaces are between 30° and 60°. The vehicle is required to be able to maneuver over planar surfaces and follow the vertical web-frame profiles, whose range and orientation have to be perceived by suitable sensors.
The vehicle mechanical design is essentially determined by the choice of the climbing system. In order to minimize the risk of detaching and crashing, while using a passive adherence system, magnetic tracks are preferred to wheels in virtue of their capability of maximizing the contact area with the metal surface. In order to allow the platform to maneuver between the vertical shell frames, carrying on a small electro-mechanical arm, the resulting length and width of the vehicle are about 49 cm and 43 cm respectively, with a height of about 18 cm excluding the boxes for data acquisition and control electronics and payload devices.
Different type of magnets and track configurations were tested during the development phase, allowing to correct the problems and improving the capabilities of the vehicle.
A couple of conical models have been initially identified on the market. The former, named N42 Nichel, weights 5.1 g, has a minimum and maximum diameter of 8 mm and 15 mm respectively and a height of 6 mm, and exerts a force of 5 Kg. The latter, named N40 Nichel, weights 15 g, has a minimum and maximum diameter of 15 mm and 20 mm respectively and a height of 8 mm, and exerts a force of 8 Kg. In order to constitute a track, the magnets are connected to a chain with the possibility of alternating rubber elements in order to reduce the adherence force and increase the friction. After a number of laboratory tests, it was evaluated that due to the reduced contact surface, given by the conical shape, the magnets tend to detach or slide over the metal surface, thus lacking of the desired adherence force and stability. Due to this problem it has been evaluated to substitute the conical shaped magnets with rectangular shaped ones, characterized by a greater surface of contact and a higher attraction force. In particular, the employed magnets are characterized by a dimension of 40x22x9 mm and a 20 Kg attraction force.
The driving element is composed by an aluminum frame where magnetic tracks, motors and traction gears are fitted.
In a first version, the driving element was equipped with four tensioner springs, two for each track, and two leaf springs, one for each track, used to tighten the tracks, consequently holding down the magnets in spite of the presence of obstacles such as cables, welding and slope changes.
Due to track-jamming and traction power dissipation problems involved by such a track/spring set-up, the configuration of the tracks and springs has been changed leading to a second version where the tensioner springs have been removed, leaving only one leaf spring per track, properly shortening each track in order to maintain the correct tightening.
Another problem occurred with the choice of permanent magnets mounted on the tracks.
Due to an incorrect implementation of the plates used to fix the magnets, they tended to jam and come off the track. Moreover, because of the short distance between each track element, it was decided to mount 8 Kg magnets alternating with 5 Kg ones.
Since the 8 Kg magnet dimensions are higher than the 5 Kg ones, the attraction of the latter ones was reduced because of absence of contacts between magnets and working surface.
To overcome the problem related to the magnets, a new release of track plate elements has been designed, requiring custom magnetic plates provided with holes for a direct mounting on the track chain. The magnetic attraction force for this new plate release is about 10 Kg per magnet. In nominal operating conditions, having the contact of 8-10 magnets for track, the corresponding attractive force is about 80-100 Kg per track (160-200 Kg in total).
A series of tests was performed in lab to evaluate the climbing capabilities of MARC. The first test-bed used for trying the vehicle in lab was an iron plate installed in a pipe scaffold and operated by means of a winch. In such a way, the slope could be modulated from 0 degrees (horizontal position - floor) to 90 degrees (vertical position wall).
Tests were performed by making MARC moving on the plate and by progressively changing the slope from 0 to 90 degrees. MARC was able to climb without problems until to a slope of about 70 degrees. For greater slopes MARC tended to capsize. The problem is due to the high position of the barycentre of the vehicle. To counteract the moment of forces which tend to make the vehicle capsize, a couple of wheels have been designed, built and mounted in the rear part of MARC behind its tracks. After this improvement MARC was able to rise until to the slope of 85.
Beyond a slope of 85 degrees, also for the reduced force exerted by the magnets, with respect to their nominal one, due to the thin sheet used in the lab, MARC tended to slide. This problem was solved by applying an anti-skid tape on the magnets.
During the tests for evaluating MARC climbing capabilities two problems arose:
1) For slopes greater than 70 degrees MARC tended to capsize. Two additional wheels in the rear part of MARC were added to solve this problem.
2) For slopes greater than 85 degrees MARC tended to slip. An anti-skid tape mounted on the magnets solved this problem.
A second series of tests was performed to evaluate the capabilities of MARC in moving through changing slopes. Experiments were repeated as abovementioned making MARC moving from an horizontal position and facing a slope (in this case during the experiments the slope was changed from 0 to 45 degrees). The two rear wheels added to avoid capsize created problems in dealing with changes in slope. Being the wheels rigidly connected to MARC, when MARC is facing a change in slope the wheels raise the rear part of MARC not allowing its tracks to adhere to the metallic surface. To solve this problem it was decided to design and build a mechanism which could keep the wheels raised when MARC is facing slope changes so that it could lowering them when MARC is moving on slopes over 85 degrees and raising them when the slope was smaller than 85 degrees. In a first phase a simple mechanism based on springs was implemented but the results weren't fully satisfactory.
To further improve the mechanism, wheels were mounted on a moving cart actuated by a DC brushless servomotor that can raise and lower the rear wheels when it is requested. Two additional switches were added near the mechanical limits of the wheeled cart. These digital inputs are necessary for avoiding failures when automatically controlling the motor position. For finding the right instant of time when raising or lowering the wheeled cart we used the measures of pitch angle provided by the inertial sensor on board. Moreover, for detecting the change in slope during the descent from a vertical wall, a laser range sensor was mounted in the rear part of the vehicle. The result was an automatically moving wheeled cart that follows the wall slope, keeping three basic positions, i.e. home, up and down, during ascent and descent of MARC.
Tests performed for evaluating the capabilities of MARC in moving through changing slopes pointed out how the two rear wheels added to avoid capsize created problems in dealing with changes in slope. For facing this problem a brand new system consisting in a wheeled cart in the rear part of the vehicle was designed and built. By adequately using the sensor measurements the wheeled cart can follows the changes in slopes and so it doesn't interfere any more with the movements of the vehicle and even helps it during ascent and descent.
A guidance and control system based on a simple Lyapunov-based design, developed for the wall following task, was implemented and tested. A first set of experiments, performed in the horizontal plane following a wall with a changing profile, demonstrated the system worked correctly following the wall profile changes with small distance and heading errors. A second experiment was performed making MARC ascending and descending a metallic wall while following a lateral wall; also in this case the guidance algorithm performed very well and the distance and heading errors were small.
For the remote guidance of the MARC vehicle, two different user interfaces were developed: a first one is a classical graphical interface runnable on every commercial computer; a second one is developed to be executed on smartphones, allowing the operator to intuitively control the vehicle by simply touching the screen on proper command icons and moving the phone, using the internal attitude sensor values as references for the vehicle motion.
MARC was designed to be able to carry on a payload for measuring and/or interacting with the walls. In particular a four degrees of freedom arm with a special tool installed at the end developed by Glafcos was installed. The tool mounted on the arm is composed by two parts, the first is a rotating grindstone used for preparing the place to be measured, the second is a commercial ultrasonic thickness sensor.
As far as the software integration is concerned, a new kernel on the MARC embedded control system containing the drivers for communicating with the arm control board was added. Moreover, an ad hoc program was developed and loaded on the MARC embedded control system that acts as a bridge allowing commands from the arm HCI to transparently reach the arm controller by means of MARC WI-FI Ethernet link.
The MARC vehicle was extensively tested during onboard trials, proving the capabilities of climbing a vertical painted metallic wall in presence of rust and pounder. The vehicle demonstrated satisfactorily performance also in operating conditions where anti-skid tape was particularly worn; the presence of rust over the tracks did not affect system performance.

Automatic NDT activities:
The survey activities described under MINOAS are analyzed to the a) visual assessment of the vessel and b) thickness measurement (UTM) of the hulls walls according to the requirements set by the Regulatory Authorities. The technologies developed and tested during the operations of MINOAS provide the underlying (carrying) platforms and the operational tools for their operation through robotic means. This activity is focused on the development of the corresponding tools that will allow the remote operation and automation of the UTM on the platforms that are responsible for the carrying out of this task.
MINOAS has examined the available commercial tools related to the NDT which are targeted (affiliated) to the marine sector and has determined that their straightforward implementation is inadequate for the specific task mainly due to the inability to mountain them on the underlying platforms, the restrictions applicable to the workspace and the requirements for the smoothness of the area under examination (structural features that prohibit their operation over large areas presence of stiffeners for example). Given these restrictions, MINOAS has developed a novel approach that uses a customizable robotic arm designed specifically for the operation over a carrying robotic platform (MARC) but with minimum dependency for the computational power and the communication with the operator. The robotic arm follows the traditional procedures as now applicable in the marine inspection field and realizes via tele-operation or automatic algorithms the three stages required for the surface preparation (surface grinding), the preparation of the UT probe (greasing) and extraction of the measurement (A-Scan and thickness measurement extraction via sophisticated data processing algorithms). The solution provided through MINOAS has undergone the design process, the mechanical manufacturing, the design and construction of the electronics and the associated software with a user-friendly environment that allows for the parametrizable and automatic interaction with the human operator. Secondary approaches have been examined that would allow for the continuous scanning of the surface (alternative of a wheel probe) or probe-bubbler topologies which facilitate the surface preparation and minimize the restrictions for the UT probe contact conditions.
The proposed schemes, the different prototypes and the technical details of the development have been discussed under deliverable D6, while the laboratory testing and the field activity have been summarized under deliverable D7.
The results show that the overall activity can be realized repetitively on the field with acceptable accuracy results and can accept a high degree of automation for the activities that have an iterative nature.
More specifically, the specific activity has developed:
1. two (2) robotic arm prototypes for the mimicking of the human activities (three stages of surface preparation and measurement extraction)
2. a dedicated end-effector that facilitates these activities and satisfies via a novel mechanical design the good contact conditions via robotic means for the grinder and the probe
3. a dedicated electronics board for the production of the ultrasound and the extraction of the thickness measurement
4. a dedicated controller for the controlling of the robotic arm and the peripherals (pump, grinder motor, etc.) and the realization of the technical features required for the operation of the overall system (forward & inverse kinematics of the arm, communication, safeguarding of proper operation of peripherals, etc.)
5. the corresponding software for the interface and the control of the system via a remote position and the interaction with the carrier platform (MARC) in a transparent way.
The alternative solutions have been extensively examined and some have sustained laboratory testing for their efficacy and the possibility to constitute either alternative solutions to the ones adopted or the possibility to substitute them in an effort to reduce the systems components and relax the initial requirements. The bubbler topology has been found to be eligible for alternative prototyping solutions, but the current prototype has been decided to be the most adequate, given its mimicking nature (closeness to) of the human (traditional) activities.
The introduction of novel tools and methodologies through MINOAS accepts much skepticism from the established actors in the field. Although the efficacy of the developed technologies has been examined through the testing and redesign activities realized throughout the project, the main actors are rather reluctant to accept new methodologies especially in the field related to safety and inspection. It has thus been decided that the novel technologies developed under MINOAS follow closely the traditional and well established methodologies in order to showcase the availability of alternative solutions and the high degree of automation and standardization achievable by advanced technological means.
The distinct components used for the integration of the final system constitute off the self-technologies which provides the redundancy properties of the system and the low-cost of production for the final product.

3D position tracker for mobile robots:
For any type of inspection it is mandatory to localize the data which was acquired during the process. This is especially true for any type of visual or NDT data. Only by accurate localization, a history of the inspection data can be set up. By allocating different measurements, which are taken over time, to a certain position inside the vessel, the data is comparable. In order to localize the acquired data within the ship, the robots have to be localized.
There exist several options to localize a robot in the environment. One option is to build a 2D or 3D map of the environment using laser range finders or optical sensors which are carried by the robot itself. The robot can then use the local scans to localize itself within the world model using for instance probabilistic methods. This localization method needs that the robot is able to carry the sensor equipment and also sufficient processing power to estimate the relative movement of the robot.
For the lightweight magnetic crawler it was not feasible to carry enough processing power or the suitable sensor packages, so other options had to be investigated during the project. Another option is to use an external tracking system which can estimate the position of a target relatively to a given reference frame. Such tracking systems exist on the market (like one system provided by the company FARO), but such systems are very expensive (around 100.000 ) and not easily portable. The existing systems provide a very high accuracy of below 1mm, but according to the MINOAS localization specification such accuracies are not needed.
For the visual inspection process, the localization of the system within a range of 20-30 cm is sufficient. Therefore a custom designed 3D Tracking unit was developed during the project. The tracking unit consists of a camera which is actuated using two servos, allowing panning and tilting of the camera. The camera is used to track a bright light source, which is carried by the magnetic crawler. The tracking unit is controlled using a laptop or a standard PC.
The tracking unit has to be connected using one firewire connector and two serial connectors using RS-485. The firewire provides the video image of the tracker. The control software, which runs on the laptop, uses the RS-485 serial connection to control the pan and tilt angle of the tracker. Several image filtering algorithms are applied to filter the bright LED of the magnetic crawler. A blob tracking algorithms is used in order to keep the bright LED spot in the middle of the cameras image plane.
In addition to the camera for blob tracking a laser range finder of the type ODSL 30 from the company Leuze is used. This laser range finder gives a single point measurement with an accuracy of less than 1cm and a maximal distance of 30 meters. This laser range finder is attached on the same mechanical link as the camera.
The control computer uses the second RS-485 serial link to read the distance from the laser range finder. The basic functional principle of the 3D tracker is that the software keeps the target (i.e. the bright LED-spot) in the middle of the camera image plane. The distance to the target (i.e. the spot on the wall where the magnetic crawler is) is measured by the ODSL 30. Given the two angles of the pan-tilt unit and the distance, the position in space of the target in the trackers reference frame can be calculated.
The tracking software uses ROS (robotic operation system) as a communication layer, so the tracking data can be fed directly into the MINOAS communication architecture.
The tracking unit was used during several experiments in laboratory environment as well as during the two ship trials, which took place in Varna, Bulgaria.
The tracking unit is mounted on a standard camera tripod and is easily man portable. During the tests, the localization accuracy was manually measured. This could be easily achieved because the laser range finder provides a visible spot where it hits the target (i.e. the crawler) or the wall the crawler is climbing on. Based on the measured distance the offset between the light source on the crawler and the visible laser spot was measured.
The localization error during the test phase was always below 25 cm up to a distance of 15 meter between the tracker and the crawler. Using the developed 3D tracking unit, the position of the magnetic crawler could be tracked in real-time.
The tracking unit can be used not only for the magnetic crawler of the project, but it is also feasible to track any other system or sensor, provided that the system carries a bright LED.
The only problems arose with the flying UAV (the helicopter), which was not traceable by the 3D tracker. The problem was that the UAV provided only a small frame on where the laser could actually hit the UAV and provide the needed distance measurement. Most of the time, the laser was therefore measuring the wall behind the helicopter and not the helicopter itself. This was not a problem for the project, because the UAV was able to carry a 2D laser range finder and provided enough processing power for a model based self-localization.
The data of the current position of the magnetic crawler is transmitted together with a system time stamp, making it easy to allocate the position of the crawler with the measurement taken by the robot. The data provided is directly used by the spatial content management system which is described in the next section.
The position of the robot, which is measured by the 3D tracking device is always within the reference frame of the tracker, not automatically in the vessels reference frame. In order to provide allocated position data, it has to be assured that the tracker is on a defined position within the vessel, e.g. the cargo hold of a ship. During the experiments in Varna, the tracking unit, which is attached on a camera tripod, was always positioned on spots which could be easily identified. Such spots are for instance welding seams on the bottom of the cargo hold or other visible points, which could be found again. In order to align the tracker with a global position of the ship, the points where the tracker is located has to be manually aligned with the ship reference frame. In general it is not mandatory to know the exact position of each measurement point. For relocation of the tracker and therefore the position of the tracked robot, it is sufficient to assure that the tracker is located at the same spot in order to make measurements and image recording comparable over time.

Software for detection of defects in metallic surfaces:
The defect detection software has been developed as another tool at the service of the surveyor while making repair/no repair decisions. This software can receive images taken by the platform and processes them looking for evidences of corrosion and cracks almost in real-time. The details can be found next.
Corrosion detection:
The corrosion detector has been built around a supervised classification scheme implemented as two stages running a weak classifier each, following a cascading approach by which fast classifiers with poor performance alone lead to a global classifier of better performance. This algorithm will be referred to as WCCD (Weak-classifier Colour-based Corrosion Detector) from now on.
The first stage of the classifier is based on the premise that a corroded area exhibits a rough texture, where roughness is measured as the energy of the symmetric gray-level co-occurrence matrix (GLCM), calculated for downsampled intensity values between 0 and 31, for a given direction a and distance d. Patches with an energy lower than a given threshold Te, i.e. exhibit a rough texture, are candidates to be more deeply inspected.
Unlike the first stage, the second stage makes use of the colour information that can be observed from corroded areas. More precisely, the classifier works over the Hue-Saturation-Value (HSV) space after the realization that pixels from corroded areas are confined in a bounded subspace of the HS plane. Although the V component has been observed neither significant nor necessary to describe the colour of corrosion, it is used to prevent the well-known instabilities when computing hue and saturation values for colours close to white or black. In those cases, the pixel is classified as non-corroded. In order to learn the HS values for image pixels known to correspond to corroded surfaces, a two-dimensional histogram is built in a previous training step.
In order to generalize the HS histogram to cases out of the training set, different standard techniques have been considered, such as downsampling and/or Parzen windows for different two-dimensional kernels. Although considerable improvements can be observed for those methods, best results have been obtained following a smoothing approach by means of a bilateral filter, which combines two Gaussians, one that operates at the spatial domain and the other at the intensity domain.
Corrosion-guided crack detection:
A crack detector guided by the output of WCCD has been implemented after the observation that most cracks in metallic surfaces coincide, at least partly, with corroded areas. This algorithm will be referred to as GPCD (Guided Percolation-based Crack Detector) from now on.
The crack detection algorithm is based on a percolation model which takes into account the crack geometry within a region-growing scheme. The region-growing procedure starts from an edge pixel suspected from being affected by corrosion. Additionally, it is required to be darker than Gs and must not belong to an already detected crack. The propagation proceeds over the dark neighboring pixels until reaching an NxN boundary. Then, the elongation of the percolated area is checked to be larger than En. If that is the case, the percolation process continues until reaching an MxM boundary. The final percolated area is classified as a crack if: (1) its average gray level is darker than a threshold Ga, and (2) its elongation is larger than Em. The elongation is computed by means of the normalized second central moments of the region.
Performance assessment:
The performance of WCCD depends on the performance of its different stages. Regarding the roughness stage, several experiments have been performed considering different values for d and a when computing the GLCM and, consequently, its energy level. The energy threshold Te affects the algorithm performance in terms of computation time as well as reducing the number of false positives, since all patches with a high energy level are discarded and only those with a low value become input for the colour checking step.
For a test set comprising a total of 7384 patches, global WCCD performance has been measured as 9.80% for the false positive percentage and 5.86%.
Regarding GPCD, its performance was assessed after a proper configuration of its different parameters. The parameters related with the expected elongation of cracks, En and Em, and the gray level thresholds Gs and Ga, were all tuned so as to reduce as much as possible the number of false positives over the test set, while the values for N and M, related with the size of the percolation boundaries, were determined using the mean value of Pratt's FOM measure calculated for all the test images. The global performance was measured as 0.72% and 0.57% for, respectively, the false positive and false negative percentages.
To finish, execution times for WCCD ranged between 7 and 15 ms for images comprising from 120.000 to 172.800 pixels, while GPCD took between 30 and 150 ms for images of similar size. Tests were performed on an Intel Core2 Duo @2.2 GHz processor with 4 GB of RAM.

3D user interface and spatial content management system for marine inspection:

In order to keep track of the acquired inspection data, which has been acquired by the MINOAS robots, the inspection data has to be stored in database. This will allow keeping a history of inspection data such as images or thickness measurements.
To make the data searchable and comparable to each other, it has to be spatially aligned with a model of a vessel. This means that the stored information has to be spatially allocated in a global or local reference frame.
A ship survey might be interested in all the information gathered at a certain part of the ship. This is achieved with a spatial content management system, which also provides interactive 3D view of the data. The spatial content management system (SCM) was developed during the MINOAS project. The stored data can be virtually everything, which can be stored as a file. This ranges from images, videos, thickness measurements, or documents such as pdf files.
The data is stored on a local file system in a file structure based on the following hierarchical structure: vessel name, robot name, recording date and recording time. This file structure can be exported to any server for backup.
In order to access the data, based on the spatial information, where it has been recorded, a XML based structure is generated. This assigns a position key and a time-stamp key to each individual data. Based on this key (spatial location and time-stamp) the data can be accessed on the local file system.
The interface provides two distinct views for the stored data. One tree view is used to visualize all the data recorded depending on the file structure. The operator can access all data based on the recording time and date. This is a similar representation as in a local file browsing system. By selecting the data on the tree structure, the data can be opened directly in the SCM. This allows running the videos, showing the images taken or opening a viewer for the document.
Beside the tree like structure for searching the data, a 3D view exists, which is based on the Open Scene Graph Library. This allows the visualization of the whole vessel, provided that a 3D model exists of the ship. During the project time, a generic bulk carrier model was used for the data visualization.
Additionally, a 3D Model of all inspection robots was generated. This includes the 3D models of the magnetic lightweight crawler, the model of the MARC heavy weight crawler as well as a model of the UAV quad-copter.
During the recording of the data, the position of the inspection robot is visualized on-line. The SCM uses the ROS framework to get access to the position data. During the ship trials in Varna, the data was provided by the 3D racking unit, which tracked the position of the magnetic crawler. Besides the position and the time stamp of the position data, also the acquired data is send via the so called ROS topics. ROS is based on the publish/subscribe paradigm which allows the subscription to all data channels within the ROS framework.
In the case of the magnetic crawler, the position data is provided with a time stamp via the 3D tracking unit. The crawler itself sends the video stream (or single camera images) to the control station, where the SCM is running. The SCM combines the camera images with the position data, coming from the 3D tracking unit and generates a data set. The visual image is stored on the local file system and the metadata (position, time-stamp, file location) is generated by the SCM.
During the inspection, the position of where the data was acquired is visualized as a sphere within the 3D view. This allows an on-line viewing of the inspection process. The operator can select the data sphere within the 3D interface and can so access all the inspection data which is associated with the position.
The user interface allows the saving of each individual run. The runs can be reloaded later and can provide data for offline analysis of the inspection.
The inspection data can be accessed in two ways. The operator can select the data within the tree view. On the 3D interface, the spot is highlighted, where the data was acquired in space with respect to the vessels model.
On the other hand, the user can select the data spheres within the 3D view and all the data associated with the position is visualized in the tree view.

Task allocator:
One of the goals of the MINOAS project is to optimize the usage of a heterogeneous robotic team, to perform inspection mission of ships' areas, like hull, ballast rooms, etc.
The main objectives of such application are:
the improvement of the safety level for human operators, that can rely on robotic platforms to inspect dangerous and/or hardly reachable areas;
the development of suitable procedures for methodological and repeatable inspections;
inspection resources optimization through formal methods.
The goal achievement mainly requires the definition of a procedure to classify the ship's areas, starting from the architectural drawings, and thus creating a table of the surfaces to be inspected, detailed by the characteristics of each area of interests: internal/external, underwater/in-air, ground/wall, etc.
A complementary classification has to be done for the vehicle classes applied for the inspection: underwater vehicles, ground robot, climbing platforms, etc.
Depending on the capabilities (speed, type of inspection, operative constraints, etc.) of each class of vehicles and the number of available robots for each class, a procedure to optimally associate and share the robotic platforms among the different zones to be inspected has to be developed. This topic, referred as task allocation problem, has the aim of associating each vehicle, in space and time, to every interested area to be inspected.
The result is a complete association and scheduling of all the inspection resources to all the interested areas; in this context, the human operator can be also scheduled, for instance in those cases where no robotic platforms have the compliant characteristics to perform the inspection. Thus, the user can decide to perform a human inspection, or discard the inspection if it represents a too risky operation.
Relying on the analysis and study of the different existing technique for task allocation, an ad-hoc task allocation algorithm is proposed for the application to the MINOAS project.
On the basis of the vehicle and area classifications, a task allocation algorithm, performing an optimal mapping search can be developed in order to assign each vehicle to the area that most fit with its characteristics and capabilities. The assignment of more vehicles to a specific area requires the further vehicle allocation in space and/or time; for instance, two observing vehicles can inspect a floor at the same time, observing half the floor each one. But one observing vehicle and one marking vehicle may be scheduled in time, i.e. the observing robot will start the task looking for damages, and then, delayed in time, the marking robot will mark the damaged zones indicated by the other vehicle.
The algorithm for the task allocation requires the following set of information:
ACTUAL_NODE: the current area, described by the tree node, to be allocated to the robotic resources;
CHILDREN_NODE_NUMBER: the number of children of the actual node;
CHILDREN: the list of children nodes;
LEAF_NODE: a flag variable specifying if the current node is a leaf of the tree or not;
TOTAL_AREA_COUNT (TAC): a recursive count of the sub-areas linked to the actual node;
AVAILABLE_ROBOT_NUMBER (ARN): the number of available robots that can assigned for area inspections;
ASSIGNED_ROBOT_NUMBER: the number of robot assigned to the actual area.
The first phase of the task allocation procedure requires the progressive count of all the basic structure composing the target of the inspection, in order to compute the TOTAL_AREA_COUNT values; in other words, for each parent node of the tree, the count stored in each child node has to be summed and then stored, starting from the lowest level of the tree, where leaf nodes have count equal to 1 by definition.
The next step is to allocate the available robots, thus while AVAILABLE_ROBOT_NUMBER is greater than zero, the following operations are executed:
the CHILD node, not allocated yet, with the minimum TOTAL_AREA_COUNT is selected;
the number of robots to be assigned is computed
if (ARN >= robots) the number of assigned robot for the child node is equal to robots, else it is equal to ARN.
When no more robotic platforms are available, the remaining not assigned areas are allocated to the robots using the following procedure: each remaining area is assigned to the robot with the current minimum number of areas to inspect.
Following these guidelines, a balanced allocation of the areas to the robotic platform is carried out.

Path planning algorithms:
The path-planning problem is decoupled in two layers handling the discrete motion of the agents through the atomic operating areas, considered as a set of contiguous cells, and the continuous motion of the agents inside a specific area respectively.
When a direct interaction with the environment is required through the manipulator, e.g. for executing thickness measurements, a third layer to plan the arm motion in order to position the end-effector in the desired place is considered.
The result is a hierarchical planning scheme handling the motion of the robotic vehicle through and inside the operating areas and the final high precision approach of the end-effector to the target:
Structure-based path-planning: the so-called structure-based path-planning, given an agent and its motion capabilities, has to solve the problem of finding a suitable sequence of connected operating areas between the start and the goal position, where, as already stated in this report the problem assumes the form of path-finding in a graph with additional constraints given by the simultaneous presence of multiple agents.
Continuous/local path-planning: the task of the local continuous path-planning is of computing a path, compatible with the manoeuvrability constraints of a given agent, connecting a couple of points inside a specific operating area. Additional constraints on the agent orientation at the beginning and at the end of the manoeuvre can be given.
Manipulator path-planning: the task of the manipulator path-planning is to compute a path, compatible with the joint configuration of the robotic arm, able to drive the end-effector in the desired position and orientation. Since the precision in the arm motion control is higher than in the case of the vehicle, e.g. in the case of its yaw orientation in the working position, the manipulator path-planning has to compensate for these uncertainties.
Structure-based path-planning:
The problem of structure-based path-planning reduces to find a suitable sequence of connected operating areas between the start and the goal position inside a graph. Indeed, the ship map assumes the form of a hierarchical tree, where the operating areas constitute the leaves. Leaves are then connected, according to their topology and robot motion capabilities, constituting a graph. Once defined, also heuristically, the cost of the transitions between each pair of connected leaves (e.g. their distance), it is possible to find (quasi-)optimal solutions to the problem of traversing the graph from a start to a goal node.
The proposed algorithm consists of three stages, the first offline and the others online.
Stage 1 - Construction of node connections and checking on the map structure:
1.1 Given the connections between the tree leaves, the connections between ancestor nodes are constructed according to the following rules:
if a leaf p2 of layer n is connected to a leaf p1 of layer m, with n > m, then all the ancestors of p2 of layer m are connected to p1;
given two nodes, n1 and n2 of the same layer, if there are two connected leaves p1 belongs to child_of (n1) and p2 belongs to child_of (n2), then n1 and n2 are connected.
1.2 For all the nodes in the tree, verify if their children constitute a connected graph, otherwise signal the consistency failure to the human operator.
Stage 2 - Construction of the hierarchical path:
Given a start and a goal leaf, a sequence of ancestors connecting them through the map tree structure is found: in practice, the tree structure is backed up from the start and goal leaves until a common ancestor or a couple of connected ancestors at the same layer (or an ancestor connected to the other leaf) are found. In this way a couple of node sequences are found, originating from the start and
goal leaf respectively. The hierarchical path is thus obtained by joining them after having reversed the goal sequence.
Stage 3 - Expansion of the hierarchical path:
Once computed, the hierarchical path has to be expanded in order to determine a suitable sequence of operating areas, i.e. leaves, connecting the start and goal ones. This is done, expanding the hierarchical path from its lower layer until there are nodes. Thus, considering a generic layer i, three steps are performed.
3.1 Nodes at layer i are expanded at layer i+1. Considered that inside the sequence representing the (expanded) hierarchical path each node has a predecessor and a successor, expanding a generic node n means to substitute it with a couple of set of its children determined as it follows. For each node n at layer i, for each connection between node n with node m, i.e. for each pair of nodes in the hierarchical path sequence, substitute node n with the set of nodes nC such that nC is child_of (n) AND nC is connected_to(m), when node m is a leaf or is at layer i + 1; or nC is connected_to(mC) where mC is child_of (m). The result is an expansion of the hierarchical path at level i+1 where the nodes with the same ancestor are not, in general, directly connected.
3.2 Determine a predicted optimal path at layer i + 1. Given the unconnected expansion of the hierarchical path at level i+1, dummy connections are established between the nodes originated by the same ancestor on the predecessor and successor sides. The result is a connected graph at layer i + 1 that can be searched with a conventional algorithm to find an optimal path between the start and goal leaves.
3.3 Find optimal paths connecting nodes with the same ancestor. The path computed at stage 2 can contain consecutive nodes, originated by the same ancestor, which are not directly connected. At this stage optimal paths connecting these pair of nodes are computed applying conventional path-planning techniques to the reduce sets of nodes with common ancestor.
Continuous/local path-planning:
Once that the ordered sequence of basic structures to be inspected, obtained by the task allocation, complemented by the list of structures that have to be traversed to reach non-adjacent inspection areas, obtained by the previous path-planning phase, then the continuous/local path-planning operation can take place. This phase basically produces the paths that the robots will have to follow in order to visit all the structures defined by the inspection plan, with the constraint of exploring all the points of interest defined for each single structure.
The goal of the continuous/local path-planning phase is to produce a feasible motion path for each class of inspection vehicle, taking into account obstacles and constraints of the interested structures and, at the same time, minimizing the path length to visit all the points of interest of each area.
For each basic structure, a set of interesting inspection points is pre-defined; as emerged by interviews with expert inspectors, plate structures as floor and sloping parts have usually five points of interest. For the web-frame structures, the number of points depends on the length of the structure itself.
The path-planning is composed by two stages: in the first one, the parts of path to connect adjacent structures are computed, while the second stage regards the construction of the part of path to visit all the points of interest, within the structure, minimizing the length of the path. During stage 1, for each pair of subsequent structures, a research among the way-point sets of each structure determines the pair of way-points, one for each of the two structures, characterized by the shortest distance. These two way-points are connected to become part of the overall path. In the case of passage through bridging structures, the transfer path can be computed automatically keeping into account the geometry of the structures involved, or adding a set of bridging points, similar to inspection waypoints but used only for robot transfer. During stage 2, a minimization algorithm is applied to compute the optimal path to visit all the way-points of each structure. The parts of path computed during stages 1 & 2 are then merged together to obtain the final path that allows the inspection of the overall structure subject to the task allocation plan.
Manipulator path-planning:
Due to the specific tasks required to the manipulator, that is assumed to work in a structured environment and suitably positioned by a manoeuvring carrier vehicle, the synergy of two (complementary) approaches have been considered for its path-planning:
(1) definition and planning of default paths also useful to support tele-operation, or human supervised operations;
(2) handling of the manipulator inverse kinematics.
In particular, the MINOAS arm has to be able to compensate possible uncertainties in the positioning of the carrier robot due to the harsh environment conditions as well as manoeuvring capabilities and sensor performances. In this context, the path-planner has been designed in order to allow human intervention focusing on the possibility of introducing local corrections to nominal conditions.
According to the general objective of the MINOAS project of developing technology that can be relatively easily introduced in field operations, and considering the operational constraints given by the presence of the structural components of the ship in the operating area, that can be seen as obstacles to be avoided, a two stage path-planning strategy is proposed:
(1) definition of default way-points for routine operations;
(2) computation of paths between default way-points through kinematic inversion.
In addition, as seen in the following, the arm prismatic joint, as well as the introduction of local waypoints, would allow the system to comply with uncertainties related to manoeuvring, sensing and environment modelling.

Networking architecture:
A networking architecture enabling a reliable communication among different mobile agents in harsh scenarios has been designed. This architecture applies where several constraints and limits exist, such as electro-magnetic interferences, obstacles, refractions.
The devised networking architecture integrates different existing wireless technologies like ZigBee and extends through the standard OSI layers comprising prescriptions for the physical positioning and layout of the network, as well as higher level networking protocols. The networking architecture can be proposed in different markets where reliable networking in difficult environment is foreseen, like for instance defense, border protection, Oil and Gas.

Potential Impact:
MINOAS had originally described potential areas that are expected to be affected by the results and the commercial usage of the overall system to involve several socio-economic factors in the wider marine sector, for the European and the international market.
The primary objectives were the raising of the competitiveness of the European products and the decreasing of life cycle costs of the transport industry. MINOAS has developed a novel system that introduces new technological tools and standardizes the required methodologies for the vessel inspection, affecting significant financial costs related to the inspection / survey process, cost of vessel downtimes and man-hour costs due to the survey duration and the time needed for the vessel preparation. The system provides also additional technological features to the existing procedures such as extensive recording of the acquired data, decision support tools for image feature extraction and automation properties to the use of the tools deployed.

The activities described in the Technical Annex have covered the areas of design, development and integration of the technical components of the system and have also included dissemination activities in order to transfer the MINOAS concept to the general public and raise the awareness of the implicated actors in the market. A Business Analysis has been carried out to showcase the potential market opportunities emerging from the transferring of this system concept to the market. The activities included in the previous stages have exposed the MINOAS concept and the developed technologies to an extensive public review in terms of implementation possibilities and technological consequences. These may in turn be analysed to short term and long term effects on the corresponding audience.
The exposure of MINOAS through the dissemination activities has included the presentation of the results in exhibitions, scientific conferences, workshops and an European broadcasted show. The feedback acquired through these activities has been a general acceptance and show of public interest for both the original idea (MINOAS concept) and the research results reached during the project. The implementation of off-the-self components in a robotic system and its introduction to a new (and rather demanding) application field has acquired the public interest as MINOAS has proposed a new application field to the research community and has demonstrated the possibilities, the difficulties and the prospects of introducing new technologies in the ship inspection activities.

Demonstration activities have concentrated the public interest raising the question of the time-window needed for the implementation of the system as a final product. The latter has been extensively discussed within the Consortium and particularly with the classification societies (main actors of the inspection process).
The introduction of new methodologies in the sensitive field of compliance with safety regulation is addressed with extreme caution by classification societies. Although the potential beneficial aspects are recognized, the efficacy of the implemented tools has to be extensively proved and accepted by all the involved actors before robots can be introduced in class surveys, hence expanding the time horizon for the application of the concept as described in the Technical Annex. However, classification societies are not the only stakeholder in ensuring the good condition of a vessel and other scenarios of employment of the MINOAS concept can be identified in addition to class surveys. For this reason, different implementation/operative scenarios, with different time horizons, have been analysed, including also cases where hull inspection is not performed in conjunction with class surveys. These scenarios represent both further new opportunities of MINOAS employment and a set of steps toward the implementation of the concept as outlined in the Technical Annex, with a related time perspective.
Analysis of the results produced by the field trials and the laboratory testing has provided the proof-of-concept for the proposed system, yet the overall system or separate system components require some additional development stages in order to be acceptable as finalized market products. The efficacy of the proposed scheme has been benchmarked against specific implementation scenarios (use of visual information from the platforms only in a non-Class survey use, use of UTM system components only and overall system evaluation). It has been determined that the introduction of the MINOAS system to the market requires two intermediate steps: a) the development of the technological platforms up to the industrial standards that guarantee their proper (certified) operation under varying environmental conditions (degree of ruggedization, mean time between failure of components according to use, etc.) and b) acceptance of the produced technologies by the implicated actors and more specifically the potential end-users ship owners / operators and classification societies.
The Business Analysis carried out from the outcomes of WP5 & WP6 has identified as a market opportunity the introduction of the MINOAS methodology through its extensive use in Condition surveys, often requested by Ship operators in order to organize maintenance and repair schedules. This requirement the need to change the attitude / persuade the main actors of the beneficial use of new technologies is the effort committed to cancelling the inertia when creating new niche markets. This new market inevitably creates new opportunities for the initial actors involved in this endeavour highly competitive SMEs operating in a new market, highly competitive Classification Societies offering advanced services at reduced times and new revenue streams for scientific / research entities that are enabled to promote their research by introducing and developing their products in a real and rather demanding growing market. The Business Analysis reveals that a Service Provider as a SME can introduce MINOAS in the existing market by providing highly competitive price and features (through the first implementation scenario) by incorporating the MINOAS activities in the existing company operational scheme. Through this market penetration scheme, the MINOAS system concept can be introduced to the market as a novel system with a highly competitive value proposition triggering the beneficial effects for the corresponding entities: the platform developers will obtain the opportunity to benefit by creating spin-off companies, the Classification Societies will have the advantage of offering a new product with advanced safety and time efficient features, ship operators will benefit from the reduced overall inspection costs and the optimum scheduling of the maintenance activities thus resulting in increased safety for the vessels operating under MINOAS, with less downtimes and reduced life-cycle costs (stemming from the fact that components in need of repair are timely identified limiting the risk and the fact that during scheduled and mandatory inspection, the associated costs are significantly less than the ones required via the traditional means).
The activities carried out during MINOAS have concentrated towards the design and realization of the overall system under the restrictions and specifications of the existing market in order to showcase the beneficial aspects of the original concept outlined in the TA. The operations undertaken during the project have demonstrated a high public acceptance and anticipation for the introduction of the system in the market as the beneficial features described in the previous have been clearly identified and conveyed through the real-life demonstrations. The research novelties achieved during the project have been documented and disseminated in international conferences and journals, as they have promoted specific research areas (mainly in robotics & robotic vision) with new ideas, although the opportunity of developing solution for a new application field has attracted much interest and has obtained its own value. In conjunction with the Business Analysis, the feedback received from the demonstration activities has shown not only the efficiency of the proof-of-concept of the system, but also the market entry strategy that can enable the adaptation of the MINOAS concept by the end-users. It is proven in this way that the initial objectives set at the first steps of the project can be reached and the potential impact areas described by these objectives will be this way influenced. The potential impact areas that have been outlined in the TA are omitted here for brevity.

Dissemination was a significant part of the activities planned for MINOAS Project, aiming to bring MINOAS results closer to the market and effectively disseminate the outcomes of the research work and build an overall strategy for the exploitation of results.

Results of the project were disseminated through:
MINOAS web-site
MinoasProject Youtube channel
MinoasProject Twitter account
Preparation & distribution of dissemination material (Project newsletters)
Participation to international scientific conferences
Participation to exhibitions
TV coverage
Radio coverage

The Technical Annex foresaw the organisation of 3 walkthrough sessions, with each event organised in a different European country and at least one in a new member state.
It was deemed by the Consortium that due to the need of reaching as a large maritime audience as possible, events such as Posidonia, SMM (Shipbuilding, machinery & marine technology) and Nor-Shipping were preferable to a walkthrough session organized by the Consortium as a stand-alone event. Therefore the choice of the locations was strongly constrained and it was found difficult to organise an event in a new member state, even though Dolphin was willing to host such a session in Bulgaria with Lloyds Registers support.
The advantage of using existing large scale maritime events instead of a stand-alone sessions are that a large audience can be reached without having to entice these to visit an event only on the grounds of the project. Furthermore the project deemed the quantity of people interested important for feedback collection on the proposed system.
For this reason, the final Project Conference was held on Posidonia 2012 after agreement with the Project Officer.

The main dissemination events are reported in the following:
Posidonia 2010, 7-11 June 2010
SMM Hamburg, September 2010
Invited session at the MED 2011 (19th Mediterranean Conference on Control and Automation) Conference, 20-23 June 2011
Innovation Convention 2011, 5-6 December 2011
Posidonia 2012, 4-8 June 2012 (The project was present both with a stand and organizing a conference)

As already stated above, MINOAS project has also activated a Youtube channel since April 2012, in order to share videos of the platforms developed in the project and the results achieved:
http://www.youtube.com/user/MINOASProject?ob=0&feature=results_main

Some unforeseen, but very welcome interest in MINOAS project also occurred during the project. This led to dissemination in a much wider form than the Consortium could have hoped at the beginning. These dissemination occasions are:
Euronews: In May 2012 an Euronews troupe was hosted at the Dolphin shipyard in Bulgaria and attended the second set of ship trials, filming the platforms in action and interviewing the project coordinator and the technical coordinator. The report was aired the 8th June 2012 within the program Innovation and provided on-line at Euronews.
ZDF, Zweites Deutsches Fernshen, German Television: The German television visited DFKI and produced a program on their work
The Gadget Show: UK Channel 5s THE GADGET SHOW interviewed for one of its features on robots DFKI and displayed its crawler. Unfortunately, the show is only to be watched online in the UK
TV IB3: 25 December 2011 - Short interview with A. Ortiz (approx. 2 minutes) on TV local channel (IB3) about project MINOAS.
Catalunya Radio: 7 May 2012 - Interview with A. Ortiz on national radio station "Catalunya Radio" about project MINOAS

Finally, following are listed other public presence of the project:
An internet article was posted about the MINOAS crawler: http://www.golem.de/1011/79626.html accessed 24/06/2011
The European Commission referenced the MINOAS project in their report: Staying ahead of the wave - Towards greener, safer, and more competitive waterborne transportation
Issue 165 (June 2010) of Solutions a Fairplay publication referenced the MINOAS project
Project fiche produced by the EC on MINOAS

List of Websites:
The address of the project website is: www.minoasproject.eu

MINOASproject YouTube channel: http://www.youtube.com/user/minoasproject?feature=results_main

Project Coordinator:
Alessia Vergine
Rina Services SpA
Via Corsica, 12 - 16128 Genova -
ITALY
Phone: +39 010 5385320
Mobile: +39 347 2697838
e-mail :alessia.vergine@rina.org
http://www.rina.org

Technical Manager:
Leonidas A. Drikos
GlafcoS Marine Ltd.
45, Fragiadon Str. 185 37 Piraeus - GREECE
Tel: +30 210 45.23.812-3
Mob. Tel: +30 6944 22.12.39
e-mail: info@glafcos-marine.com
http://www.glafcos-marine.com

final1-minoas-final-report-photo.pdf

Final Report Summary - MINOAS (Marine INspection rObotic Assistant System)

Related documents

Share this page

Download