Periodic Reporting for period 3 - VISION (Validation of Integrated Safety-enhanced Intelligent flight cONtrol)
Reporting period: 2019-03-01 to 2019-08-31
VISION is a Europe-Japan collaborative research project, intending to develop and to validate smarter technologies for aircraft Guidance, Navigation and Control (GN&C) by integrating onboard vision system and advanced fault detection and resilient methods. The project aims at contributing to the global civil aviation goal of accident rate reduction. Critical flight situations are targeted, especially during the final approach and landing phases where nearly half of fatal accidents in the last decade have occurred. To detect and overcome the flight anomalies, recent European and Japanese projects have evaluated advanced GN&C solutions, but their transfer to industry is slowed down by lack of flight validations. VISION aims at capitalizing on the know-how and experience independently acquired by both sides to mature the TRL of such techniques.
VISION is tackling 2 different types of fault scenarios: i) flight control performance recovery from actuator or sensor failures (e.g. actuator loss of efficiency), and ii) navigation and guidance performance recovery from sensor failure (e.g. lack of GPS or ILS) or flight path obstruction. In i), Fault Detection and Diagnosis (FDD) as well as Fault Tolerant Control (FTC) techniques are implemented. In ii), onboard vision is used to improve the integrity of the classical navigation sensors, and also to augment the situational awareness for obstacle clearance. The developed GN&C solutions are validated on real aircraft platforms including JAXA MuPAL-alpha in Japan and USOL K50 in Europe. VISION proposes flight-evaluated advanced GN&C solutions with increased TRL, which is beneficial for aircraft industries. Throughout these collaborative research and flight testing activities, VISION project also promotes research and people exchanges between Europe/Japan universities and research institutes.
VISION is tackling 2 different types of fault scenarios: i) flight control performance recovery from actuator or sensor failures (e.g. actuator loss of efficiency), and ii) navigation and guidance performance recovery from sensor failure (e.g. lack of GPS or ILS) or flight path obstruction. In i), Fault Detection and Diagnosis (FDD) as well as Fault Tolerant Control (FTC) techniques are implemented. In ii), onboard vision is used to improve the integrity of the classical navigation sensors, and also to augment the situational awareness for obstacle clearance. The developed GN&C solutions are validated on real aircraft platforms including JAXA MuPAL-alpha in Japan and USOL K50 in Europe. VISION proposes flight-evaluated advanced GN&C solutions with increased TRL, which is beneficial for aircraft industries. Throughout these collaborative research and flight testing activities, VISION project also promotes research and people exchanges between Europe/Japan universities and research institutes.
WP2: Fault scenario definition
Considering the operational relevance (checked by Dassault Aviation) and the flight experiment constraints, fault scenarios were defined for each of the flight control performance recovery (WP3) and the navigation and guidance performance recovery (WP4).
WP3: Flight control performance recovery
1) 3 EU partners (ONERA, Universities of Exeter and Bristol) and 2 Japan partners (University of Tokyo and JAXA) have built FDD/FTC approaches on the works initiated in previous projects: indirect and direct adaptive controls, sliding mode control, and structured H-infinity control.
2) The flight test campaigns have been conducted at JAXA’s facility in Japan. Each partner had several test sessions where they achieved code implementation, Hardware-In-the-Loop simulation, and flight experiments of their algorithms on MuPAL-alpha aircraft. These tests were extremely successful. To our knowledge, it was the first time that such advanced FDD/FTC control techniques have been flight-tested on a full-scale piloted aircraft.
3) RICOH and JAXA have proposed an innovative vision-based control surface monitoring system for assisting pilot or FDD function by providing additional information on control surface deflection angles. They have conducted a flight test using MuPAL-alpha to collect image data of its aileron surfaces, and RICOH tested their aileron angle estimation algorithm on the real image sequence.
4) University of Bristol hosted a student from University of Tokyo and a researcher from JAXA, each for a month to work jointly on a FTC controller designs. This contributed to reinforce scientific collaborations between Europe and Japan.
WP4: Navigation and guidance performance recovery
1) USOL has manufactured an unmanned aerial vehicle, called K50-Advanced, dedicated to VISION. Its high payload capacity (100 litters, 20kg) allows it to carry different onboard sensors and systems. The airframe was delivered in October 2016 to ONERA, who has instrumented and ‘dronized’ it with autonomous flight capability. K50 flight authorization was issued by French civil aviation authority in January 2018.
2) Each of SZTAKI and RICOH has developed a prototype of their monocular- and stereo-vision systems to detect and track runway features for aircraft final approach navigation purpose. Both prototypes have been integrated on K50, and flight-tested for image acquisition and real-time image processing. RICOH’s prototype is the world’s first long-range stereo camera.
3) ONERA and SZTAKI have designed vision-integrated navigation filters which fuse the vision system outputs with other navigation sensor measurements (GPS, ILS, IMU, etc.).
The navigation filters are augmented with integrity monitoring function for fault detection and protection level computation. Their navigation performance was evaluated through closed-loop and open-loop simulations with emulated and real sensor data respectively, against sensor faults modelled by ENRI.
4) University of Tokyo and RICOH proposed vision-based obstacle avoidance algorithm by online trajectory planning under the detection uncertainty. They performed numerical simulation evaluation for trajectory optimization, and flight experiments for stereo vision-based flying object detection.
Considering the operational relevance (checked by Dassault Aviation) and the flight experiment constraints, fault scenarios were defined for each of the flight control performance recovery (WP3) and the navigation and guidance performance recovery (WP4).
WP3: Flight control performance recovery
1) 3 EU partners (ONERA, Universities of Exeter and Bristol) and 2 Japan partners (University of Tokyo and JAXA) have built FDD/FTC approaches on the works initiated in previous projects: indirect and direct adaptive controls, sliding mode control, and structured H-infinity control.
2) The flight test campaigns have been conducted at JAXA’s facility in Japan. Each partner had several test sessions where they achieved code implementation, Hardware-In-the-Loop simulation, and flight experiments of their algorithms on MuPAL-alpha aircraft. These tests were extremely successful. To our knowledge, it was the first time that such advanced FDD/FTC control techniques have been flight-tested on a full-scale piloted aircraft.
3) RICOH and JAXA have proposed an innovative vision-based control surface monitoring system for assisting pilot or FDD function by providing additional information on control surface deflection angles. They have conducted a flight test using MuPAL-alpha to collect image data of its aileron surfaces, and RICOH tested their aileron angle estimation algorithm on the real image sequence.
4) University of Bristol hosted a student from University of Tokyo and a researcher from JAXA, each for a month to work jointly on a FTC controller designs. This contributed to reinforce scientific collaborations between Europe and Japan.
WP4: Navigation and guidance performance recovery
1) USOL has manufactured an unmanned aerial vehicle, called K50-Advanced, dedicated to VISION. Its high payload capacity (100 litters, 20kg) allows it to carry different onboard sensors and systems. The airframe was delivered in October 2016 to ONERA, who has instrumented and ‘dronized’ it with autonomous flight capability. K50 flight authorization was issued by French civil aviation authority in January 2018.
2) Each of SZTAKI and RICOH has developed a prototype of their monocular- and stereo-vision systems to detect and track runway features for aircraft final approach navigation purpose. Both prototypes have been integrated on K50, and flight-tested for image acquisition and real-time image processing. RICOH’s prototype is the world’s first long-range stereo camera.
3) ONERA and SZTAKI have designed vision-integrated navigation filters which fuse the vision system outputs with other navigation sensor measurements (GPS, ILS, IMU, etc.).
The navigation filters are augmented with integrity monitoring function for fault detection and protection level computation. Their navigation performance was evaluated through closed-loop and open-loop simulations with emulated and real sensor data respectively, against sensor faults modelled by ENRI.
4) University of Tokyo and RICOH proposed vision-based obstacle avoidance algorithm by online trajectory planning under the detection uncertainty. They performed numerical simulation evaluation for trajectory optimization, and flight experiments for stereo vision-based flying object detection.
Contributions to the state of the art:
1) The FDD/FTC designs proposed by the partners have been flight-validated in real-time onboard the full-scale research aircraft. This is the first step necessary towards maturing their TRLs, prior to flying them on the real test planes of aircraft manufacturers.
2) An innovative onboard vision-based control surface monitoring system was developed in this project. RICOH submitted a patent application in Japan.
3) The world’s first long-range stereo camera system was developed by RICOH for aviation application, based on their matured technology acquired in automobile application.
4) Unlike for robotic applications, a navigation system for civil aviation application needs to satisfy stringent requirements on its precision and integrity. VISION proposes an onboard vision-integrated navigation system for aircraft final approach with integrity monitoring function, and provides its feasibility study.
The advanced GN&C solutions developed in WP3 and WP4 are flight-demonstrated on real aircraft platforms. In the end, VISION proposes TRL-matured “smarter” GN&C techniques to aircraft and aircraft-system manufacturers.
Potential impacts of VISION include:
1) The flight-evaluated advanced GN&C technologies proposed in VISION give new solutions to enhance the flight safety in civil aviation. These will also contribute to a current trend of the aircraft manufacturers towards “more automated” aircraft.
2) The project results are also applicable to “drone” systems for increasing their flight safety. Since the certification/commercialization process is lighter for drones than for civil aviation, the project will impact such drone industries (often SMEs) relatively rapidly by bringing TRL-matured solutions to secure drone operations.
3) The project has given unprecedented opportunities for partners to test FDD/FTC schemes on a full-scale real aircraft operated by JAXA, hence contributing to mature their TRL. At the same time, VISION eases the access of the European aeronautic community to the matured visual sensing technology of Japanese industry (RICOH), and vice versa.
4) Throughout the EU-Japan joint work sessions, the scientific collaboration between the EU and Japan partners is reinforced and resulted in a number of joint publications.
1) The FDD/FTC designs proposed by the partners have been flight-validated in real-time onboard the full-scale research aircraft. This is the first step necessary towards maturing their TRLs, prior to flying them on the real test planes of aircraft manufacturers.
2) An innovative onboard vision-based control surface monitoring system was developed in this project. RICOH submitted a patent application in Japan.
3) The world’s first long-range stereo camera system was developed by RICOH for aviation application, based on their matured technology acquired in automobile application.
4) Unlike for robotic applications, a navigation system for civil aviation application needs to satisfy stringent requirements on its precision and integrity. VISION proposes an onboard vision-integrated navigation system for aircraft final approach with integrity monitoring function, and provides its feasibility study.
The advanced GN&C solutions developed in WP3 and WP4 are flight-demonstrated on real aircraft platforms. In the end, VISION proposes TRL-matured “smarter” GN&C techniques to aircraft and aircraft-system manufacturers.
Potential impacts of VISION include:
1) The flight-evaluated advanced GN&C technologies proposed in VISION give new solutions to enhance the flight safety in civil aviation. These will also contribute to a current trend of the aircraft manufacturers towards “more automated” aircraft.
2) The project results are also applicable to “drone” systems for increasing their flight safety. Since the certification/commercialization process is lighter for drones than for civil aviation, the project will impact such drone industries (often SMEs) relatively rapidly by bringing TRL-matured solutions to secure drone operations.
3) The project has given unprecedented opportunities for partners to test FDD/FTC schemes on a full-scale real aircraft operated by JAXA, hence contributing to mature their TRL. At the same time, VISION eases the access of the European aeronautic community to the matured visual sensing technology of Japanese industry (RICOH), and vice versa.
4) Throughout the EU-Japan joint work sessions, the scientific collaboration between the EU and Japan partners is reinforced and resulted in a number of joint publications.