Speech recognition and AI help take the pressure off aircrew
Aircrew workloads peak when faced with a combination of unpredictable situations: meteorological conditions; high-density traffic; system failures; and flight operations like take-off, climb, descent, approach and landing. The amount of information and number of actions that need to be processed by the crew may become unmanageable, affecting flight safety. The EU-funded VOICI project addressed this threat by developing an intelligent ‘natural crew assistant’ for the cockpit environment. The system comprises three main technologies, namely sound recording, speech recognition and artificial intelligence (AI). This includes a cockpit-embedded speech-processing system that understands aviation terminology, as well as an array of low-noise optical microphones and optimised array processing for it. The VOICI system also features a new and more efficient speech synthesis, adapted to aviation terminology and noise levels.
Assessed under realistic conditions
Project partners aimed to provide a proof-of-concept demonstrator capable of listening to all communications in the cockpit, both between crew members, and between crew and air traffic control. “The VOICI system should recognise and interpret speech content, interact with the crew, and fulfil crew requests to simplify crew tasks and reduce cognitive workload,” outlines project coordinator, Tor Arne Reinen. Researchers also developed a realistic audio evaluation environment for technology experiments. This facilitated the development of the crew assistant and enabled evaluation of its performance, including the speech capture and recognition technologies for use in a noisy cockpit, together with the intelligent dialogue system with automatic speech synthesis as its main output. The audio testing environment involved a 3D physical model of a Falcon 2000S cockpit, including loudspeaker reproduction of noise recordings from a real flight. “We have demonstrated that the crew assistant is feasible under the very high noise levels of an aviation cockpit,” Reinen explains.
Multiple benefits
Speech capture is achieved through both the pilot's headset and an ambient microphone array. Speech recognition using deep neural networks and the dialogue system were developed explicitly for the cockpit environment and include aviation terminology and robustness to handle high levels of background noise. The systems function independently of cloud-based systems and employ dedicated language models for the cockpit scenario. According to Reinen, all the algorithms underlying the dialogue system have been implemented and tested: from the Natural Language Understanding unit that understands natural requests to the Dialogue Core which handles the conversation flow. “Particular emphasis has been placed on the ability of the voice assistant to use contextual data,” he notes. By reducing crew workload, VOICI will contribute to: optimisation of operations, flight safety and crew awareness; better maintenance; reduced cost of operations; and generally higher efficiency and lower stress. “VOICI comprises both small and medium-sized enterprises (SMEs) and research institutes, and cooperation within the consortium will contribute to innovation and job creation,” Reinen adds.
Keywords
VOICI, cockpit, aviation, speech recognition, natural crew assistant, neural networks
