Explainable AI for increased trust in air traffic management software
While artificial intelligence (AI) is used in air traffic management (ATM) support functions, like post-incident analysis or demand forecasting, it is not yet fully integrated across operations. AI does not fit into traditional engineering work cycles, which favour linear steps with predictable outcomes. This makes it especially difficult for national authorities to validate and certify for safety-critical functions. According to José Manuel Cordero, from www.crida.es (CRIDA) (website in Spanish only): “While AI can enhance ATM performance – increasing capacity, reducing delays and improving safety – as controllers are still ultimately responsible for their decisions, building trust remains fundamental to AI adoption.” The TAPAS (Towards an Automated and exPlainable ATM System) project was funded within the SESAR Joint Undertaking, a public-private partnership set up to modernise Europe’s ATM system. Coordinated by CRIDA, it has identified which additional features should be considered in AI systems to increase their acceptance. The project has designed and tested two explainable AI (XAI) prototypes, while also developing AI in ATM guidelines.
Unpacking the ‘black box’ of AI
AI techniques are often not readily understandable to human operators. The self-contained design of most AI means that users typically don’t understand why one decision was taken over another, making it difficult to reverse-engineer a successful or failed outcome. “We need to increase AI’s explainability, that is to make its inner workings – its rules, abilities and limitations – more transparent to potential users,” adds Cordero, TAPAS project coordinator. So TAPAS used XAI techniques which explain decisions, while the system is actually making them. The project first identified key criteria, building an explainability framework. To then calibrate the detail and range of explanations required, this framework was integrated into real-time simulations of various scenarios. Air traffic controllers from ENAIRE (the Spanish air navigation service provider), trainers and industry representatives tested these simulations at CRIDA in Madrid. TAPAS added enhanced AI features to existing ATM systems running on traditional (deterministic) algorithms. Different levels of automation were explored, with machines progressively taking over more functions and humans becoming more monitor than operator. “It’s like chess: some plays are self-explanatory, some need a little explanation, others a lot more. But there is a point where exhaustive explanations don’t add value because they are too complex and users simply have to trust the underlying logic,” explains Cordero. These processes enabled TAPAS to develop their XAI prototypes combined with visual analytics, which extracts information from the AI’s functioning, displaying it on-screen in user-friendly formats. The project tested their XAI in two use cases: non-safety-critical air traffic flow and capacity management, and safety-critical conflict detection and resolution. These were conducted with air traffic controllers and air traffic control operational platforms for a week. To explore a range of system behaviours, different automation and traffic levels were again explored, including during challenging conditions. “Observation and feedback indicated impressive user acceptance. We have also included recommendations in our general explainability framework to help guide future applications,” remarks Cordero.
Ready for take-off
The AI techniques explored by TAPAS are considered by the European ATM Master Plan key enablers of higher aviation automation levels, promising to boost air travel capacity – an EU aviation strategy priority. With scientific papers already available in the Outcomes section of the project website and a workshop in the pipeline, Cordero points out that TAPAS is ready to start making its contribution. “Our explainability framework could already help spur on adoption of ‘capacity on demand’, for example, as a priority automated application,” he says. The European Union Aviation Safety Agency has also expressed interest in incorporating the results of the project’s work on higher automation levels into the next edition of their AI guidelines.
Keywords
TAPAS, air traffic controller, AI, transparent, safety-critical, aviation, automation, travel, algorithm
