Final Report Summary - BRAINFLIGHT (Brain controlled aircraft flight using multiple feedback mechanisms)
Executive Summary:
Several studies have revealed that the activity of neurons is sufficiently capable of providing enough data to enable the control of an electronic device using solely signals provided by the brain. Project BRAINFLIGHT proposes the application of this kind of control in the world of aircraft, to enable people to control aircraft using only neural signals emitted from their brain. This kind of approach constitutes a novel concept for the control of these platforms, and was until recently only conceived as science fiction.
The goal of project BRAINFLIGHT is to create a novel approach to aircraft control, and to assess the performance of this concept. This project is mainly focused on investigating what are the best approaches that allow fast learning to control an aircraft using brain signals, while allowing pilots to multitask. Project BRAINFLIGHT will test an innovative approach for brain control of flight, which takes advantage of the amazing ability that the brain has of learning to use novel tools using operant conditioning. The project will test how the use of this approach, as well as different feedback mechanisms for the pilot (visual and/or tactile), will affect learning and performance compared to conventional control. The project will also test the ability of pilots to multi-task using brain control by performing experiments where integration and interference between multimodal signals are investigated. In addition, if the control described above does not allow control of many degrees of freedom or allow for multitasking, we will test an alternative approach in which pilots will use their neural activity to select particular tactile sensory stimuli in order to control the aircraft.
Finally, after the best approach and parameters are selected, the functionality of this brain control scheme will be tested in a high fidelity simulator as well as in a real life UAV, which will provide the basis for the applicability of this project to future transport systems.
Project Context and Objectives:
The objective of any control system is to provide clear and easy to use means to control an equipment with the minimum amount of complexity and with a user friendly interface. Ideally, users should be able to exert control with the least amount of training and maximum efficiency while performing other tasks. The rising number of flights over recent years and the increase in the complexity of aircraft (A/C) systems and technologies has contributed to:
• Increased costs and duration of pilot training;
• Limited access of piloting to individuals with particular characteristics;
• Growing complexity of cockpit environments;
• Greater workloads for pilots (which correlate to diminished situational awareness);
Hence, aeronautics research has encouraged the development of innovative means to control A/C and innovative cockpit environments. BRAINFLIGHT proposes to investigate the application of an emerging technology in Neuroscience and Neuroengineering, as a radical new approach to guide and control an A/C. The question behind BRAINFLIGHT is: “Is it possible to control an aircraft using solely the signals emitted from the human brain, without any other intermediary control devices?” The capability to guide and control an aircraft using the brain is a radically new approach that the consortium believes could eventually decrease the complexity of cockpit environments and the resulting downsides mentioned above. It could enable people currently prevented from piloting to do so, and it might free up capacity to do secondary tasks at the same time. This proposal aims to create a system that allows pilots to learn to control an A/C using brain activity and eventually consolidate it and automate it in such a way that it allows intuitive control and releases the pilot’s higher cognitive functions to other activities.
Using visual information to provide feedback about A/C movement is perhaps most natural and intuitive for a pilot. However, tactile information has also been shown to be quite intuitive (e.g. if somebody taps you on the shoulder, you automatically turn your head and know where to look), and has the advantage of freeing auditory and visual information channels. Furthermore, tactile stimuli applied around the waist have proven to be successful as navigation display. Recent work describes the combination of tactile stimuli with BMI. This proposal will combine new BMI techniques with tactile feedback and stimuli. After testing these ideas separately, the best working concepts will be combined and tested in a simulator for interference with each other and with secondary tasks. Finally the BRAINFLIGHT concept will be taken into the field for a first real world tests with Unmanned Aerial Vehicles (UAV), as a precursor to progressing this technology’s readiness level towards real A/C testing.
The consortium has established two precise overarching goals for BRAINFLIGHT:
• To Demonstrate the Feasibility And Identify Benefits And Pay-Offs of replacing conventional control inceptors by a Brain Machine Interface (BMI) in pilot “manual” closed-loop type flight control tasks;
• To bring the TRL level of brain controlled aircraft up to TRL3 (Analytical and experimental critical function and/or characteristic proof-of-concept);
This goal can be divided into several lower level objectives with associated foreseen achievements.
- Increase the maturity of applying BMI techniques from Neuroscience to Aeronautical guidance and control.
- Identify which BMI approach and sensory feedback mechanisms that can be used to control a plane
- Demonstrate the feasibility of replacing conventional control inceptors by a Brain Machine Interface (BMI) in A/C guidance and control
- Identify benefits and pay-offs of the BRAINFLIGHT approach
- Identify roadblocks towards the further development and maturing of the BRAINFLIGHT approach
Project Results:
During this first year of the BRAINFLIGHT project, the consortium has made some progress concerning the contextualization analysis and design of the BRAINFLIGHT concept and a possible BRAINFLIGHT system. Possible aeronautical applications for the BRAINFLIGHT system were presented based on the development of five different scenarios:
• General Aviation (GA) flight under Instrument Flight Rules;
• Commercial Aviation (CA) landing under IFR in low visibility;
• Unmanned Aerial Vehicle (UAV) flight in stressful conditions
• Acrobatic aircraft flight
Three different BCI approaches were defined:
• Active (created from conscious brain activity to try to control);
• Reactive (response to an external stimuli);
• Passive (from non-voluntary control activity from the brain).
A matching exercise between the opinion of interviewed pilots and the application scenarios was performed to assess how and when the BRAINFLIGHT system could be beneficial for the pilots to use during a normal flight mission
The high-level system and sub-systems requirements were defined and the functionalities of the different blocks composing the overall BRAINFLIGHT system were defined, as well as their interfaces and the flows in the system architecture.
Validation scenarios for the BRAINFLIGHT system were defined based on reference flight missions, the fact that BCI concepts pick up various signals including artefacts from muscular activity and possible UAV manoeuvres to be performed in the flight testing phase.
Although the following were not concluded during the first year, the consortium defined and started implementing system components and modules, specifically the signal acquisition pipeline. Codingand optimization of the decoder brain signals was initiated with good preliminary results. A dry electrode was developed and adaptations to the Ground Control System (GCS) from TEK were implemented to comply with the BRAINFLIGHT architecture, where hardware in the loop simulations with the GCS using X-Plane was performed:
• The system is now capable to receive multiple control channels.
• Current state of update of the control is 15ms.
• Direct link from the ground to aircraft is delayed by about 100ms (one way delay)
A tactile suit was designed, consisting of multiple straps, and a control module with 16 tactors. Some tests were performed in combining tactile feedback with manual control in TUM’s flight simulator, which led to improvements in the tactile display design.
Based on the analysis of flight handling qualities as defined by the European Aviation Safety Agency’s Certification Specifications for Normal, Utility, Aerobatic, and Commuter Category Aeroplanes CS 23; the US military specifications MIL-F-8785C and MIL-HDBK-1797, and guidance material from various authors some characteristics of the BRAINFLIGHT flight control system were detailed. Possible command variables were identified and their suitability for brain control in terms of handling analysed. Based on this work a framework for the initial design of two controllers was created:
• The first controller is dedicated to brain control and incorporates a number of unconventional design features
• the second controller is more similar to controllers for manual control.The second year started with the following achievements mainly for the third work package of the project:
• Definition and implementation of system components and modules, specifically signal acquisition pipeline, system architecture and software/file standards for interface and exchange information requirements.
• Coding and optimization of the decoder brain signals.
• Definition and testing of the features signal for the training and control of a cursor.
• Test and optimization of the suitable channels and features of the EEG signals used for the decoder. • Tactile suit was designed, consisting of multiple straps, and a control module with 16 tactors. The tactors were placed using the straps, on the front and back, legs and arms.
• Updated on the design of the haptic belt have been performed to accommodate better the feedback required by the pilot
• Analysis of current handling qualities criteria and requirements, such as those contained in several certification specifications from EASA and MIL.
Work package 4 had the following achievements:
• Adaptations have been performed to both the GCS and the Diamond DA42 simulator to accommodate the EEG system provided by FCHAMP
• Signal filtering and modification have been performed in order for the pilot to better have the feeling of control of the aircraft
• Preliminary connection tests have been performed and signal processing has been conducted to establish the flow between the BMI system from FCHAMP and both the Diamond DA42 simulator and the GCS from the UAV.
The validation of the system ensued in WP5, in which the BRAINFLIGHT system was tested in the simulator environment from the DA42 and learning test were conducted for the pilot to learn the control paradigm.
The project ended with the field test of the Brainflight system using the UAV platform from TEKEVER, the AR4. Good results were obtained from the real life testing.
In terms of dissemination, four scientific papers were published and presented throughout the project and project BRAINFLIGHT had a big media coverage with several newspapers and websites publishing news and videos generated from the project.
Potential Impact:
We believe BRAINFLIGHT will enable people who currently are unable to pilot aircraft to do so still during this century (at least for GA aircraft). Currently, people with physical disabilities cannot pilot commercial aircraft or pilot GA aircraft (although some specific cases might be able to pilot specially adapted GA aircraft). Pilots with physical disabilities are almost inexistent in commercial aircraft because of the simple fact that this would require all aircraft in a fleet to be modified to cater for the pilot’s specific disability. Individual owners of GA may and can modify their own aircraft to account for their disabilities. By removing the requirement for common control mechanisms such as centre sticks, control columns or rudder pedals, BRAINFLIGHT can open the door for disabled people to pursue pilot careers throughout Europe, be it at GA level (medium term) or as commercial pilots (long term) and will enlarge the number of people who could get involved in GA traffic. Of course, further studies, in particular concerning safety of both pilots and aircraft and the failure rate of BCI/BMI systems will need to be carried out. BRAINFLIGHT will have a positive impact in providing access to aircraft control to larger groups of people with no need for special physical skills. The feasibility of this impact can be easily demonstrated by running tests with people with physical disabilities that would prevent them from flying a GA aircraft without special adaptation.
BRAINFLIGHT proved the concept of controlling (either high level control or low level control) an aircraft with neural signals. A large set of human skills is done almost unconsciously by people (e.g. breathing being the most notorious, balancing, walking, etc.) – i.e. people do not need to think about the act of breathing or how they walk to perform these actions. In addition to these basic skills, other more advanced skills can still be learned and trained in such a way they free up higher cognitive functions to focus on other aspects. Perhaps the most obvious example concerns professional sportsmen. These professionals are capable of carrying out their basic game skills (such as dribbling, passing, shooting or tackling) while focusing their attention on the way the game plays out just as fighter pilots can focus on the way dogfights play out without the need to focus on the execution of a particular manoeuvre. The “subconscious” capability enables people to focus their attention and cognitive skills elsewhere.
If the results from the project tests are encouraging, further research, coupled with (extensive) training, could lead to a decrease of pilot basic workload.
BRAINFLIGHT could result in the removal of the need for conventional cockpits altogether or, at least, in the possibility to redesign completely the cockpit paradigms of the future. With the introduction of a brain control device, the pilot is able to control the airplane without needing the conventional feedback devices that are present in the cockpit, but instead use the feedback mechanisms developed within this project. The possibility to remove particular instruments from the cockpit or the entire replacement or re-design of cockpits would need the following steps to be taken by the community.
Previous research in the field of neuro-sciences have shown positive results in terms of the time required by subjects’ brains to naturally learn to use new tools/instruments by an operant process of trial and error (some research has shown that in 2 weeks approximately, 80% efficiency in control using brain signals could be achieved for primates and rodents). If BRAINFLIGHT research corroborates these results, an important impact in terms of pilot training could be achieved. For example, if it’s possible for humans to achieve 80% efficiency in control in 2 weeks in a similar way to rodents and primates, then pilot training could be significantly reduced or conversely improved.
In terms of dissemination activities, the following activities were performed:
- The BRAINFLIGHT project has been presented at the Origins of Innovation 2013 Workshop in Brussels on the 22nd of April 2013.
- A paper was issued and presented in the HCI International 2013, on the 26th of July 2013, named:Zander, T. O., Brönstrup, J., Klose, E., Sonnenberg, R. S., Vos, W. K., & Grootjen, M. (2013).
“Towards implicit control through steady-state somatosensory evoked potentials.”
In Universal Access in Human-Computer Interaction. Design Methods, Tools, and Interaction Techniques for eInclusion(pp. 682-688). Springer Berlin Heidelberg.
-A paper was issued and presented in the BCI Meeting 2013, on June 2013, named:
“Towards implicit control through steady-state somatosensory evoked potentials”
Proceedings of the Fifth International Brain-Computer Interface Meeting 2013
Thorsten Zander*, Team PhyPA, TU Berlin; Jonas Broenstrup, Team PhyPA, TU Berlin; Elisa Klose, Team PhyPA, TU Berlin; Robert Sonnenberg, Team PhyPA, TU Berlin; Wouter Vos, Eagle Science; Marc Grootjen, Eagle Science
- A pamphlet was created and presented at the AIRTEC 2013 fair between 5 and 7 of November 2013.
- The following paper will be presented at the 63rd edition of the Luft- und Raumfahrtkongress 2014 in September 2014:
“FIRST PILOT-IN-THE-LOOP SIMULATOR EXPERIMENTS ON BRAIN CONTROL OF HORIZONTAL AIRCRAFT MOTION” - T. Fricke, T.O. Zander, K. Gramann, F. Holzapfel
- A master thesis was developed taking into account some of the results obtained from the work in BRAINFLIGHT:
C.Vermorken (2014). Enhancing BCI Based Navigation by Adding Intention
“Detection and a Virtual Assistant. Thesis, TU Delft, the Netherlands.”
- Partners of BRAINFLIGHT have created several Press Releases and conducted some interviews regarding the project; some of them are detailed as follows:
TEK
http://tek.sapo.pt/noticias/computadores/projeto_de_aviao_que_se_controla_com_o_cerebr_1387564.html
http://exameinformatica.sapo.pt/noticias/mercados/2014-06-02-Os-pilotos-que-controlam-o-aviao-com-a-mente
http://www.publico.pt/tecnologia/noticia/projecto-europeu-mostra-aviao-controlado-pelo-cerebro-humano-1637987
http://www.tsf.pt/PaginaInicial/Vida/Interior.aspx?content_id=3941125 o TUM
https://www.tum.de/en/about-tum/news/press- releases/short/article/31531/
http://www.airportzentrale.de/hirngesteuertes-fliegen-funktioniert- erstaunlich-gut/33235/
EAGLE
http://www.nu.nl/tech/3787697/duitse-vluchtsimulator-kan-met-gedachten-bestuurd-worden.html
Several interviews have been conducted regarding the work performed by the BRAINFLIGHT consortium in several countries
TEK:
- TV channel NTN24
- Talk Radio Europe o EAGLE:
- http://www.dw.de/brain-controlled-interfacing-what/av-17710616
An invitation has been received by the consortium to create a book relating to the success story of the BRAINFLIGHT project
List of Websites:
http://www.fp7-brainflight.eu
Several studies have revealed that the activity of neurons is sufficiently capable of providing enough data to enable the control of an electronic device using solely signals provided by the brain. Project BRAINFLIGHT proposes the application of this kind of control in the world of aircraft, to enable people to control aircraft using only neural signals emitted from their brain. This kind of approach constitutes a novel concept for the control of these platforms, and was until recently only conceived as science fiction.
The goal of project BRAINFLIGHT is to create a novel approach to aircraft control, and to assess the performance of this concept. This project is mainly focused on investigating what are the best approaches that allow fast learning to control an aircraft using brain signals, while allowing pilots to multitask. Project BRAINFLIGHT will test an innovative approach for brain control of flight, which takes advantage of the amazing ability that the brain has of learning to use novel tools using operant conditioning. The project will test how the use of this approach, as well as different feedback mechanisms for the pilot (visual and/or tactile), will affect learning and performance compared to conventional control. The project will also test the ability of pilots to multi-task using brain control by performing experiments where integration and interference between multimodal signals are investigated. In addition, if the control described above does not allow control of many degrees of freedom or allow for multitasking, we will test an alternative approach in which pilots will use their neural activity to select particular tactile sensory stimuli in order to control the aircraft.
Finally, after the best approach and parameters are selected, the functionality of this brain control scheme will be tested in a high fidelity simulator as well as in a real life UAV, which will provide the basis for the applicability of this project to future transport systems.
Project Context and Objectives:
The objective of any control system is to provide clear and easy to use means to control an equipment with the minimum amount of complexity and with a user friendly interface. Ideally, users should be able to exert control with the least amount of training and maximum efficiency while performing other tasks. The rising number of flights over recent years and the increase in the complexity of aircraft (A/C) systems and technologies has contributed to:
• Increased costs and duration of pilot training;
• Limited access of piloting to individuals with particular characteristics;
• Growing complexity of cockpit environments;
• Greater workloads for pilots (which correlate to diminished situational awareness);
Hence, aeronautics research has encouraged the development of innovative means to control A/C and innovative cockpit environments. BRAINFLIGHT proposes to investigate the application of an emerging technology in Neuroscience and Neuroengineering, as a radical new approach to guide and control an A/C. The question behind BRAINFLIGHT is: “Is it possible to control an aircraft using solely the signals emitted from the human brain, without any other intermediary control devices?” The capability to guide and control an aircraft using the brain is a radically new approach that the consortium believes could eventually decrease the complexity of cockpit environments and the resulting downsides mentioned above. It could enable people currently prevented from piloting to do so, and it might free up capacity to do secondary tasks at the same time. This proposal aims to create a system that allows pilots to learn to control an A/C using brain activity and eventually consolidate it and automate it in such a way that it allows intuitive control and releases the pilot’s higher cognitive functions to other activities.
Using visual information to provide feedback about A/C movement is perhaps most natural and intuitive for a pilot. However, tactile information has also been shown to be quite intuitive (e.g. if somebody taps you on the shoulder, you automatically turn your head and know where to look), and has the advantage of freeing auditory and visual information channels. Furthermore, tactile stimuli applied around the waist have proven to be successful as navigation display. Recent work describes the combination of tactile stimuli with BMI. This proposal will combine new BMI techniques with tactile feedback and stimuli. After testing these ideas separately, the best working concepts will be combined and tested in a simulator for interference with each other and with secondary tasks. Finally the BRAINFLIGHT concept will be taken into the field for a first real world tests with Unmanned Aerial Vehicles (UAV), as a precursor to progressing this technology’s readiness level towards real A/C testing.
The consortium has established two precise overarching goals for BRAINFLIGHT:
• To Demonstrate the Feasibility And Identify Benefits And Pay-Offs of replacing conventional control inceptors by a Brain Machine Interface (BMI) in pilot “manual” closed-loop type flight control tasks;
• To bring the TRL level of brain controlled aircraft up to TRL3 (Analytical and experimental critical function and/or characteristic proof-of-concept);
This goal can be divided into several lower level objectives with associated foreseen achievements.
- Increase the maturity of applying BMI techniques from Neuroscience to Aeronautical guidance and control.
- Identify which BMI approach and sensory feedback mechanisms that can be used to control a plane
- Demonstrate the feasibility of replacing conventional control inceptors by a Brain Machine Interface (BMI) in A/C guidance and control
- Identify benefits and pay-offs of the BRAINFLIGHT approach
- Identify roadblocks towards the further development and maturing of the BRAINFLIGHT approach
Project Results:
During this first year of the BRAINFLIGHT project, the consortium has made some progress concerning the contextualization analysis and design of the BRAINFLIGHT concept and a possible BRAINFLIGHT system. Possible aeronautical applications for the BRAINFLIGHT system were presented based on the development of five different scenarios:
• General Aviation (GA) flight under Instrument Flight Rules;
• Commercial Aviation (CA) landing under IFR in low visibility;
• Unmanned Aerial Vehicle (UAV) flight in stressful conditions
• Acrobatic aircraft flight
Three different BCI approaches were defined:
• Active (created from conscious brain activity to try to control);
• Reactive (response to an external stimuli);
• Passive (from non-voluntary control activity from the brain).
A matching exercise between the opinion of interviewed pilots and the application scenarios was performed to assess how and when the BRAINFLIGHT system could be beneficial for the pilots to use during a normal flight mission
The high-level system and sub-systems requirements were defined and the functionalities of the different blocks composing the overall BRAINFLIGHT system were defined, as well as their interfaces and the flows in the system architecture.
Validation scenarios for the BRAINFLIGHT system were defined based on reference flight missions, the fact that BCI concepts pick up various signals including artefacts from muscular activity and possible UAV manoeuvres to be performed in the flight testing phase.
Although the following were not concluded during the first year, the consortium defined and started implementing system components and modules, specifically the signal acquisition pipeline. Codingand optimization of the decoder brain signals was initiated with good preliminary results. A dry electrode was developed and adaptations to the Ground Control System (GCS) from TEK were implemented to comply with the BRAINFLIGHT architecture, where hardware in the loop simulations with the GCS using X-Plane was performed:
• The system is now capable to receive multiple control channels.
• Current state of update of the control is 15ms.
• Direct link from the ground to aircraft is delayed by about 100ms (one way delay)
A tactile suit was designed, consisting of multiple straps, and a control module with 16 tactors. Some tests were performed in combining tactile feedback with manual control in TUM’s flight simulator, which led to improvements in the tactile display design.
Based on the analysis of flight handling qualities as defined by the European Aviation Safety Agency’s Certification Specifications for Normal, Utility, Aerobatic, and Commuter Category Aeroplanes CS 23; the US military specifications MIL-F-8785C and MIL-HDBK-1797, and guidance material from various authors some characteristics of the BRAINFLIGHT flight control system were detailed. Possible command variables were identified and their suitability for brain control in terms of handling analysed. Based on this work a framework for the initial design of two controllers was created:
• The first controller is dedicated to brain control and incorporates a number of unconventional design features
• the second controller is more similar to controllers for manual control.The second year started with the following achievements mainly for the third work package of the project:
• Definition and implementation of system components and modules, specifically signal acquisition pipeline, system architecture and software/file standards for interface and exchange information requirements.
• Coding and optimization of the decoder brain signals.
• Definition and testing of the features signal for the training and control of a cursor.
• Test and optimization of the suitable channels and features of the EEG signals used for the decoder. • Tactile suit was designed, consisting of multiple straps, and a control module with 16 tactors. The tactors were placed using the straps, on the front and back, legs and arms.
• Updated on the design of the haptic belt have been performed to accommodate better the feedback required by the pilot
• Analysis of current handling qualities criteria and requirements, such as those contained in several certification specifications from EASA and MIL.
Work package 4 had the following achievements:
• Adaptations have been performed to both the GCS and the Diamond DA42 simulator to accommodate the EEG system provided by FCHAMP
• Signal filtering and modification have been performed in order for the pilot to better have the feeling of control of the aircraft
• Preliminary connection tests have been performed and signal processing has been conducted to establish the flow between the BMI system from FCHAMP and both the Diamond DA42 simulator and the GCS from the UAV.
The validation of the system ensued in WP5, in which the BRAINFLIGHT system was tested in the simulator environment from the DA42 and learning test were conducted for the pilot to learn the control paradigm.
The project ended with the field test of the Brainflight system using the UAV platform from TEKEVER, the AR4. Good results were obtained from the real life testing.
In terms of dissemination, four scientific papers were published and presented throughout the project and project BRAINFLIGHT had a big media coverage with several newspapers and websites publishing news and videos generated from the project.
Potential Impact:
We believe BRAINFLIGHT will enable people who currently are unable to pilot aircraft to do so still during this century (at least for GA aircraft). Currently, people with physical disabilities cannot pilot commercial aircraft or pilot GA aircraft (although some specific cases might be able to pilot specially adapted GA aircraft). Pilots with physical disabilities are almost inexistent in commercial aircraft because of the simple fact that this would require all aircraft in a fleet to be modified to cater for the pilot’s specific disability. Individual owners of GA may and can modify their own aircraft to account for their disabilities. By removing the requirement for common control mechanisms such as centre sticks, control columns or rudder pedals, BRAINFLIGHT can open the door for disabled people to pursue pilot careers throughout Europe, be it at GA level (medium term) or as commercial pilots (long term) and will enlarge the number of people who could get involved in GA traffic. Of course, further studies, in particular concerning safety of both pilots and aircraft and the failure rate of BCI/BMI systems will need to be carried out. BRAINFLIGHT will have a positive impact in providing access to aircraft control to larger groups of people with no need for special physical skills. The feasibility of this impact can be easily demonstrated by running tests with people with physical disabilities that would prevent them from flying a GA aircraft without special adaptation.
BRAINFLIGHT proved the concept of controlling (either high level control or low level control) an aircraft with neural signals. A large set of human skills is done almost unconsciously by people (e.g. breathing being the most notorious, balancing, walking, etc.) – i.e. people do not need to think about the act of breathing or how they walk to perform these actions. In addition to these basic skills, other more advanced skills can still be learned and trained in such a way they free up higher cognitive functions to focus on other aspects. Perhaps the most obvious example concerns professional sportsmen. These professionals are capable of carrying out their basic game skills (such as dribbling, passing, shooting or tackling) while focusing their attention on the way the game plays out just as fighter pilots can focus on the way dogfights play out without the need to focus on the execution of a particular manoeuvre. The “subconscious” capability enables people to focus their attention and cognitive skills elsewhere.
If the results from the project tests are encouraging, further research, coupled with (extensive) training, could lead to a decrease of pilot basic workload.
BRAINFLIGHT could result in the removal of the need for conventional cockpits altogether or, at least, in the possibility to redesign completely the cockpit paradigms of the future. With the introduction of a brain control device, the pilot is able to control the airplane without needing the conventional feedback devices that are present in the cockpit, but instead use the feedback mechanisms developed within this project. The possibility to remove particular instruments from the cockpit or the entire replacement or re-design of cockpits would need the following steps to be taken by the community.
Previous research in the field of neuro-sciences have shown positive results in terms of the time required by subjects’ brains to naturally learn to use new tools/instruments by an operant process of trial and error (some research has shown that in 2 weeks approximately, 80% efficiency in control using brain signals could be achieved for primates and rodents). If BRAINFLIGHT research corroborates these results, an important impact in terms of pilot training could be achieved. For example, if it’s possible for humans to achieve 80% efficiency in control in 2 weeks in a similar way to rodents and primates, then pilot training could be significantly reduced or conversely improved.
In terms of dissemination activities, the following activities were performed:
- The BRAINFLIGHT project has been presented at the Origins of Innovation 2013 Workshop in Brussels on the 22nd of April 2013.
- A paper was issued and presented in the HCI International 2013, on the 26th of July 2013, named:Zander, T. O., Brönstrup, J., Klose, E., Sonnenberg, R. S., Vos, W. K., & Grootjen, M. (2013).
“Towards implicit control through steady-state somatosensory evoked potentials.”
In Universal Access in Human-Computer Interaction. Design Methods, Tools, and Interaction Techniques for eInclusion(pp. 682-688). Springer Berlin Heidelberg.
-A paper was issued and presented in the BCI Meeting 2013, on June 2013, named:
“Towards implicit control through steady-state somatosensory evoked potentials”
Proceedings of the Fifth International Brain-Computer Interface Meeting 2013
Thorsten Zander*, Team PhyPA, TU Berlin; Jonas Broenstrup, Team PhyPA, TU Berlin; Elisa Klose, Team PhyPA, TU Berlin; Robert Sonnenberg, Team PhyPA, TU Berlin; Wouter Vos, Eagle Science; Marc Grootjen, Eagle Science
- A pamphlet was created and presented at the AIRTEC 2013 fair between 5 and 7 of November 2013.
- The following paper will be presented at the 63rd edition of the Luft- und Raumfahrtkongress 2014 in September 2014:
“FIRST PILOT-IN-THE-LOOP SIMULATOR EXPERIMENTS ON BRAIN CONTROL OF HORIZONTAL AIRCRAFT MOTION” - T. Fricke, T.O. Zander, K. Gramann, F. Holzapfel
- A master thesis was developed taking into account some of the results obtained from the work in BRAINFLIGHT:
C.Vermorken (2014). Enhancing BCI Based Navigation by Adding Intention
“Detection and a Virtual Assistant. Thesis, TU Delft, the Netherlands.”
- Partners of BRAINFLIGHT have created several Press Releases and conducted some interviews regarding the project; some of them are detailed as follows:
TEK
http://tek.sapo.pt/noticias/computadores/projeto_de_aviao_que_se_controla_com_o_cerebr_1387564.html
http://exameinformatica.sapo.pt/noticias/mercados/2014-06-02-Os-pilotos-que-controlam-o-aviao-com-a-mente
http://www.publico.pt/tecnologia/noticia/projecto-europeu-mostra-aviao-controlado-pelo-cerebro-humano-1637987
http://www.tsf.pt/PaginaInicial/Vida/Interior.aspx?content_id=3941125 o TUM
https://www.tum.de/en/about-tum/news/press- releases/short/article/31531/
http://www.airportzentrale.de/hirngesteuertes-fliegen-funktioniert- erstaunlich-gut/33235/
EAGLE
http://www.nu.nl/tech/3787697/duitse-vluchtsimulator-kan-met-gedachten-bestuurd-worden.html
Several interviews have been conducted regarding the work performed by the BRAINFLIGHT consortium in several countries
TEK:
- TV channel NTN24
- Talk Radio Europe o EAGLE:
- http://www.dw.de/brain-controlled-interfacing-what/av-17710616
An invitation has been received by the consortium to create a book relating to the success story of the BRAINFLIGHT project
List of Websites:
http://www.fp7-brainflight.eu