Periodic Reporting for period 1 - MED1stMR (Medical First Responder Training using a Mixed Reality Approach featuring haptic feedback for enhanced realism)
Reporting period: 2021-06-01 to 2022-11-30
Mass-casualty incidents (MCIs) with a large number of injured persons, human-made or caused by natural disasters, are increasing on the global level. In such situations, medical first responders (MFRs) need to perform triage, basic life support and other first aid procedures rapidly and under high stress. Training for MCIs is rare and very cost intensive.
The main aim of MED1stMR is to better prepare MFRs for such stressful and highly complex MCIs by developing a new generation of mixed reality (MR) training that combines haptic medical patient simulators with virtual environments and smart scenario control options including stress measurement and analysis during training. The MED1stMR training solution will strongly support and enhance the capabilities of MFRs for MCI preparedness and improve first responders’ resilience and their effective performance in medical emergencies in highly complex and unpredictable situations.
The four main objectives of MED1stMR are:
Obj. 1: Developing a pioneering MR training approach for enhanced realism
Obj. 2: Developing effective training scenarios and a training curriculum through agile and user-centred design with cross-sectoral MFR
Obj. 3: Realisation of a physiological signal and trainee behaviour feedback loop and smart scenario control to enhance the effectiveness of MR training
Obj. 4: To position the pioneering MR training approach across Europe
The main aim of MED1stMR is to better prepare MFRs for such stressful and highly complex MCIs by developing a new generation of mixed reality (MR) training that combines haptic medical patient simulators with virtual environments and smart scenario control options including stress measurement and analysis during training. The MED1stMR training solution will strongly support and enhance the capabilities of MFRs for MCI preparedness and improve first responders’ resilience and their effective performance in medical emergencies in highly complex and unpredictable situations.
The four main objectives of MED1stMR are:
Obj. 1: Developing a pioneering MR training approach for enhanced realism
Obj. 2: Developing effective training scenarios and a training curriculum through agile and user-centred design with cross-sectoral MFR
Obj. 3: Realisation of a physiological signal and trainee behaviour feedback loop and smart scenario control to enhance the effectiveness of MR training
Obj. 4: To position the pioneering MR training approach across Europe
The MED1stMR project started in June 2021 and has made great progress in the individual objectives described in the following.
Obj1. - In intensive cooperation with the end users, current forms of training were investigated, requirements for VR training were collected and concepts were developed with practitioners. To gather the requirements, a literature review and end user centred research through 5 interviews with medical first responder groups (n = 30), 4 observations of simulation training (n = 123), 5 co-creation workshops and focus groups (n = 41) were conducted. From these requirements, the technical requirements emerging from the overarching wants and needs of the end users were derived and prioritized for development. A technology framework was developed to incorporate the manikin in the MR training system. The first manikin for first triage training has been completed for integration and has tangible carotid, radial and femoral pulses and is able to change pulse frequency and strength. Additionally, to get early feedback on the technical needs of end-users in interaction with a tangible manikin and medical equipment, an experience prototype was built and tests with this prototype provided positive and valuable feedback for further development. An easy-to-use editor was developed for scenario creation that allows users to create and adapt the scenarios. Besides the editor, the exercise control developments were started, and a first design was created, which will be evaluated in further tests and field trials.
Obj. 2 - Based on the requirements collected from the workshops with end users and real training observations, consequences for the scenario guidelines were drawn. To operationalise these scenario guidelines, a template was developed. It contains considerations on learning objectives, scenario environment, event type and considers the functionalities and possibilities of the MR system. This template was used for a first MCI scenario dealing with a bus accident. This scenario can be extended with challenging environmental conditions, different medical and injury patterns and serves as a baseline scenario for further developments and contains the essential elements for training. Another important point is to advance the understanding of effective performance in medical emergencies (EPME). Therefore, a theoretical background was created to understand how MFRs act under stressful circumstances and how the model can be empirically validated.
Obj. 3 – To enhance the effectiveness of MR training a physiological signal and trainee behaviour feedback loop with body sensors will be integrated. A concept for the physiological measurement suite for stress assessment was established. Wearable body sensors to monitor the stress level of trainees have already been developed. The first prototype includes a single lead electrocardiography (ECG) sensor that acquires raw data from ECG, accelerometer and magnetometer data and will be further extended with an electrodermal activity (EDA) sensor.
Obj. 4 - Dissemination and communication are carried out by the MFR partners and especially by the Johanniter International as an umbrella organisation of national emergency responder charity organisations from 15 European countries, supporting networking, dissemination, awareness raising and exploitation of results across Europe. The project idea, results, training concepts and guidelines were published in scientific journals, presented at conferences, events and made available to MFR organisations, presented at CERIS events and to other H2020 projects in the Disaster Resilient Societies domain.
Obj1. - In intensive cooperation with the end users, current forms of training were investigated, requirements for VR training were collected and concepts were developed with practitioners. To gather the requirements, a literature review and end user centred research through 5 interviews with medical first responder groups (n = 30), 4 observations of simulation training (n = 123), 5 co-creation workshops and focus groups (n = 41) were conducted. From these requirements, the technical requirements emerging from the overarching wants and needs of the end users were derived and prioritized for development. A technology framework was developed to incorporate the manikin in the MR training system. The first manikin for first triage training has been completed for integration and has tangible carotid, radial and femoral pulses and is able to change pulse frequency and strength. Additionally, to get early feedback on the technical needs of end-users in interaction with a tangible manikin and medical equipment, an experience prototype was built and tests with this prototype provided positive and valuable feedback for further development. An easy-to-use editor was developed for scenario creation that allows users to create and adapt the scenarios. Besides the editor, the exercise control developments were started, and a first design was created, which will be evaluated in further tests and field trials.
Obj. 2 - Based on the requirements collected from the workshops with end users and real training observations, consequences for the scenario guidelines were drawn. To operationalise these scenario guidelines, a template was developed. It contains considerations on learning objectives, scenario environment, event type and considers the functionalities and possibilities of the MR system. This template was used for a first MCI scenario dealing with a bus accident. This scenario can be extended with challenging environmental conditions, different medical and injury patterns and serves as a baseline scenario for further developments and contains the essential elements for training. Another important point is to advance the understanding of effective performance in medical emergencies (EPME). Therefore, a theoretical background was created to understand how MFRs act under stressful circumstances and how the model can be empirically validated.
Obj. 3 – To enhance the effectiveness of MR training a physiological signal and trainee behaviour feedback loop with body sensors will be integrated. A concept for the physiological measurement suite for stress assessment was established. Wearable body sensors to monitor the stress level of trainees have already been developed. The first prototype includes a single lead electrocardiography (ECG) sensor that acquires raw data from ECG, accelerometer and magnetometer data and will be further extended with an electrodermal activity (EDA) sensor.
Obj. 4 - Dissemination and communication are carried out by the MFR partners and especially by the Johanniter International as an umbrella organisation of national emergency responder charity organisations from 15 European countries, supporting networking, dissemination, awareness raising and exploitation of results across Europe. The project idea, results, training concepts and guidelines were published in scientific journals, presented at conferences, events and made available to MFR organisations, presented at CERIS events and to other H2020 projects in the Disaster Resilient Societies domain.
For the effectiveness of MFRs, MR training of realistic scenarios will have a high impact. MED1stMR´s will develop a MR environment, that allows first responders to experience and train in highly realistic, reproducible situations with less effort and risk.
By integrating haptic high-fidelity patient simulation manikins and medical equipment into the MR, MED1stMR advances the state of the art and offers a much richer sensory experience compared to other VR training. Trainees will be able to feel vital signs and perceive limbs, head and face of their patient through tactile and visual interaction. This will bring virtual training closer to reality and enables systematic manipulation of a large set of influence factors in order to optimize training effects. The developed wearables allow the monitoring of the trainees’ stress level and behaviours in real-time and will provide an innovative approach to personalise and adapt training to their needs. Training sessions are recorded to deepen learning effects through reflection during and after execution with trainers. The research-based fundament built in executed studies extends the present scientific knowledge of effective performance in medical emergencies (EPME) and fosters personalising and adapting training to the trainees’ needs, manually by the trainer or automated by smart scenario control using an artificial intelligence approach.
The innovative training solution will allow faster deployment and result in direct economic benefits for the first responder organisations in the long-term. The training solution eliminates actors, travel and accommodation costs, consumables and allows multi-site training to make optimal use of the available budget. This will lead to an increase in performance and resilience of MFRs, thereby making the lives of European people safer.
Within MED1stMR, guidelines and frameworks for MR training of first responders will be created. These guidelines facilitate managerial and organisational decisions and have a strong added value in laying the foundation for the application of future MR first responder training. Above that, well-trained MFRs introduce positive impacts on the quality of medical teams within the European society and therefore contribute to the improvement of security and well-being of EU citizens.
By integrating haptic high-fidelity patient simulation manikins and medical equipment into the MR, MED1stMR advances the state of the art and offers a much richer sensory experience compared to other VR training. Trainees will be able to feel vital signs and perceive limbs, head and face of their patient through tactile and visual interaction. This will bring virtual training closer to reality and enables systematic manipulation of a large set of influence factors in order to optimize training effects. The developed wearables allow the monitoring of the trainees’ stress level and behaviours in real-time and will provide an innovative approach to personalise and adapt training to their needs. Training sessions are recorded to deepen learning effects through reflection during and after execution with trainers. The research-based fundament built in executed studies extends the present scientific knowledge of effective performance in medical emergencies (EPME) and fosters personalising and adapting training to the trainees’ needs, manually by the trainer or automated by smart scenario control using an artificial intelligence approach.
The innovative training solution will allow faster deployment and result in direct economic benefits for the first responder organisations in the long-term. The training solution eliminates actors, travel and accommodation costs, consumables and allows multi-site training to make optimal use of the available budget. This will lead to an increase in performance and resilience of MFRs, thereby making the lives of European people safer.
Within MED1stMR, guidelines and frameworks for MR training of first responders will be created. These guidelines facilitate managerial and organisational decisions and have a strong added value in laying the foundation for the application of future MR first responder training. Above that, well-trained MFRs introduce positive impacts on the quality of medical teams within the European society and therefore contribute to the improvement of security and well-being of EU citizens.