Final Report Summary - L4S (Learning 4 Security)
Executive Summary:
L4S project exploits the reality and the needs of organizations to help their managers and personnel in order to develop key skills and competencies in crisis management area through an innovative, highly involving, effective and easy deployable life-long learning service.
More precisely, L4S simulation-based learning experiences on Crisis Management reflect in a very realistic way crisis management and collaboration dynamics in modelled disastrous situations in air and sea transportations. Players are involved in role-playing crisis incidents in which they can validate and develop specific competencies through learning-by-doing in realistically modelled scenarios.
All L4S learning experiences’ applications are designed and developed in order to improve:
- the crisis management and collaboration competencies of European corporate personnel, decision-makers and academic learners;
- the crisis readiness of organizations, both in the private and public sector Europe-wide.
The L4S learning simulations address three forms of competencies with a specific regard to the challenges of crisis management at all different organizational levels mentioned above:
- Knowledge that focuses on cognitive abilities;
- Attitudes that focuses on affective and normative facets of competence, and
- Skills focus on the ability to perform action.
The L4S project successfully designed and developed the “L4S Learning Experiences service” comprised by advanced simulation games and learning / networking applications. The high-value learning service is built upon the L4S framework, an experience-based pedagogical framework that address the effective development of crisis management and collaboration competencies for corporate and academic learners.
L4S Learning Experiences Portfolio, includes the following simulation games on air and sea transport crisis management: i) “IMPACT: The Crisis Readiness Online Simulation Experience” a set of 3-leveled simulation games, ii) “RECKON&CHOOSE! Air Simulation” and iii) “CRISIS TEAM”. Apart from simulation games the portfolio contains the WEB 2.0 advanced networking and sharing tool “CRISIS TUBE Leadership Learning Network” as well as the supportive online workshop tool “OWL4S”.
L4S partners have verified their commitment to continue collaborating on L4S outcomes in areas such scientific research and publications on L4S multi-disciplinary domains, simulation games research and development and enhanced teaching and training in security and crisis management using simulation games.
The individual exploitation plans of the partners are formed in three distinct types of utilizing the L4S applications:
- Internally: organizations will provide the L4S Learning Experiences internally to its employees and executives, adopting the L4S applications portfolio in its internal executive training programs;
- Externally: commercial entities will distribute the L4S Learning Experiences as a service to its customers in various industries. Moreover, the Learning Experiences may be bundled with other existing business products or services;
- Academically: educational and academic institutions will integrate in their curricula the L4S training applications and they can also provide executive and vocational training.
From a business perspective we believe that both the diffusion and the financial targets set can be achieved, with value generated at all levels; from the personnel and decision makers experiencing the value of the proposed simulation-based learning sessions, to the facilitators, the organizations’ representatives, the network of companies diffusing the L4S Learning Experiences Europe-wide, the designers of advanced crisis management learning simulations and the academic staff and learners that will experience these learning environments.
A multi-level exploitation plan has been thus defined, and its execution is commencing straightforward after the project’s completion. The work done clearly indicates that the currently available and immediately deployable “L4S Simulation Games and Applications” can provide impact and visibility, along with the generation of a strong stream of revenue to support this growth. The longer-term strategy and vision is to set up an efficient Europe-wide B2B Channel for the diffusion of Game-based Learning Experiences.
Project Context and Objectives:
The aim of L4S project is to provide a clear understanding and to further develop existing know-how, in both interdisciplinary scientific/academic models and best/worst practices and experiences in the field of transportation. It focuses on the factors inhibiting effective collaboration dynamics in crises and leading to the failure of effective crisis management and of the interventions required to reduce these risks.
This know-how is integrated into an innovative framework for addressing the development of collaboration competencies of crisis managers in the transportation sector. This L4S Framework is validated through the implementation of state-of-the-art highly interactive and experiential learning solutions that enable the effective understanding and management of the challenges in crisis situations, as validated with the participation of practitioners in the field. These challenges include:
(1) acting under extreme time pressure
(2) facing the lack, ambiguity, and/or asymmetries of information
(3) dealing with human factors like cognitions, attitudes and emotions, and
(4) addressing the interpersonal relationship dimension like fast relationship building and activation for the mobilization of social resources, trust building, cohesion and role definitions and also handling diversity and conflicts.
The L4S objectives are to develop simulation based learning experiences and provide supporting guides and tools for developing the soft skills necessary in effective crisis management. These Learning Experiences are deployable for a variety of purposes, including:
- Training security managers and crisis managers in organization, particularly in the transportation sector
- Preparing and testing security/crisis readiness of organizations and of their employees
- Auditing the organizational preparedness for crises in security domains and beyond (i.e. financial crises)
- Preparing the broader public on appropriate behaviors and responses to security incidents
- Conducting research (e.g. experimental tests) on the impact of different responses to the reaction of the public or the effects of the security incident.
In terms of outputs, the project delivers significant contributions:
1. A comprehensive Online Knowledge Community Integrating a Knowledge Base and an active Virtual Learning Community on Advanced Collaboration Dynamics and Technologies of Leaning for Security (L4S).
The project contributes innovative interdisciplinary research in the area of collaboration dynamics for crisis management combined with best practices and experiences from the transportation industry. This know-how is structured and collected within a dynamic Knowledge Base accessible online and maintained up-to-date through contributions from the members of a Virtual Learning Community from both academia and transportation industry.
2. L4S Framework: An experience-based learning Framework that address the effective development of collaboration competencies for crisis managers
The project contributes an innovative framework (L4S framework) targeting the design of pedagogically effective simulation based learning experiences employing Advanced Organizational Simulation Games (based on computer-enhanced collaborative and experiential learning models and simulation games design principles). Broad application and sustainability of the L4S framework is guaranteed through its inclusion in the L4S learning section of the online L4S Knowledge Base and Virtual Learning Community, that extends access to professors and trainers on crisis management.
3. Four (4) L4S Simulation Games: Validated experienced based learning solutions deployable in educational and organizational contexts
The project contributes with the development of significantly more effective eLearning solutions (“L4S Simulation Games”). The eLearning solutions developed are validated in both academic and industrial contexts through a series of Workshops and are packaged for easy employability in companies and universities to enhance the efficiency and quality of learning and competency development in the critical area of collaborative dynamics for crisis management in the transportation sector. More specifically four simulation prototypes are being deployed based on the requirements provided by the industrial partners (air and sea transportation actors) participating in the consortium.
4. L4S Deployment Package: Effective instruments and tools for the generation of simulation based learning experiences
Finally, there exist different deployment modes for simulation based learning experiences, for example within or between organizations, among homogenous or diverse trainees, in a co-located or distributed setting. The L4S Deployment Package provides an integrated solution for the different type of simulation deployment models and includes as components: pedagogical guidelines and notes, practical deployment guidelines and scenarios, different types of simulation games, an online workshop tool, a monitoring and assessment tool, and access to the L4S knowledge community.
Project Results:
The scientific and technological methodology and associated work plan of L4S project evolved from a model for the design process of Small World Simulations developed by Angehrn (2006 ). The L4S project is advanced through a continuous interplay between technological and educational development, on field trials and observations, implementation and validation of simulation learning activities in different learning environments. Domain experts remain strongly involved in key roles throughout the process. The evaluation of the project’s impact is performed according to a concrete and methodological scheme which is developed in the framework of the project. A dissemination strategy was devised at the beginning of the project and has been performed in stages during the project’s duration. Additionally, an evaluation plan covering both technical as well as learning aspects ensures that the final deliverables are suitable for their purposes and can be naturally integrated in the different environments.
The implementation scheme of the project was planned to be developed across three main axes:
- collaboration challenges and related competencies & dynamics for crisis managers (WP1)
- knowledge processes and community building (WP2)
- advanced technological tools for simulation games (WP3).
The pathways document the full network of options and the choices actually explored within the project. They also document separate trends of development that may occur as a result of the different trial environments. More concretely, the objective of WP1 is to produce a comprehensive integrated collection of models and insights emerging from both scientific theory and practical experiences relevant to the human collaboration dimension in crisis situations as a foundation to be integrated in the L4S knowledge community (WP2), in the L4S framework and in the L4S simulation games (WP3).
The objective of WP2 is the design and development of two tools to support the simulation design and deployment: The L4S Knowledge community (KC) and the L4S Online Workshop Tool (OWL4S).
The objective of WP3 is to develop a first version of the L4S Framework for the design of effective simulation based learning experiences addressing collaboration dynamics and competencies development in crisis management, and to use it to drive the design and implementation of a first prototype of four L4S Simulation Games reflecting the knowledge gathered in WP1 and WP2.
All three pathways have produced at regular intervals analytical reports from the meetings and the workshops, observations and reports from the field trips, technical, pedagogical and evaluation reports produced during the development of the tools -four simulation games prototypes - and their components. These reports also compare with the state of the art in their respective areas. The interconnection amongst the three pathways is facilitated through the extended pilots that were realized during the second year of the project and strengthen the involvement of domains expertise.
During the first pilot round, a total of 10 different pilots has been carried out, involving 246 pilot users, while in the second piloting phase an additional set of 12 different pilots was conducted involving a total of additional 201 pilot users in which the L4S Prototypes and their components have been deployed. It is evident that a significant number of pilot trials were conducted, in a variety of corporate and academic environments, engaging a respective number of players (close to 500) and creating significant opportunities for the exploitation dimension of the L4S prototypes. Important pilot and commercial activities are already planned for the post-L4S phase, engaging hundreds of players. Furthermore, a lot of effort was carried out in the refinement and fine tuning of all applications, in order to deliver learning experiences of high value.
Concerning the identification of the best way of diffusing this type of high-value learning experiences Europe-wide, the consortium has conducted a market analysis, which produced detailed and relevant market information focusing primarily on the countries that partners are originating from.
The L4S consortium has now clear evidence that L4S Learning Experiences have a high potential for diffusion and impact through various exploitation streams, that offer to the L4S partners distinct but valuable high quality assets in their internal and/or external operations and to transportation and security stakeholders an innovative portfolio of advanced learning experiences on crisis management.
The following paragraphs present concisely the scientific and technological achievements of the L4S project, structured on the work plan of the project.
1.3.1 WP1: Knowledge Harvesting and Integration
The first milestone of the project was achieved through the preparation and completion of the report on the findings of the Work Package. In sum, Work Package 1 produced a comprehensive analysis of the relevant knowledge and important insights that emerged from the exploration of both the academic theories and the practical experiences. To provide the knowledge harvesting and integration according to the tasks description, a three-dimensional research framework has been developed and consequently applied in the research work in WP1 (see Figure I). This framework takes into account that all current and future challenges for an effective crisis management must be addressed at different, but interdependent levels – the individual/interpersonal level, the group level, as well as the organizational and inter organizational level. Moreover, since the main focus of the L4S project is effective learning, an important dimension of our research framework is related to learning objectives, especially to the competence dimension. This dimension differentiates three forms of competencies that are addressed by the L4S learning solutions – knowledge, attitudes/norms and skills. Within trainings and educational contexts, this taxonomy is widely applied and can be thought of as "the objectives of the training process" in the L4S learning context. This focus on competences is in also line with the agenda of the European Union on security (European Security Research Advisory Board, 2006). Finally, the L4S research framework puts emphasis on crisis management in the transportation sector. Accordingly, as a third dimension of the research framework we applied the transportation dimension that includes primarily the "air" and "sea" transportation. The L4S research framework is depicted in Figure 1: L4S Research Framework
(see attached document).
WP1 provided a clear understanding anchored in both interdisciplinary scientific/academic models and best/worst practices and experiences of the factors inhibiting effective collaboration dynamics and leading to the failure of collaboration initiatives in crisis management (“Traps & Challenges”), and of the interventions required to reduce these risks (“Crisis-Related Collaboration Management Competencies”). The results of this report contributed to an innovative framework (ACDT Framework) targeting the design of effective technology-enhanced learning solutions that are built on Advanced Simulation Games (based on computer-enhanced collaborative and experiential learning models and simulation games design principles) in the WP3.
Based on its research framework, the Work Package 1 was able to meet the following scientific goals that reflect the most significant results of this Work Package:
- Identification of the most essential causes and forms of crises, emergencies and security-related incidents;
- Detection of the most critical problems and challenges that the crisis and security managers have to master in crisis situations with a particular emphasis to the transportation sector;
- Exploration of knowledge, attitudes as well as social and cognitive skills that the crisis managers have to possess to be able to successfully meet the crisis challenges in the transportation sector;
- Discovering what are the key biases and the most critical misbehaviors that negatively affect the decision-making and communication by managers in crisis situations;
- Uncovering main social mechanisms of collaboration in crisis management teams (CMT);
- Analysis of what are the main phases of the organizational crisis management process and which phase-specific management activities determine the crisis preparedness of an organization;
- Finding out which processes govern the inter-organizational collaboration in crisis situations and the factors that affect the effectiveness and efficiency of these collaboration processes;
- Localization of critical practice-driven requirements to the learning methods, technologies and tools that have not been appropriately met by the existing approaches to the training of crisis managers in the sea and air transportation sector
- A critical appraisal of the state-of-the-art learning technologies that are in use for crisis management training and the identification of requirements that have to be addressed by the L4S learning experiences;
- Development of learning objectives for the L4S learning solutions that have to be addressed to increase the efficiency and effectiveness of crisis management training with particular emphasis on the transportation sector.
1.3.2 WP2: Knowledge Community
A significant result produced is the setting up of the Knowledge Community. This milestone summarizes a varied set of actions such as the L4S Knowledge Community design, implementation, and finally the launch process taking into account the results produced from the corresponding task and deliverable and mainly based on the potential applications of the Learning community platform.
The initial purpose of the L4S Knowledge Community is to support/stimulate collaboration towards the final objective of this project. The long-term scope of the L4S Knowledge Community is to provide the researchers, crisis managers and simulation designers with a comprehensive knowledge repository and a community space that enable them to prepare a learning experience that effectively exploits the simulation tools in a way that meets their training needs.
The L4S Knowledge Community (KC) is composed of two main domains, the Knowledge Base and the Virtual Learning Community. Practically the two areas are fully hyperlinked and most of the tools are tightly integrated. The platform uses the latest Web 2.0 features. The following figure (Figure 2: KC interface) depicts the KC’s interface (see attached document).
Community has incorporated all the knowledge assets gathered in WP1, as well as additional ones that were and will be progressively inserted in the Knowledge Base throughout the project and beyond the project timeline. The Knowledge Community plays an important role supporting and encouraging the knowledge exchanges and the collaborative work of the consortium members and their networks toward the iterative design of the simulation-based learning experiences. During the duration of the L4S project the platform has been used by the consortium members for internal communication, knowledge sharing and classification, and support for the daily work. The KC restricted area supports the project management, encouraging the consortium daily exchanges and documents sharing.
Furthermore, some features of the KC are used to manage collaboratively the content of the L4S project public site (www.l4s-project.info) and the ISCM website (www.ISCM2011.eu).
From a wider prospective, the L4S Knowledge Community provides researchers, security managers and staff, crisis management experts and trainers with a comprehensive knowledge repository and a community space focusing on the challenges of security and crisis management in the transportation sector.
During the pilot rounds the Knowledge Community has been used to support trainers and facilitators with content, case studies and scenarios to be used for the design of effective learning experiences and training sessions.
In the other hand, the workshop participants and learners have in their disposal though the KC content, manuals and additional educational and academic material to enrich the learning experience.
In parallel, during and after the project lifetime, the KC is a tool, together with other dissemination channels, of promoting the project’s outcomes and to diffuse the research results to the international academic and research community.
Several types of users are targeted for using the knowledge community, for different purposes. These target groups are categorized as:
Academic user, researcher. This user could be interested in using the KC to retrieve relevant contents on the issue of security and crisis management that could be useful for publication aims. On one side, the KC is an important repository of knowledge and theories on security and crisis management. The KC can also support the academic user in discussing and sharing ideas and insights with colleagues. The academic user can be a facilitator or a simulation designer.
Decision makers in transportation organizations. This category of users can be considered as potential buyer/client of the L4S learning solutions. They can access knowledge assets, information and marketing material and also access videos, leaflet and promotional material of the project outputs.
Simulation developer/designer. This user could be interested in using the contents, i.e. theories and models, retrieved into the KC to build on these theoretical bases a simulation scenario he intends to create. The KC could then represent a good starting point to address those knowledge harvesting activities that represent the first step of a simulation development.
Facilitator/Teacher (academic or corporate sector). This user accesses the KC and uses its contents to structure a simulation-based workshop experience. This is a category of users that is supposed to do an intensive uptake of the KC.
Learner/student/participant to the L4S learning experiences. This user can find interesting material for his learning interests. Before and after the L4S learning experiences, the learners can access the KC to download material, open discussions, and participate to discussions.
The L4S Online Workshop Tool (OWL4S) is also developed during WP2 to support both the collaboration within the Knowledge Community and the simulation based learning experiences. The Online Workshop Tool is an enhanced environment designed to support L4S learning experiences. Its main interface is presented in
Figure 3: OWL4S participant’s interface (repository area)
(see attached document). Based on requirements and suggestions coming from some of the most relevant players in the Simulation Games learning providers and Industrial partners in the specific domains of security and transportation (L4S partners, their networks and the external experts involved in the Scientific Committee), the OWL4S has been designed including functionalities and features like:
- Flexibility and customization of all the elements (interface, contents, structure)
- Possibility to integrate any media (video, web pages, on-line applications) into the OWL4S pages
- Possibility to inject new real-time elements in the game (like an unexpected event) and add elements (like a brainstorming), while the participants are playing.
- Different ways to be delivered (co-located participants/distributed on-line, synchronous/asynchronous, individual/group, autonomously/with a facilitator)
- Different options of usage and application fields like:
o support brainstorming
o submit questionnaires
o voting pool
o presentation area
o repository to download pedagogical material
These functionalities and features allow the usage of this application as:
- Tool to support and integrate the different phases and elements of a learning experience
o providing documentation before the workshop to the learners
o supporting the briefing phase, integrating presentations with different media type
o encouraging the collaboration and ideas sharing, thanks to the brainstorming feature
o supporting the debriefing phase, integrating presentations with different media type
o collecting feedback and questionnaires
o distributing informative material after the workshop
o supporting a post workshop discussion and ideas sharing, thanks to the brainstorming feature
- Environment to design micro games (to focus on specific pedagogical dimensions)
- Dilemma-based simulation game, integrating videos, presentations and dilemmas in a pre-defined flow.
During the first year of the L4S project, the OWL4S has been used as a working tool for brainstorming and to collect ideas on the games design within the L4S consortium members. This usage has been very important in order to make improvements and changes on the interface and structure of the tool.
During the pilot rounds the OWL4S has been used as part of the L4S learning experience:
- To support phases of the learning experience (i.e. during briefing and debriefing phases, as brainstorming environment, as collaborative decision making support, etc.)
- Environment integrating all the elements of a simulation games learning experience (from the briefing phase to the game, from the interactive moments to the debriefing, from the questionnaires to the distribution of additional material to the participants)
- Flexible Space to design micro games (i.e. for the Reckon & Choose game to focus on specific pedagogical dimensions, so called “zoom in”)
- To expand the simulation possibilities, thanks to the exchange of data from the game to the OWL4S (like in the Reckon & Choose! simulation game).
After the L4S project conclusion the OWL4S will be exploited and promoted initially through the partners’ networks and clients, and, at a second stage proposed to the market, as part of the L4S learning package, following the exploitation plan.
The OWL4S has three main target users:
Learning experiences developer/designer: The Learning experiences developer/designer is the player who design the learning experience based on the client’s learning requests/needs. This user can also play the facilitator’s role or provide the learning experience to facilitators/teachers to be delivered to learners.
This user will find in the OWL4S an interesting platform, fully customizable (from the interface to structure and contents), to be used as an environment to design and deliver learning experiences. The OWL4S can also be used as part of an existing learning package, to support specific needs (i.e. brainstorming, voting, questionnaire submission).
Facilitator/Teacher (academic or corporate sector). The facilitator is the player who delivers the learning experience to the learners. He is the principal target user for the OWL4S because the system empowers him with several interesting features to better deliver the learning contents. The Facilitator can easily customize any content of the OWL4S depending on his needs.
Learner/student/participant to the L4S learning experiences. This user is the beneficiary of the learning experience. He will find in the OWL4S an interesting environment to better enjoy the leaning contents (i.e. documents, presentations with different media type) and interact with the teacher and the other learners.
And an indirect target user:
Client (after the L4S project conclusion). The client is an indirect target user of the OWL4S because he is not expected to use the platform. The client is the firm or institution that requires (and usually pays for) the learning package for his employees. The client expresses the learning goals to the facilitator/trainer in order to allow him to design a learning experience.
1.3.3 WP3: L4S Framework and Simulation Prototypes
Building on the information gathered from the industrial partners and referring to the management-related educational needs elicited in WP1, the consortium worked on the L4S Simulation Games Prototypes’ design and specification of scenarios, implemented by L4S sub-working teams addressing either the Maritime or the Air Transportation sector (“Sea Team” and “Air Team”). The insights gained in the extensive study of crisis management-related knowledge sources (summarized in the D1.1) has enabled the consortium to identify a number of factors determining the success or failure of crisis management in organizational and inter-organizational contexts, as well as a number of levels (individuals, teams/groups, organizations, inter-organizational entities) at which crisis management (or crisis management breakdowns) can be observed – and potentially influenced. In the WP3 deliverable on “L4S Framework, Simulation Scenarios and Design” (D3.1) the Design Framework is analyzed, i.e. the steps which led gradually to the design, specification and initial implementation of a first set of four prototypes of L4S Simulation-based Learning Experiences. In doing so, the attempt to deploy state-of-the-art pedagogical models and learning technologies (from multimedia to Web 2.0) was carried out. A first important design step was the way to structure the knowledge gathering process, conducted in the first phase of the project, through the integration of a variety of sources addressed from multiple perspectives. The second key step consisted in identifying three different L4S Simulation Levels according to the specific crisis management -related dynamics and breakdowns they focus on, namely:
(a) Organizational and Inter-Organizational Dynamics: Such Simulations and Learning Experiences focus particularly on factors determining crisis management dynamics in different organizational and inter-organizational contexts/cultures, their productive or dysfunctional manifestations in everyday organizational life, and the possibility to influence such dynamics through managerial action.
(b) Group/Team Dynamics: Such Simulations focus particularly on factors determining success or failure of crisis management groups or teams, their manifestation in the context of specific situations, and the possibility to influence such group dynamics in organizational and inter-organizational contexts, as well as in real-time or asynchronous crisis management situations.
(c) Individual/Interpersonal Dynamics: Such Simulations focus particularly on factors determining both motivation and capability to manage crisis situations at the individual level, their manifestation in inter-personal exchanges/crisis management contexts, and the possibility to influence them.
For each prototype, in the corresponding deliverable (D3.1) is presented the Pedagogical Foundations and Deployment Approach, the Underlying Models, as well as how its Implementation started and evolved. All prototypes were designed in a way to support their rapid development and incremental improvement, as well as the validation and deployment of the L4S Prototypes in the pilot settings made available by the Consortium partners.
The L4S prototypes designed and developed as well as their positioning matrix in relation with the organizational level that they address are depicted in the following figure. Apart from the designing and educational dimensions, this positioning will assist the effective promotion of the applications to the market as well as to cover the distinct needs of the various target segments and organizations Europe wide. For each prototype one-page summary is provided in the following pages. Figure 4: Positioning Matrix of the L4S Prototypes (see attached document).
Impact: Crisis Readiness Simulation
“IMPACT” is the name of the first of the Simulation Prototypes developed by the Sea Team, aimed to target key managers and decision makers in the Maritime Transportation sector EU-wide. The IMPACT prototype provides, through a set of connected, video- and simulation-based, online collaborative learning experiences, deployable both on-line and on-site or through a combination of both, an engaging experience based on a complex, multi-stage and multi-focus simulation game in which the users/players experience the challenges of crisis management at the individual, team and organizational level, and hence gain insights on how to best address these challenges in their own contexts. The game is designed to be played flexibly on-site (traditional workshop of 2 days) or online over a period of several weeks (from 4 to 9) and is articulated at 3 Levels, each of which addresses relevant competences and insights related to Crisis Management.
At Level 1 the players – operating in teams – are involved (through a first set of interactive video episodes) in 3 crisis situations that they will have to address by collaboration in distributed and diverse teams. The storyline features a company needing to address complex decisions in situations requiring immediate reaction through team discussions in order to identify the best way to react to the presented crisis situations and take difficult decisions that determine how the story unfolds and either succeed or fail in productively resolving the crisis situations.
At Level 2 the focus shifts from the team to the organizational perspective of effective crisis management. The level evolves though three basic steps. In the first step (“Maritime Transportation Crisis Analysis”), the teams review a video which describes a problem caused by a major accident due to a storm in the future (2016) and identify the key characteristics of the situation at hand as well as real cases and examples of similar real crisis situations and produce a first set of key managerial decisions to be taken to face the crisis situation and avoid its escalation. In the second step (“Organizational Crisis Immediate Reaction Analysis”) the teams are invited to review a video of how the simulated organization reacts to the crisis event through the activation of a number of key individuals distributed across Europe and their reaction to the crisis event. The team must compare the “ideal” behavior to the “actual” one that would take place in their organizations, and identify the top 3 Organizational Enablers of such behaviors (ideal and actual). In the third step (“Organizational Crisis Resolution Analysis”) the storyline underlying the simulation describes in detail how the simulated organizations succeed in productively resolving the crisis situation through the smooth collaboration of the key individuals involved in spite of their different roles and position within the organization. This part of the simulation provides again an “ideal” reference point for comparison of crisis resolution pattern featured in the simulation with the current reality of their organizations, collaboration and coordination analysis of the key players and which key competences and organizational enablers are involved.
Level 3 involves a more personal dimension through an analysis of the learners own, personal Crisis Readiness. This last phase aims at making learners aware of the leadership characteristics necessary to productively address crisis situations, and is directly linked to the current literature on leadership behavior by creating an intensive (stressful) individual online crisis management experience in which learners are exposed real-time to a continuous flow of information sources and requests for decisions and actions, facing them real-time to a set of overlapping complex crisis situations involving several people and issues which develop over time as a function of what the learner decides or not to do. The analysis of this intensive experience is then debriefed (in a traditional session or directly online, through a set of debriefing videos) based on the analysis of the learners decisions and behavior during the simulation and the data captured by the simulation real-time, providing a number of direct and personalized insights related to how to best address and improve the Crisis Readiness of the individual players.
Figure 5: Overview of User Interface at Level 3 of the IMPACT Simulation
(see attached document).
Reckon & Choose! – Tactical Emergency Collaboration
Reckon & Choose! simulation experience reflects the recognition of the particular situation of the current European landscape that is characterised by the lack of a consolidated body of knowledge and best practices regarding this coordinating tactical level in the air transportation industry. Today, there is high heterogeneity of approaches across nations, with additional legal and regional variations. Analysis of current performance in larger-scale emergencies exposes a general consensus that there is room for significant improvement and a specific need for standardisation.
The Reckon & Choose prototype was conceived to contribute to meeting this need by not only providing a learning environment for individual trainees, but by furthermore effectively serving as a knowledge elicitation and collection tool that gathers explicit and tacit knowledge from expert players and facilitators, producing a knowledge base that grows with individual deployments.
Throughout the game small teams of players interact with a model of reality, i.e. an abstracted excerpt that however preserves and concentrates key characteristics such as time pressure, partial information and dynamic (and not necessarily coherent, i.e. possibly contradicting or out-of-order) information flow. The players would not themselves be the decision makers in the simulated world, but rather be placed in observers roles. Setting out from the starting event of an airplane crashing into an airport building, they trace and follow one out of a set of time courses already taken. This trace is defined incrementally through specific choice points, at which the players are offered a very limited amount of different options (typically only two) to choose from: Each of these choice points should present a realistic, interesting, and relevant dilemma, selected out of a range of challenges derived from the Athens Airport Case document and informed by the feedback and complementary information provided by the Air Team partners.
In their answers, trainees’ are requested to specify explicitly the reasons (pros and cons) of each decision option as they see it and how they aggregated these pieces of information into their final decision for this challenge. Each of the possible paths from an opening scene to an ending scene would comply with the requirements described previously, e.g. of a coherent dramatic arc connecting the choice points to the first and the last scenes, and the decision options being offered in terms of outcomes, and not operational procedures.
According to this approach, trainees are expected to follow up on decisions made previously by tracking the pros and cons of those decisions and trying to optimise the path taken in the subsequent decisions– cf. the relevance of coherent behaviour and leadership. Generally, it is the utmost urgent and critical task for all leaders to communicate and make clear what their decision is, and which pros and cons they are seeing, in order to make the whole team follow this track of the decision taken — no matter whether some other decision would have been possible.
Along this line, the fact that trainees are forced to take a decision out of a finite set of options can be defended on the grounds that in facing a crisis there are no bad decisions, but rather it is important to try to maximise the outcome of any given path taken. Also, the choice dilemma formulations and the options offered are not to be selected based on considerations of actual best and worst practice, but should rather be defined along didactical criteria, such as e.g. making it difficult to come to a decision based on clear-cut arguments or a clear difference in overall assessment value.
Figure 6: The main display of the Reckon & Choose! Prototype, structured into the “view mode” (right hand side) and “action mode” (left hand side) (see attached document).
Crisis Team
The Crisis Team simulation is primarily targeted to non-experts in airport crisis management and aims to provide an experiential collaborative context through which they can engage with the subject matter. The learning goal is two-fold; first to engage participants with essential actions and the roles of key actors in crisis response and second to immerse participants in the distressed work environment of crisis situations where communication is patchy, the environment is volatile and there are multiple conflicting demands in compressed time.
In this simulation a team of players is asked to work with other teams on a particular airport crisis scenario for figuring out which actions to take and in what order. This approach allows for the emergence of collaborative behaviors and breakdowns originating actually from the players themselves. Moreover it forces participants to debate the content, objectives, resources and relative significance of different crisis response actions. There are no virtual characters; the players themselves are immersed in the collaborative experience with team dynamics and conflicts emerging out of this collaborative experience. Crisis Team is a short term learning experience of several hours. The online platform serves as the medium through which teams communicate and is designed to constrain communication possibilities so as to induce the difficulties and breakdowns that realistically arise in stressful collaboration contexts. The facilitator controls communication facilities, the time pressure and extraordinary events.
The Crisis Team scenario involves six teams of players each representing a different actor of the crisis management team of a major international. Whereas team members are collocated, teams are assumed to be distributed, communicating via the online Collaboration Platform which is design to induce artificial, yet realistic, stress and breakdowns. The mission is the operational implementation of the airport’s crisis management plan with the specific goal of selecting the most appropriate actions from a list of potential actions and of prioritizing them. Together the teams have to determine the actions to be implemented as well as the order in which to implement them. The collaboration platform is designed in a way that enables the players to experience how an initially collaborative environment can disintegrate into a competitive one in which teams begin to display non collaborative behaviors such as blame, accusation, frustration and because of the absence of face-to-face interactions, negative misattributions to the other teams’ choices, as well as refocusing of resources on competition rather than looking for reachable win-win solutions or a healthy compromise.
The simulation process unfolds in two main phases. Initially the actor is pre-assigned three actions representatives of the priority agenda of that actor. In the first phase, and after each team of participants has taken up an actor identity, they nominate their three actions, one at a time, each team in turn. Thus all teams have the opportunity to observe and to contemplate their own agenda and the actions nominated by other teams. In the second phase, teams take turns reassigning the priorities of all actions (nominated by all teams). An essential feature of the simulation is that teams are given mission objectives which are slightly differentiated and conflicting in the minor details. This recreates the diversity and knowledge integration challenge in which teams experience the difficulty of working with other teams that operate under different mindset(s), criteria and agendas.
By virtue of the way it is set up the Crisis Team simulation seeks to immerse players in the difficulties of collaborating in diverse teams that work under pressure. A second challenging feature of the simulation is the emergence of group and team breakdowns that can occur during the collaborative process. Through the Crisis Team simulation, players will experience how misunderstandings and miscommunications arise due to technological limitations and the absence of face-to-face interactions, mismatches between different objectives or agendas, the difficulties of knowledge integration in diverse teams, and other non-collaborative behaviors that can emerge in these situations. Another characteristic of the Crisis Team simulation is the opportunity to manage external, unplanned pressures and events that can occur during the collaborative process (and executed through the intervention agent). While teams might have established a common understanding and objective and may be working in a positive direction toward the accomplishment of their goals, external events might occur (as often do in reality), which might disrupt the momentum, pace and focus of collaboration. These events include external pressures such as political intervention, new related or unrelated emergency incidents, failure in resource mobilization, and others. Such events are under the discretionary control of the training facilitator.
In the standard setting, the collaboration platform allows extremely restricted communication, namely each team only sees the outcome of every other team’s decisions. There are no means of deliberation. In addition the facilitator controls whether teams can have access to a chat facility or not. Thus different collaboration phenomena can be produced. In addition, the facilitator controls an optional timer which can put even more time pressure to participants.
Figure 7: Operating Team’s Interface in CRISIS TEAM simulation
(see attached document).
CRISISTUBE Leadership Network
“CRISISTUBE” targets top managers and decision makers in the Maritime Transportation sector EU-wide. It provides collaborative inter-organizational Knowledge Exchange and development of enhanced quality standards & learning approaches related to Crisis Management through state-of-the-art Web 2.0 based Learning Networks technology to support effective exchange of Best Practices & experiences.
CRISISTUBE prototype provides an opportunity for key players addressing crisis management in the Maritime Transportation sector to join an active learning community operating continuously and over time, accessing information about other players (Networking Dimension), engaging in relevant learning-oriented exchanges related to crisis management fueled continuously by experts (Learning through the discussion of Cases) or by the users themselves (user-generated exchanges with sharing and discussion of experiences/best or worst practices). After having accessed the website through its Authentication Component users will typically see the overall interface of CRISISTUBE and start navigating through the Knowledge Network.
The Knowledge Network underlying CRISISTUBE is a highly dynamic environment in which Knowledge Assets of different types (videos, documents of different types, links, but also comments and discussions related to existing Knowledge Assets) can be browsed through, searched, as well as added flexibly
The Knowledge network is based upon three concepts:
- Challenges: users (typically experts) can describe Problem Areas or desirable Targets which are critical when it comes to effective crisis management in the Maritime Transportation sector and can be best addressed through inter-organizational collaboration.
- Experiences: all the users can describe their own experiences, particularly those related to facing one or more of the Challenges. Experiences can be both positive and negative, but are a means to contribute in addressing an existing Challenge, or to identify a new one. Experiences can also be directly connected to other Experiences, supporting best/worst practice sharing in critical areas.
- Ideas: all the users can describe their own ideas for inter-organizational collaboration, particularly those related to facing one or more of the Challenges. Ideas can also be directly connected to other Ideas, i.e. build on each other over time, supporting the collaborative identification of projects to be launched at the inter-organizational level.
Figure 8: Operating the Networks Visualization Pane in CRISISTUBE
(see attached document).
The L4S consortium during the 2nd period of L4S finalised the work carried on in the WP3 by actually further developing the simulation games and applications in order to make them effectively usable in the pilots by the L4S group and, more in general, by the L4S community first, and by all interested crisis-related communities eventually. Therefore, all implementations have been installed on the servers and made available to the pilots.
The tasks accomplished were the development and implementation of the server back-end and the implementation of the rich client front-end (regarding all of communication with the server, realisation of client-side game logic, and user interface/interaction management). Effort was put on creating various workshop formats, catering for both varying lengths and varying numbers of workshop participants to meet the variety of implementation possibilities the complex, multi-stage and multi-focus IMPACT simulation game can assume. This includes implementation testing of on-line and on-site collaborative learning experiences as well as a combination of both approaches. The implementation of the workshop format also concerns the design and implementation of interface formats, i.e. incorporation of briefing and debriefing aspects, linking results and workshop outcomes between all three levels of the IMPACT learning simulation to ensure a smooth learning transition for participants and aiding the sophistication of the simulation game end-product. For Reckon&Choose! the “air-team” worked on the facilitator console main contributions were made to the storyline. For the same game, partners collaborated regarding the media assets and technological solutions for user interface components, development of the “Country book” and of debriefing materials.
In addition, air and sea team interactively provided to each other content-related and technical feedback. All these research and development (RTD) efforts were interleaved with the WP4-related demonstration activities. Pilot partners provided detailed feedback and suggestions regarding the structure and content of the briefing and debriefing sessions of the learning experience by making best-practice transfer.
To sum up, all prototypes were successfully designed and developed in a way to support their rapid development and incremental improvement, as well as the validation and deployment of the L4S Prototypes in the pilot settings made available by the Consortium partners, as planned in the Work Package 4.
1.3.4 WP4: Pilots and Prototyping Cycles
Work Package 4 was dedicated to the running of Pilots and the progressive fine-tuning of the L4S Framework, the L4S Simulation Games (from WP3), the underlying L4S Knowledge Community (from WP2) and the Online Workshop tool, to progressively converge through cycles of evaluation-redesign-implementation towards final versions of these outputs.
1st Pilot Round August 2010 – January 2011
During the first pilot round, a total of 10 different pilots has been organized and carried out, involving a total of 246 pilot users, in which the four L4S Prototypes and their components have been deployed.
Table 1: Overview Table of Pilots conducted in the first round
L4S Prototype (see attached document)
With the help of all the Consortium members we were able to conduct the pilots during this phase already with participants belonging to different target groups (and not only with participants from the consortium members involved in the need analysis and the design and development of the L4S Prototypes, as initially planned. The advanced stage of the prototypes and their attractiveness for the target groups made it possible to organize sessions in which we were able to interact with university students (to test the deployment in educational settings) as well as with professional managers (to test the deployment in corporate training and life-long learning/executive development programs).
The pilots conducted have enabled us to gain new insights and to move one significant step forward in validating that the L4S Prototypes we have designed and developed during WP3:
- match the expectations of real users in terms of usability and learning value
- are built on software and media components which work as expected also in real deployment contexts (and not just under “lab” conditions)
- can be effectively deployed by other educators/trainers based on the pedagogical/ deployment guidelines provided
- are effective with a variety of audiences/target groups
Furthermore, a number of feedback collection methods (from questionnaires over brainstorming to focus groups) were experimented with and generated a variety of improvements which have emerged from the pilots. These insights were reviewed, integrated and implemented to produce new versions of the L4S Prototypes themselves, as well as of the related pedagogical material (teaching and deployment guides).
Overall, based on the experiences gathered during this first pilot round, the four L4S Prototypes appeared to meet well the expectations we set in our initial design document (D3.1) in particular in terms of:
- attractiveness (willingness-to-try)
- deployability (from technical and organizational points of view)
- innovativeness (compared to other approaches to address Crisis Mgmt)
- learning value (in terms of addressing effectively relevant competences)
This has been recognized in different contexts (from traditional, on-site workshops to online deployment contexts) by the very challenging audiences we were able to involve in the ten Pilots.
To conclude with the first pilot round of L4S, the pilots conducted have enabled us to gain new insights and to move one significant step forward in:
- Improving the reliability and quality of the software and media components
- Improving the debriefing approach (instructions for facilitators)
- Exploring how the L4S Prototypes can be “packaged” in order to fit the needs (in terms of crisis management competences) and be widely deployed with an even broader Target Group, namely top managers and decision makers in the Transportation Sector, but also in other relevant sectors like Energy, Industry or Services (like IT).
2nd Pilot Round February 2011 – June 2011
In the second piloting phase we have been able to organize an additional set of 12 different Pilots involving a total of additional 201 pilot users – with 850 additional committed users to conduct further pilots during Fall 2011 - in which the L4S Prototypes and their components have been deployed.
Table 2: Overview Table of Pilots conducted in the second round
L4S Prototype (see attached document)
Like in round one, we were able to conduct the Pilots during this phase already with participants belonging to different Target Groups. The advanced stage of the Prototypes and their attractiveness for the target groups made it possible to organize sessions in which we were able to interact with university students as well as with professional managers attending company training courses or Management School programs.
Through this second round of Pilots we have confirmed further that the L4S Prototypes:
- match the expectations of real users in terms of usability and learning value
- are built on software and media components which work as expected also in real deployment contexts (and not just under “lab” conditions)
- can be effectively deployed by other educators/trainers based on the pedagogical/ deployment guidelines provided
- are effective with a variety of audiences/target groups
The additional insights were reviewed, integrated and implemented to produce new versions of the L4S Prototypes themselves, as well as of the related pedagogical material (teaching and deployment guides). Effort was focused on thorough testing, documentation, third-party deployment, packaging, marketing, and production in order to prepare L4S prototypes becoming ready for deployment beyond the L4S project’s consortium members and pilot organizations
During the second pilot round, the Knowledge Community was also used mainly to support the consortium collaborative work and to enrich the dissemination actions to the target groups. The L4S facilitators used it as a knowledge repository to prepare their workshops and offered an additional knowledge repository environment to the participants. Following user’s reports, some technical bugs have been fixed, and some minor functionality has been improved. After the end of the L4S project the platform will be used in two ways:
- As a free Knowledge Community open to researchers, simulation games designers, facilitators and organizations interested in sharing contents and knowledge about crisis management in the transportation sector and beyond. The subscribed members beyond the L4S consortium will be using it as a knowledge repository and knowledge management tool.
- As an open platform to support collaborative knowledge sharing in different domains (lately is used to support the co-financed project Help4Mood – ICT-2009.5.1 – 248765)
During the ICT proposer’s day in Budapest (19 and 20 May 2011), the L4S Knowledge Community was presented to potential users. The environment has raised a good interest, mainly for enhanced learning applications.
For the second pilot round the OWL4S application was used as part of the L4S simulation-based learning experiences and prototypes. The following table lists the sessions that were supported by the OWL4S.
Table 3: Overview Table of Pilots supported by the OWL4S(see attached document)
Discussing with trainers and facilitators, some ideas on how to use the OWL4S in alternative domains and applications are the following:
- The OWL can be also used as a game environment to address specific learning goals and micro-scenarios. This can be done using a video and a related brainstorming session.
- It can also be used to fast draft and test a scenario before implementing a complex simulation game based on it.
- The OWL4S can be used to support zoom-in (or micro-games) for complementing another learning experience (this solution has been tested successfully with Reckon&Choose!).
As a conclusion for the two pilot rounds of L4S project, it is evident that a significant number of pilot trials was conducted, in a variety of corporate and academic environments, engaging a respective number of players (close to 500) and creating significant opportunities for the exploitation dimension of the L4S prototypes. Furthermore, a lot of effort was carried out in the refinement and fine tuning of all applications, in order to deliver learning experiences of high value.
1.3.5 WP6: Embedded, Continuous Evaluation
The objective of work package 6 was to generate through the evaluation process insights related to the technical features and educational, pedagogical and social values of a number of simulations that focused on assessing the extent to which it is possible to “learn”, (or increase ability) to cope with crisis situations using IT. As such, the project outputs have been evaluated from technical, technological and pedagogical points of view. The former aims to understand whether the knowledge community, the online workshop tool, and the simulation games work, are innovative and have the expected functionalities; the latter concentrates on the effects hat these tools have on learning processes, derived from the perceived value of the proposed solutions from users’ perspectives.
During the first 7 months of the project, the consortium has been working on the definition of the evaluation plan. The evaluation plan was dedicated to the continuous assessment of the outputs and applications developed throughout the project and to the documentation of the evaluation methodology and tools. More specifically, the objectives of the evaluation methodology adopted were:
- To assess the effectiveness of the technical solutions.
- To support the understanding of the user’s and market’s needs as well as the relevant functionalities provided by the technical solutions.
- To evaluate the strategic concept, learning value and potentialities of exploitation of L4S outputs.
The aim of the evaluation process was to:
- Develop guidelines for the development of the L4S simulation learning games,
- Provide guidelines and useful input from the users for improving the usability of the final prototypes and
- Investigate the impact that L4S simulation game based learning technology and the overall L4S learning experience has on end users.
Within the Evaluation Plan the need for both the technical/technological assessment and pedagogical evaluation of the L4S outputs has been addressed and justified by relevant frameworks mostly used in real life and diffused in literature. As for the technical evaluation, the L4S outputs were evaluated adopting the model from the ISO 9126 framework documented and unpacked by Chua and Dyson (2004).
As far as the pedagogical evaluation is concerned, the evaluation framework was based on Donald Kirkpatrick's Four Level Evaluation Model. According to the contribution of Kirkpatrick, each training session should be assessed at the following four-levels of evaluation:
- Reaction: how the learners react to the training experience.
- Learning: the extent to which the learners gain new knowledge, change attitudes and improve their skills.
- Behavior: the capability to perform the learnt skills while back on the job/working environment.
- Results: over the long-term, how the organization has improved its monetary results, efficiency, moral, etc. as a result of attending a specific training.
For each of the four levels, the relevance and practicability has been identified as well as the most appropriate assessment procedure in terms of both sample of analysis and tool of analysis.
Technical and Pedagogical Evaluation Criteria and Metrics report has presented the details of each framework and the specific questions to be addressed in each phase of the evaluation plan. The specific report was dedicated to the detailed investigation of the criteria and metrics that were used during the assessment analysis. While the technical perspective refers to the assessment of the technical and technological features of the outputs (technological quality of the tools in terms of usability and functionality), the pedagogical perspective analyses the impact of the tools on learning processes and dynamics identified by all partners using a number of learning objectives.
Following the evaluation plan and the related evaluation criteria and metrics documented, the consortium implemented the types and phase of assessments:
Internal Round Assessment: collection of the internal feedback and insights to the first version of the Knowledge Community and Simulation Games.
Evaluation of the First Prototype: collection of all users’ insights, feedback and suggestions for improvement during the first assessment round (Pilot A).
Evaluation of the Final Prototype: collection all the insights, feedback and suggestions for improvement from the second assessment round (Pilot B).
Project’s Assessment / Comparative Analysis: provision of insights on the potential use of the project’s outputs, their benefits and transferability to different areas of application
A brief summary of the principles and key findings of this extensive on-going evaluation are presented in the paragraphs below.
The L4S project planned two rounds of piloting evaluations to be conducted within the project lifecycle, involving overall a total number of 433 participants within the period August 2010 – June 2011, involving target users with different backgrounds.
Figure 9: Piloting users in Round A and B
(see attached document)
Round A involved participants which were mainly students. Round B focused mostly on crisis management experts from the sea and air domains, belonging to both partners’ organizations and their networks. Post-graduate students and academics that participated in the pilot trials of Round B were selected from disciplines closed to L4S project research interest.
The number of piloting sessions was more or less the same (10 sessions in Round A, 12 sessions in Round B), with a greater piloting of the IMPACT simulation based learning session, followed by the Reckon&Choose! and the Crisis Team ones.
A greater number of participants were involved in the piloting of the IMPACT based sessions both in Round A and B. The second most piloted simulation game was the Reckon&Choose!.
The figure below illustrates how the total number of users is distributed per simulation game-based session, within the project (with no distinction among Round A and Round B). 53% piloted the IMPACT game, followed by the Reckon&Choose! (29%), the Crisis Team (11%) and the Crisis Tube (7%).
Figure 10: Participants involved in the piloting of the different simulation games within the project (see attached document).
The expertise of the piloting users involved in Round B is different than the one of Round A. In particular, a greater number of experts of crisis management within the air and sea domains but not only were involved in Round B (66% of participants). This is much higher than the 18% involved in Round A, where the organized pilots involved mainly students and non experts of crisis management (73% vs. 18% of experts). The composition in terms of expertise of the two groups of participants (Round A and Round B) allowed the L4S Consortium and in particular the facilitators of the learning sessions to focus on different aims and objectives set up for the different sessions. On one hand, Round A focused much more on the technicalities of the games and usability, clarity of the games’ flow and their objectives. For those aims, the non experts were good piloting users to be involved. On the other hand, the involvement of experts within the field of crisis management in Round B allowed the facilitators to focus much more than within Round A on the pedagogical value and impact on learning of the games, discovering in greater details how to better exploit the value of the learning sessions.
Feedback on the learning experiences and technical quality of the simulation games was collected during both pilot rounds, mainly through structured questionnaires distributed to the participants both on paper and in electronic format through the OWL4S but also focus groups and interviews with participants were used. The assessment questionnaires submitted to the Workshop’s participants included the following subjects of analysis:
- Overall workshop assessment, including Likert-based and open questions on the quality of workshop’s contents, its comprehensiveness, role of the facilitator, schedule and overall workshop’s organization and degree of involvement of participants
- Realism and value of the simulation game with respect to the scenario, mission and complexity of the crisis management challenges and decisions the participants are required to take and implement through the simulation-based experience.
- Impact on knowledge, skills and attitudes, presenting Likert-based questions on the pedagogical value of the simulation-based learning experience (validation within educational and organizational contexts). According to the theoretical frame of this refers to Kirkpatrick’s learning level’s assessment. The decision to assess the simulations’ impact on knowledge, skills and attitudes is coherent with the methodological foundation presented within WP1 deliverable.
- Technical assessment of the simulation games, including Likert-based questions aiming at evaluating the games’ technical performance and features.
- User manual evaluation, including Likert-based and open questions for the assessment of manual’s clarity and comprehensiveness for the playing of the simulation game.
Additionally, an assessment form to be filled in by facilitators was produced.
Before submission to participants and facilitators, both assessment forms were revised and enriched with proposals, ideas and comments. We involved in the revision process also faculty members and pedagogical professionals external to the project and three full-time vocational professors for management learning and executive education, in their capacity as didactic workshop experts. Assessment forms were adapted to each simulation game, covering the perspectives on the assessment of specific technical features developed within and injected into the games.
Within Pilot A, ten (10) learning sessions were organized which involved the following partners:
- Partners organizing, leading and involved in the development of the simulation-based learning experiences workshops: OFAI, AlphaLabs, Deloitte, UCSC, Frequentis, AIA, WHL, and ALBA.
- Organizations and partner members/associates not involved in the development, invited to attend the simulation-based learning experiences evaluation workshops: CNS Solutions &Support GmbH, UCSC students, University of the Aegean (Chios) professors and students, J Lauritzen A/S Shipowners, Erasmus University of Rotterdam, INSEAD, Mercantile Marine Academy of Chios, Kreab Crisis Management Academy.
Pilot A, performed within August 2010 – January 2011, allowed us to collect directly from piloting users (professors and academics, experts of crisis management and students as well as non experts) a substantial number of suggestions and feedback on how to improve the first deployed versions of the L4S simulation games, the learning experiences based on their use and the OWL4S as a tool supporting the learning experience itself.
It was interesting to note that across different learning experiences, both facilitators and participants stressed the fact that:
- As far as the organization of simulation game-based learning experiences was concerned, the following aspects are of importance:
o Supporting material (briefing and debriefing slides, supporting readings, case studies etc.)
o Time allocation during the learning sessions (better structuring of the learning sessions, balancing the time dedicated to briefing, playing session and debriefing
o Preparation of a detailed – and interactive - introduction for non experts (as they are not familiar with the technical language, and the crisis management processes and procedures)
o Allocation of a more adequate time to allow the participants to get used to the simulation game user interface
- As far as the satisfaction of participants was concerned, we noted that:
o While playing, non expert users indeed complained about and expert players confirmed key characteristics of a crisis! (i.e. too stressful, too much time pressure, need to keep many variables under control, etc.)
o Especially true for non experts, they liked to play and play and play … emphasizing the importance of “pulling users out of the simulation game”, after having successfully immersed them into it.
- As far as the realism and complexity of the simulation game-based learning experiences was concerned, gathered feedback highlighted:
o The complexity of the technical language for non experts (which could be addressed e.g. by providing on-site glossaries for crisis management abbreviations, etc.)
o As suggested by experts, verbal communication should be allowed while playing, to the extent compatible with the abstractions and aims of the modeled scenario.
- Finally, as highlighted by several facilitators of the simulation game-based learning experiences based on the sessions they worked on and informal feedback gathered from participants, it would be desirable to work on:
o Translation of the simulation games, in order to allow a greater number of participants to have access to them (i.e. we especially refer to the operative level who does not have a good level of English)
o The possibility to zoom into decision making processes taking place during a crisis
o The possibility to inject (more complex) events depending on the expertise of the users
In the context of Pilot B round, and up to the end of June 2011, a total number of 12 sessions have been organized by the consortium partners, involving a total number of 201 final users. Round B pilots involved mainly and almost entirely experts of crisis management and executives of key organizations involved in the crisis management procedures, within the air and sea domain. This evidence allowed the consortium to gather relevant insights from those who are routinely and daily involved in crisis management practices and procedures within the sea and air domains nut also and beyond, collecting feedback not anymore merely on the technical features of the simulation games, which were quite consolidated after the end of Round A pilots, but especially and more important on the pedagogical value and learning impact of the simulation game-based learning Workshops.
This kind of feedback allowed the Consortium to strengthen the pedagogical material to be delivered together with the simulation games and illustrating the suggested ways of use of the games themselves to maximize learning gains.
As foreseen within the Evaluation Plan (D6.1) a final strategic evaluation was conducted in June 2011. This final evaluation aimed at analyzing the project’s impact in different fields and application areas and in particular it helped the L4S Consortium to forecast the benefits and transferability of the L4S solutions to different industries and across different European regions.
The assessment explored the following topics:
- Added-value and impact of simulation game-based trainings on organizational and individual learning;
- Linking the L4S outputs to the target user organizations: perceived value and strengths;
- Potential barriers to the use of the L4S simulation games in organizational contexts;
- Identified and suggested target users of the L4S simulation games and OWL4S;
Consortium partners were provided with an interview form and guidelines to conduct the interviews. Consortium partners were asked to target any decision maker who had an influence on training courses (i.e. HR managers, training directors if any, master program directors etc.), or any employed within the partner organization or its network able to give an idea on if and how the L4S outputs could be exploited after the end of the project. Not necessarily people to be interviewed had to be involved in piloting sessions. For this reasons, partners were asked to illustrate the value of the simulation game based learning sessions, the simulation games and the OWL4S through the dissemination material created throughout the project life (i.e. videos, leaflets, illustrating material, pictures etc.) and available on the project website and on the Knowledge Community.
To broaden the number of people involved in the strategic assessment, a strategic assessment form was also created on the OWL4S to be accessed by the participants attending the L4S final event, the International Symposium on Crisis Management. Participants of the Symposium were contacted by Deloitte after the conference and were sent the link and access information to enter the OWL4S and fill out the online questionnaire, composed by open ended and multiple choice questions. They were given the possibility to answer the questionnaire in anonymously.
The key finding of the strategic assessment highlights that the L4S package is relevant within the domain of crisis management, appreciated also by organizations not directly involved in crisis management within the air or sea domains and also by organizations which do not deal directly with crisis management practices. The issue of collaboration within unexpected and unforeseen situations and interaction under time pressure are in fact real ones for any type of business.
As a consequence, the L4S package has a number of identified fields of applications which include the following:
- National and international organizations dealing with crisis management in transportation including all management staff and first responders from the parties involved in the emergency operation (e.g. Rescue & Fire Fighting Services, Medical Services, Police, Civil Aviation etc.) having emergency roles/duties (strategic/tactical and also those at the operational level).
- Large and multinational companies interested in the L4S package as a training tool to manage collaboration in situations of stress, time pressure, unexpected flow of events (crisis management is linked to management and business continuity practices).
- Academic contexts, preferably MBA students and researchers.
As far as future exploitation of the L4S package is concerned, a number of stakeholders involved in the piloting cycles expressed their interest for future usage of the simulation games within their training sessions and courses, beyond the life of the project.
Potential Impact:
1.4.1 Potential impact
L4S project addresses the third functional group of multi-mission capabilities targeted by the ESRAB Report, namely Training and Exercises. These capabilities are specified as follows :
• Improve front line skill levels through training initiatives such as scenario and situation modelling, computer aided training, and simulation
• Improve Education of citizens on the manner in which to behave in case of crisis
• Test and audit of front line staff and facilities
The simulation-based learning experiences produced by this project target specifically behavioural and attitudinal skills, competencies and capabilities that are of critical significance in the context of managing crises. Prior experience with equivalent simulation-based training in “regular” management situations in both the private and public sectors has shown that it is the best way to solidify theoretical insight and transform it into actionable skills that managers and administrators deploy in their regular everyday work. The problem with established traditional methods of training (seminars, case studies) is that they remain purely cognitive exercises; they inform the participants’ espoused beliefs but they cannot alter the participants’ action when it matters. As a result we observe the ubiquitous knowing-doing gap: Everybody knows what should be done “in theory”, but, when putting the theory to practice, everybody reverts to entrenched and familiar habits. Simulation-based training is not only more realistic, it forces participants to engage in action under conditions of heightened pressure, intensified complexity, and forced collaboration breakdowns. In simulation-based learning experiences, participants engage in action, reflect on their actions, and then try again in order to solidify their lessons.
This kind of learning experience is particularly relevant in training and preparing for responding to crises, for at least two reasons. First, crisis management demands high-performance execution. Security crises do not afford the luxury of time to reflect extensively or to proceed with trial and error. The relevant skills and competencies must be fully assimilated in advance, both at individual as well as collective levels, so that appropriate actions and behaviours are seamless. Second, crises do not represent regular or commonly recurrent events. Therefore there is limited or no opportunity to practice in real world instances in order to establish effective work routines. Simulation of the proposed kind is just about the only available method for training and maintaining preparedness for responding to security crises.
More broadly, L4S simulation-based learning experiences can be used to incite and provoke behavioural responses to external crises or collaboration breakdowns, in a simulated manner, in a controlled and safe environment.
The theoretical models underlying both the simulation software as well as the logic of workshop structure and conduct were drawn from the relevant social sciences and sub-disciplines (psychology, leadership, complexity theory, organization theory, knowledge management, etc.). More specifically, the simulation experience exposes participants in a variety of intensely conflicting, paradoxical or generally problematic situations. These situations are based in real world scenarios and thus resonate with participants’ own experience. In dealing with such situations during the simulation, participants also fall into various behavioural, cognitive and attitudinal traps, which are also common in real world experience. The debriefing session that follows clarifies and organizes experience into actionable lessons that can be applied immediately in practice. The L4S simulation-based learning experiences have a critical impact in the ways in which key players in transportation security perceive, understand and act upon complex human and organizational actions and reactions. Much more so, since the deployment scenarios that were piloted and will distributed after the completion of the project, have direct relevance and applicability to the priority concerns of this class of industry users, namely agents, owners and operators of critical infrastructures in transportation.
Furthermore, the ESRAB Report explicitly recognizes the three pillars of effective security preparation and response, namely Technology, Organization, and People. The following diagram illustrates this point. Although information and communication infrastructures present tremendous promise both in prevention and in crisis response, their effectiveness is limited by the people responsible for managing and implementing security and crises are able to collaborate effectively in complex settings involving a multitude of private and public organizations, security agencies, critical infrastructure (CI) owners and operators, and others.
Figure 11: The three pillars of security (amended from the ESRAB Report) (see attached document)
Changes in one apex of the triangle affect the other two. Threat prevention and crisis response can only be as good as the weakest link in this triangle. L4S project addressed all three apexes of this security triangle in a balanced manner. It developed and deployed advanced simulation technologies, aimed at the cultivation and enhancement of “soft” competencies and skills for people in complex organizational and inter-organizational settings. More specifically, by “soft skills” in this case we refer to competences for effective collaboration in contexts where high performance in critical, resources (human, organizational, or technical) may be limited, overburdened or compromised, and at four distinct levels, namely individual, group, organizational and inter-organizational, and, more generally, in complex networks of people, organizations and technical resources.
The impact-generating outcomes of L4S project can be classified in three main categories. First, simulation-based learning experiences are integrated solutions combining technology (computer simulations) and educational processes and techniques. These workshops can be deployed in a variety of contexts (deployment scenarios), serving different objectives every time (e.g. training, experimenting, maintaining alertness, etc.) Second, the identification and development of collaboration skills, competencies and capabilities at individual, group, organizational and inter-organizational levels. Here the focus is on those collaboration and coordination competencies that serve specific requirements in crisis management. The L4S project offers a repertoire of theoretical and applied insights that can be effectively exploited to reflect on change current collaborative (or non-collaborative) behaviours. Third, the project presents an exploitation plan for the growth of a broader market of simulation products aimed at stimulating and exercising behavioural and attitudinal skills and competencies related to security prevention and response.
The further deployment of the L4S simulation-based learning experiences can make a significant contribution to the overall effectiveness and efficiency of security measures and crisis management training in critical infrastructures and in the broader European society. Greater preparedness and more effective collaboration during the management of a security incident will:
- minimize the damage to life and property
- reduce insurance claims from critical infrastructure operators
- restore order sooner
- maintain public confidence
- maintain economic and political stability
- minimize overall costs of security measures
Security measures represent a significant cost and they do not contribute directly to economic growth (with the exception of the value of the security industry per se, which can be significant). It follows that minimizing both the damages from a security incident and the costs of responding to that incident corresponds to a direct contribution to the overall net output of the European economy, its global competitiveness, and the well-being of European citizens. Appropriate investment to human and organizational capabilities is a critical determinant of the extent to which technology infrastructures and security planning perform as expected.
1.4.2 Dissemination activities
Work Package 5 was dedicated to the continuous dissemination of the project work throughout the duration of the project, and to the final packaging of the L4S simulation games, documentation and dissemination of the project outputs resulting from WP4 and from previous WPs in an easily deployable version. The project’s partners focused on the organization and implementation of dissemination activities and have established collaboration with the potential end users of the L4S Simulation Prototypes during the reporting period.
The L4S project approached two large groups of transportation and security actors. Figure 12: L4S Target Groups schematically presents the linkage between the two groups as well as the main actors that constitute each group (see attached document).
The target group with higher priority is titled as the “primary” target group. The “primary” target group consists of organisations that belong to the transportation sectors and mainly in maritime and aviation. These types of organisations are airports, ports, maritime companies, aviation and naval authorities, air couriers, cargo companies and transportation security authorities. The primary target group includes organisations that face similar security challenges as the project’s pilot partners. The pilot testing of the effectiveness of the learning solutions developed through the active participation and involvement in validation of the pilot partners has produced valuable feedback regarding the customization needs and the dissemination and exploitation of the L4S outputs towards the primary target group.
The second target group named after as the “secondary” group consists of educational and training institutions, as well as first responders and civil protection bodies. Professors, researchers and students from technical universities and business schools as well as specialized firms in corporate training for transportation industry players, change management companies, transportation consultants and relevant NGOs are defined as the L4S project’s outputs secondary target group. The group is complemented with executives, employees and human resources trainers from first response and civil protection bodies.
The main activities that took place in the context of the L4S project, addressing the two target groups, follow.
Organization of Dissemination Events and Activities
- Organization and run of the “International Symposium on Crisis Management 2011 – ISCM 2011” held in Athens for two days
The final international dissemination event of the L4S project, entitled “International Symposium on Crisis Management - ISCM 2011” took place the 9th and 10th of June 2011 at the Hotel Divani Apollon Palace and Spa in Vouliagmeni, Athens, Greece. The theme of the Symposium was “Competencies Development and Challenges in Transportation”. The organization of the event was leaded by the coordinator of the L4S project, Deloitte Business Solutions SA with the co-organization and support of the L4S partners. The Symposium attracted around 100 international participants from the air and sea transportation and the academic sector as well as policy makers, first responders and civil protection organizations.
The participants had the opportunity to attend the research results of the project and participate in special workshops experiencing the L4S Simulations and tools. Moreover, two parallel special workshops were conducted. In the first workshop, facilitated by Professor Albert Angehrn, the participants experienced the “Mayday Mayday" Simulation Game. Multimedia means (video) were used during the workshop which analyzed in a systemic and constructive way the organizational barriers that occur during crisis management in the transport and business, proposing ways for the improvement of the organization’s Crisis Readiness.
In the second workshop, facilitated by the Professor of Information Systems of the ALBA Graduate Business School Dr. Nicholas Mylonopoulos, the participants who came mainly from the air transportation experienced the “Crisis Team” Simulation Game. The “Crisis Team” immersed the participants in the distressed work environment of the management of an air accident highlighting the significance of cooperation during this situation. During the ISCM 2011, the participants attended the keynote speaker of the Symposium, Professor Sanjay Sarma from the Massachusetts Institute of Technology (MIT, USA) presenting the key challenges in the field of transport security and the modern technologies to be implemented in the near future (sensors, RFID, etc). Moreover, top level executives from leading organizations such as Mr. A. Aravanis, COO from the Athens International Airport and Mr. E. Vordonis, Executive Director of Thenamaris Ships Management, performed speeches on crisis management in the transportation sector.
In addition, two panel sessions were held on the development of competencies development in the air and sea transport. The panels involved distinguished guests from international organizations and institutions specializing in transport such as FREQUENTIS (Austria), the Catholic University of Milan (Italy), the Port of Barcelona (Spain), the University of Greenwich (England) etc. Research projects on a crisis management and security that are co-funded by the7th Framework Program of the European Commission (BEMOSA, PANDORA, BRIDGE, E-SPONDER and CAST) were also presented during the symposium.
- Organization of the “L4S Panel” during the “8th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2011)”
The L4S consortium was represented at the 8th International Conference on Information Systems for Crisis Response and Management - ISCRAM 2011 by organising and conducting the Panel Session “Experiential models for developing crisis management competencies in sea and air transport” on 10 May 2011 in Lisbon, Portugal. The session was held at the auditorium of LNEC´s Congress Center with the participation of 40 persons. The aim of the panel was to contribute to the learning effectiveness of crisis response training by sharing the L4S knowledge and experience by discussing alternative experiential models for developing advanced collaboration competences at the tactical and strategic levels of crisis management.
- Organization and of six National Dissemination Events in the following countries: Greece (two events), France, Germany, Italy and Austria, inviting key stakeholders of the security and corporate community
- Organization or participation in 65 workshops and meetings in order to raise awareness of the L4S project’s objectives and outputs in Europe and North America.
Creation of Dissemination and Marketing Material
- Creation and continuous update of the L4S website and the “International Symposium on Crisis Management -ISCM 2011” website
- Design of the project’s brochure and of five leaflets for the L4S applications. All these publications were made available in English, French, German, Spanish, Italian and Greek.
- Development of three editions of e-Newsletters which were disseminated to all interested and participating individuals and organizations during the second year of the project’s life time, quarterly
- Creation of the L4S poster
- Publication of eight press releases on the international and national dissemination events
- Publications of eleven articles and papers
- Communication of the L4S project objectives, outputs and events in more than seventeen websites
- Tutorials disseminating the L4S Simulation Prototypes
- Production of videos regarding L4S work progress
- Production of twelve video based interviews
- Shooting videos and photos of the events
1.4.3 Exploitation of results
The main activity implemented under this task is the development of the Deliverable 5.4 “Final Exploitation Plan and Distribution Strategy Report”. The deliverable was based on data collected regarding the identified market for the exploitation of the L4S applications in each participating country. Meetings, discussions, questionnaires and interviews were put in place by the L4S consortium in order to gain valuable feedback on the deployment of the exploitation plan of the L4S outcomes. Moreover, all partners used their network of collaborators in order to identify potential users of the L4S applications. The main conclusions of the “Final Exploitation Plan and Distribution Strategy Report” follow.
During the second year of the L4S project, there has been made constructive progress and development on the identification of L4S exploitation perspectives and diffusion strategy, starting from the first effort of the “Initial Exploitation and Dissemination Plan” (D5.1) to the Final Version of “Exploitation Plan and Distribution Strategy Report”.
L4S project exploits the reality and the needs of organizations to help their managers and personnel in order to develop key skills and competencies in crisis management area through an innovative, highly involving, effective and easy deployable life-long learning service.
More precisely, L4S simulation-based learning experiences on Crisis Management reflect in a very realistic way crisis management and collaboration dynamics in modelled disastrous situations in air and sea transportations. Players are involved in role-playing crisis incidents in which they can validate and develop specific competencies through learning-by-doing in realistically modelled scenarios.
All L4S learning experiences’ applications are designed and developed in order to improve:
- the crisis management and collaboration competencies of European corporate personnel, decision-makers and academic learners
- the crisis readiness of organizations, both in the private and public sector Europe-wide
The L4S Learning Experiences service is comprised by the simulation games and learning-networking tools developed during the L4S FP7 project; built upon the L4S framework, an experience-based learning framework that address the effective development of crisis management and collaboration competencies for corporate and academic learners.
L4S Learning Experiences Portfolio, includes the following simulation games on air and sea transport crisis management: i) “IMPACT – The Crisis Readiness Online Simulation Experience” a set of 3-leveled simulation games, ii) “RECKON&CHOOSE! Air Simulation” and iii) “CRISIS TEAM”. Apart from simulation games the portfolio contains the WEB 2.0 advanced networking and sharing tool “CRISIS TUBE Leadership Learning Network” as well as the supportive online workshop tool “OWL4S”.
Regarding the impact that the L4S games bring both to individuals and organizations, a set of value propositions of L4S service has been deduced, assisting the easier comprehension of its core competencies.
Value Proposition 1
L4S Simulation-based Learning Experiences provide an innovative Learning-by-Playing edutainment service of advanced pedagogical value for the domain of Crisis Management
L4S Learning Experiences particularly aim at the development of the competences critical to the management of a crisis and an emergency. This is achieved by involving corporate and academic learners in experiencing and debriefing a number of Simulated Crisis Situations that:
- are based on realistic and complex scenarios to make sure that the insights, knowledge and competencies acquired are linked to specific not-cognitive- only experience [Crisis Management “Knowing –Doing Gap” Trap]
- virtually replicate challenging role-playing situations, which require decisions making, coordination, collaboration and real time knowledge exchange competences under conditions of time pressure, emotions, complexity, diversity of actors involved and other constraints/complicating factors [Specific multi-level Crisis Management Traps]
- provide an experience of operation in teams (online and face-to-face) addressing a challenging task to review principles, enabling and supporting more productive collaboration particularly in teams which are often distributed and highly diverse [Crisis Management Team Collaboration Traps]
- address the link between the behaviour of individuals and teams in crisis situations and the overall organizational and inter- organizational contexts in which they operate , as this can be more or less conductive to effective crisis management [Crisis Management Organizational Collaboration Traps]
Value Proposition 2
L4S Learning Experiences promote the collaborative inter-organizational knowledge exchange and development of enhanced quality standards and learning approaches related to Crisis Management
L4S learners in parallel with the L4S simulations, have the opportunity to get involved in Collaborative Learning Networks and Applications (“Crisis Tube”, “OWL4S”, “L4S Knowledge Community”) based on Web 2.0 platforms with key functions to:
- support the continuous developments and updating of the insights gained and competencies acquired [Crisis Management Competencies Obsolescence Trap]
- enable and stimulate dynamic experience and best/worst practices exchange [Crisis Management Tacit Knowledge Loss Trap]
- assist and motivate the collaborative identification of new target areas and to further improve critical competencies /capabilities of effective Crisis Management at the individual , team, organizational and inter-organizational level [Crisis Management Parochial Perspective Trap]
Value Proposition 3
L4S service enhances the competencies, attitudes and the organizational factors required to improve the crisis readiness of individuals and organizations
An important concept introduced by the L4S games is the one of “Crisis Readiness”, a concept which stimulates the reflection of managers and decision makers on the readiness of their organizations, their teams as well as themselves as individuals/leaders when it comes to handle crisis situations. The L4S simulation games provide players with a challenging experience of collaborating and taking decisions in distributed and diverse (multi-functional, transnational) teams addressing complex crisis situations. Moreover, they motivate the players to assess and gradually improve their own competences related to crisis management at the individual, group and organizational levels. In more detail, they provide the opportunity, methodology and tools to enhance the following skills of corporate and academic learners:
- Situation Assessment Competences
- Stress & Panic Management Competences
- Collaboration & Coordination Competences
- Inter-cultural Decision Making & Negotiations Competences
- Organizational, inter-organizational & general Communication Competences
- Design & Management of org. and inter-org. Crisis Management Systems
Value Proposition 4
L4S service can be applied to different types of organizations, irrelevant of their size, sector, status or location addressing their specific needs
The simulation-based experience provided by L4S service can be easily tailored to the needs of various sectors and customer’s environment as its pedagogical content is vast and can be easily adapted and if needed enriched according to the demands of different audience (i.e. strategical, operational, or tactical level) and organizational structures (individual, teams/groups, organizational and inter-organizational). Therefore, it is apparent how such tools are appropriate not only in the L4S application fields where they have been firstly deployed (transport) but, practically, in all domains where a potential risk component holds. For this reason, all of these tools apply to all organisations due to the large variety of emergencies and crises they may be exposed. Organisation’s size may just determine the impact reach (e.g. by involving more or less stakeholders) a crisis may provoke.
Concerning the identification of the best way of diffusing this type of high-value learning experiences Europe-wide, the consortium has conducted a market analysis, which produced detailed and relevant market information focusing primarily on the countries that partners are originating from. As far as the exploitation perspectives are concerned, all L4S partners have verified their commitment to continue collaborating on L4S outcomes in areas such as:
- Scientific research and publications on L4S multi-disciplinary domains
- Simulation games research and development
- Enhanced teaching and training in security and crisis management using simulation games
- Exploring areas of security that simulation-based capability development can be researched and obtained, while pursuing and designing suitable simulation-based learning experiences
The post-L4S deployment phase will be primarily based on individual exploitation of the project outcomes by each L4S partner. The individual exploitation plans of the L4S partners are categorized in three distinct types, according to their entity nature, core business and professional expertise. The exploitation direction can be either:
- Internal: Organizations provide the L4S Learning Experiences internally to its employees and executives, adopting the L4S applications portfolio in its internal executive training programs.
- External: Commercial entities distribute the L4S Learning Experiences as a service to its customers in various industries. Moreover, the Learning Experiences may be bundled with other existing business products or services.
- Academic: Educational and Academic institutions integrate in their curricula the L4S training applications and they can also provide executive and vocational training to corporations and organizations.
The individual exploitation plan of each partner is provided in the respective report (D5.4).
Immediately after the project’s closure a “Deployment Assessment Committee” (DAC) will be established comprising from one representative per partner. The objective of this committee will be to monitor and assess the exploitation phase of the L4S Learning Experiences as well as to exchange best practices of applying the L4S outcomes mainly on pedagogical and technological dimensions. Furthermore, a “royalty system” will be after the completion of the project established that will reflect a revenue sharing policy of the consortium for the commercial exploitation activities of the partners. A Commercial Advisory Group, from the business partners is formed, in order to monitor the royalty system and consult the commercial partners.
Last but not least, a sustainability plan of the external deployment phase –concerning the four L4S commercial partners- has been carried out. There, a financial and viability analysis is present for each of the four entities, complementing with key hypotheses and assumptions.
The L4S consortium has clear evidence that L4S Learning Experiences have a high potential for diffusion and impact through various exploitation streams, that offer to the L4S partners distinct but valuable high quality assets in their internal and/or external operations and to transportation and security stakeholders an innovative portfolio of advanced learning experiences on crisis management.
From a business perspective, on the basis of the work conducted, we believe that both the diffusion and the financial targets set can be achieved, with value generated at all levels – from the personnel and decision makers experiencing the value of the proposed simulation-based learning sessions, to the facilitators, the organizations’ representatives, the network of companies diffusing the L4S Learning Experiences Europe-wide, the designers of advanced crisis management learning simulations and the academic staff and learners that will experience these learning environments. There is a real opportunity to make significant impact.
A multi-level exploitation plan has been thus defined, and its execution is commencing straightforward after the project’s completion. The work done clearly indicates that the currently available and immediately deployable L4S Simulation Games and Applications can provide impact and visibility, along with the generation of a strong stream of revenue to support this growth. The longer-term strategy and vision is to set up an efficient Europe-wide B2B Channel for the diffusion of Game-based Learning Experiences.
Such a channel would:
- Be positioned to run in parallel and synergistic to traditional publishing or software diffusion channels, including consulting
- Achieve the mission of efficiently diffusing new Game-based Learning Experiences (developed by a variety of players) which fulfil quality standards set for L4S Simulation Games
- Aim at rapid extension to other relevant educational/impact areas. The development plan is to expand the basic project results and customise them to the needs/ characteristics of other Critical Infrastructures (energy, financial services, food industry, etc.) and throughout Europe and beyond
- Support the growth of a variety of players , increasing our involvement in the design, production and delivery of simulation games and game-based learning experiences.
This is an ambitious but, we believe, achievable long-term objective and vision. Its focus extends over and beyond the horizon addressed in this FP7 project.
List of Websites:
L4S Knowledge Community (http://L4S.fvaweb.eu)
The L4S website (www.L4S-Project.info)
The ISCM website (www.ISCM2011.eu )
L4S project exploits the reality and the needs of organizations to help their managers and personnel in order to develop key skills and competencies in crisis management area through an innovative, highly involving, effective and easy deployable life-long learning service.
More precisely, L4S simulation-based learning experiences on Crisis Management reflect in a very realistic way crisis management and collaboration dynamics in modelled disastrous situations in air and sea transportations. Players are involved in role-playing crisis incidents in which they can validate and develop specific competencies through learning-by-doing in realistically modelled scenarios.
All L4S learning experiences’ applications are designed and developed in order to improve:
- the crisis management and collaboration competencies of European corporate personnel, decision-makers and academic learners;
- the crisis readiness of organizations, both in the private and public sector Europe-wide.
The L4S learning simulations address three forms of competencies with a specific regard to the challenges of crisis management at all different organizational levels mentioned above:
- Knowledge that focuses on cognitive abilities;
- Attitudes that focuses on affective and normative facets of competence, and
- Skills focus on the ability to perform action.
The L4S project successfully designed and developed the “L4S Learning Experiences service” comprised by advanced simulation games and learning / networking applications. The high-value learning service is built upon the L4S framework, an experience-based pedagogical framework that address the effective development of crisis management and collaboration competencies for corporate and academic learners.
L4S Learning Experiences Portfolio, includes the following simulation games on air and sea transport crisis management: i) “IMPACT: The Crisis Readiness Online Simulation Experience” a set of 3-leveled simulation games, ii) “RECKON&CHOOSE! Air Simulation” and iii) “CRISIS TEAM”. Apart from simulation games the portfolio contains the WEB 2.0 advanced networking and sharing tool “CRISIS TUBE Leadership Learning Network” as well as the supportive online workshop tool “OWL4S”.
L4S partners have verified their commitment to continue collaborating on L4S outcomes in areas such scientific research and publications on L4S multi-disciplinary domains, simulation games research and development and enhanced teaching and training in security and crisis management using simulation games.
The individual exploitation plans of the partners are formed in three distinct types of utilizing the L4S applications:
- Internally: organizations will provide the L4S Learning Experiences internally to its employees and executives, adopting the L4S applications portfolio in its internal executive training programs;
- Externally: commercial entities will distribute the L4S Learning Experiences as a service to its customers in various industries. Moreover, the Learning Experiences may be bundled with other existing business products or services;
- Academically: educational and academic institutions will integrate in their curricula the L4S training applications and they can also provide executive and vocational training.
From a business perspective we believe that both the diffusion and the financial targets set can be achieved, with value generated at all levels; from the personnel and decision makers experiencing the value of the proposed simulation-based learning sessions, to the facilitators, the organizations’ representatives, the network of companies diffusing the L4S Learning Experiences Europe-wide, the designers of advanced crisis management learning simulations and the academic staff and learners that will experience these learning environments.
A multi-level exploitation plan has been thus defined, and its execution is commencing straightforward after the project’s completion. The work done clearly indicates that the currently available and immediately deployable “L4S Simulation Games and Applications” can provide impact and visibility, along with the generation of a strong stream of revenue to support this growth. The longer-term strategy and vision is to set up an efficient Europe-wide B2B Channel for the diffusion of Game-based Learning Experiences.
Project Context and Objectives:
The aim of L4S project is to provide a clear understanding and to further develop existing know-how, in both interdisciplinary scientific/academic models and best/worst practices and experiences in the field of transportation. It focuses on the factors inhibiting effective collaboration dynamics in crises and leading to the failure of effective crisis management and of the interventions required to reduce these risks.
This know-how is integrated into an innovative framework for addressing the development of collaboration competencies of crisis managers in the transportation sector. This L4S Framework is validated through the implementation of state-of-the-art highly interactive and experiential learning solutions that enable the effective understanding and management of the challenges in crisis situations, as validated with the participation of practitioners in the field. These challenges include:
(1) acting under extreme time pressure
(2) facing the lack, ambiguity, and/or asymmetries of information
(3) dealing with human factors like cognitions, attitudes and emotions, and
(4) addressing the interpersonal relationship dimension like fast relationship building and activation for the mobilization of social resources, trust building, cohesion and role definitions and also handling diversity and conflicts.
The L4S objectives are to develop simulation based learning experiences and provide supporting guides and tools for developing the soft skills necessary in effective crisis management. These Learning Experiences are deployable for a variety of purposes, including:
- Training security managers and crisis managers in organization, particularly in the transportation sector
- Preparing and testing security/crisis readiness of organizations and of their employees
- Auditing the organizational preparedness for crises in security domains and beyond (i.e. financial crises)
- Preparing the broader public on appropriate behaviors and responses to security incidents
- Conducting research (e.g. experimental tests) on the impact of different responses to the reaction of the public or the effects of the security incident.
In terms of outputs, the project delivers significant contributions:
1. A comprehensive Online Knowledge Community Integrating a Knowledge Base and an active Virtual Learning Community on Advanced Collaboration Dynamics and Technologies of Leaning for Security (L4S).
The project contributes innovative interdisciplinary research in the area of collaboration dynamics for crisis management combined with best practices and experiences from the transportation industry. This know-how is structured and collected within a dynamic Knowledge Base accessible online and maintained up-to-date through contributions from the members of a Virtual Learning Community from both academia and transportation industry.
2. L4S Framework: An experience-based learning Framework that address the effective development of collaboration competencies for crisis managers
The project contributes an innovative framework (L4S framework) targeting the design of pedagogically effective simulation based learning experiences employing Advanced Organizational Simulation Games (based on computer-enhanced collaborative and experiential learning models and simulation games design principles). Broad application and sustainability of the L4S framework is guaranteed through its inclusion in the L4S learning section of the online L4S Knowledge Base and Virtual Learning Community, that extends access to professors and trainers on crisis management.
3. Four (4) L4S Simulation Games: Validated experienced based learning solutions deployable in educational and organizational contexts
The project contributes with the development of significantly more effective eLearning solutions (“L4S Simulation Games”). The eLearning solutions developed are validated in both academic and industrial contexts through a series of Workshops and are packaged for easy employability in companies and universities to enhance the efficiency and quality of learning and competency development in the critical area of collaborative dynamics for crisis management in the transportation sector. More specifically four simulation prototypes are being deployed based on the requirements provided by the industrial partners (air and sea transportation actors) participating in the consortium.
4. L4S Deployment Package: Effective instruments and tools for the generation of simulation based learning experiences
Finally, there exist different deployment modes for simulation based learning experiences, for example within or between organizations, among homogenous or diverse trainees, in a co-located or distributed setting. The L4S Deployment Package provides an integrated solution for the different type of simulation deployment models and includes as components: pedagogical guidelines and notes, practical deployment guidelines and scenarios, different types of simulation games, an online workshop tool, a monitoring and assessment tool, and access to the L4S knowledge community.
Project Results:
The scientific and technological methodology and associated work plan of L4S project evolved from a model for the design process of Small World Simulations developed by Angehrn (2006 ). The L4S project is advanced through a continuous interplay between technological and educational development, on field trials and observations, implementation and validation of simulation learning activities in different learning environments. Domain experts remain strongly involved in key roles throughout the process. The evaluation of the project’s impact is performed according to a concrete and methodological scheme which is developed in the framework of the project. A dissemination strategy was devised at the beginning of the project and has been performed in stages during the project’s duration. Additionally, an evaluation plan covering both technical as well as learning aspects ensures that the final deliverables are suitable for their purposes and can be naturally integrated in the different environments.
The implementation scheme of the project was planned to be developed across three main axes:
- collaboration challenges and related competencies & dynamics for crisis managers (WP1)
- knowledge processes and community building (WP2)
- advanced technological tools for simulation games (WP3).
The pathways document the full network of options and the choices actually explored within the project. They also document separate trends of development that may occur as a result of the different trial environments. More concretely, the objective of WP1 is to produce a comprehensive integrated collection of models and insights emerging from both scientific theory and practical experiences relevant to the human collaboration dimension in crisis situations as a foundation to be integrated in the L4S knowledge community (WP2), in the L4S framework and in the L4S simulation games (WP3).
The objective of WP2 is the design and development of two tools to support the simulation design and deployment: The L4S Knowledge community (KC) and the L4S Online Workshop Tool (OWL4S).
The objective of WP3 is to develop a first version of the L4S Framework for the design of effective simulation based learning experiences addressing collaboration dynamics and competencies development in crisis management, and to use it to drive the design and implementation of a first prototype of four L4S Simulation Games reflecting the knowledge gathered in WP1 and WP2.
All three pathways have produced at regular intervals analytical reports from the meetings and the workshops, observations and reports from the field trips, technical, pedagogical and evaluation reports produced during the development of the tools -four simulation games prototypes - and their components. These reports also compare with the state of the art in their respective areas. The interconnection amongst the three pathways is facilitated through the extended pilots that were realized during the second year of the project and strengthen the involvement of domains expertise.
During the first pilot round, a total of 10 different pilots has been carried out, involving 246 pilot users, while in the second piloting phase an additional set of 12 different pilots was conducted involving a total of additional 201 pilot users in which the L4S Prototypes and their components have been deployed. It is evident that a significant number of pilot trials were conducted, in a variety of corporate and academic environments, engaging a respective number of players (close to 500) and creating significant opportunities for the exploitation dimension of the L4S prototypes. Important pilot and commercial activities are already planned for the post-L4S phase, engaging hundreds of players. Furthermore, a lot of effort was carried out in the refinement and fine tuning of all applications, in order to deliver learning experiences of high value.
Concerning the identification of the best way of diffusing this type of high-value learning experiences Europe-wide, the consortium has conducted a market analysis, which produced detailed and relevant market information focusing primarily on the countries that partners are originating from.
The L4S consortium has now clear evidence that L4S Learning Experiences have a high potential for diffusion and impact through various exploitation streams, that offer to the L4S partners distinct but valuable high quality assets in their internal and/or external operations and to transportation and security stakeholders an innovative portfolio of advanced learning experiences on crisis management.
The following paragraphs present concisely the scientific and technological achievements of the L4S project, structured on the work plan of the project.
1.3.1 WP1: Knowledge Harvesting and Integration
The first milestone of the project was achieved through the preparation and completion of the report on the findings of the Work Package. In sum, Work Package 1 produced a comprehensive analysis of the relevant knowledge and important insights that emerged from the exploration of both the academic theories and the practical experiences. To provide the knowledge harvesting and integration according to the tasks description, a three-dimensional research framework has been developed and consequently applied in the research work in WP1 (see Figure I). This framework takes into account that all current and future challenges for an effective crisis management must be addressed at different, but interdependent levels – the individual/interpersonal level, the group level, as well as the organizational and inter organizational level. Moreover, since the main focus of the L4S project is effective learning, an important dimension of our research framework is related to learning objectives, especially to the competence dimension. This dimension differentiates three forms of competencies that are addressed by the L4S learning solutions – knowledge, attitudes/norms and skills. Within trainings and educational contexts, this taxonomy is widely applied and can be thought of as "the objectives of the training process" in the L4S learning context. This focus on competences is in also line with the agenda of the European Union on security (European Security Research Advisory Board, 2006). Finally, the L4S research framework puts emphasis on crisis management in the transportation sector. Accordingly, as a third dimension of the research framework we applied the transportation dimension that includes primarily the "air" and "sea" transportation. The L4S research framework is depicted in Figure 1: L4S Research Framework
(see attached document).
WP1 provided a clear understanding anchored in both interdisciplinary scientific/academic models and best/worst practices and experiences of the factors inhibiting effective collaboration dynamics and leading to the failure of collaboration initiatives in crisis management (“Traps & Challenges”), and of the interventions required to reduce these risks (“Crisis-Related Collaboration Management Competencies”). The results of this report contributed to an innovative framework (ACDT Framework) targeting the design of effective technology-enhanced learning solutions that are built on Advanced Simulation Games (based on computer-enhanced collaborative and experiential learning models and simulation games design principles) in the WP3.
Based on its research framework, the Work Package 1 was able to meet the following scientific goals that reflect the most significant results of this Work Package:
- Identification of the most essential causes and forms of crises, emergencies and security-related incidents;
- Detection of the most critical problems and challenges that the crisis and security managers have to master in crisis situations with a particular emphasis to the transportation sector;
- Exploration of knowledge, attitudes as well as social and cognitive skills that the crisis managers have to possess to be able to successfully meet the crisis challenges in the transportation sector;
- Discovering what are the key biases and the most critical misbehaviors that negatively affect the decision-making and communication by managers in crisis situations;
- Uncovering main social mechanisms of collaboration in crisis management teams (CMT);
- Analysis of what are the main phases of the organizational crisis management process and which phase-specific management activities determine the crisis preparedness of an organization;
- Finding out which processes govern the inter-organizational collaboration in crisis situations and the factors that affect the effectiveness and efficiency of these collaboration processes;
- Localization of critical practice-driven requirements to the learning methods, technologies and tools that have not been appropriately met by the existing approaches to the training of crisis managers in the sea and air transportation sector
- A critical appraisal of the state-of-the-art learning technologies that are in use for crisis management training and the identification of requirements that have to be addressed by the L4S learning experiences;
- Development of learning objectives for the L4S learning solutions that have to be addressed to increase the efficiency and effectiveness of crisis management training with particular emphasis on the transportation sector.
1.3.2 WP2: Knowledge Community
A significant result produced is the setting up of the Knowledge Community. This milestone summarizes a varied set of actions such as the L4S Knowledge Community design, implementation, and finally the launch process taking into account the results produced from the corresponding task and deliverable and mainly based on the potential applications of the Learning community platform.
The initial purpose of the L4S Knowledge Community is to support/stimulate collaboration towards the final objective of this project. The long-term scope of the L4S Knowledge Community is to provide the researchers, crisis managers and simulation designers with a comprehensive knowledge repository and a community space that enable them to prepare a learning experience that effectively exploits the simulation tools in a way that meets their training needs.
The L4S Knowledge Community (KC) is composed of two main domains, the Knowledge Base and the Virtual Learning Community. Practically the two areas are fully hyperlinked and most of the tools are tightly integrated. The platform uses the latest Web 2.0 features. The following figure (Figure 2: KC interface) depicts the KC’s interface (see attached document).
Community has incorporated all the knowledge assets gathered in WP1, as well as additional ones that were and will be progressively inserted in the Knowledge Base throughout the project and beyond the project timeline. The Knowledge Community plays an important role supporting and encouraging the knowledge exchanges and the collaborative work of the consortium members and their networks toward the iterative design of the simulation-based learning experiences. During the duration of the L4S project the platform has been used by the consortium members for internal communication, knowledge sharing and classification, and support for the daily work. The KC restricted area supports the project management, encouraging the consortium daily exchanges and documents sharing.
Furthermore, some features of the KC are used to manage collaboratively the content of the L4S project public site (www.l4s-project.info) and the ISCM website (www.ISCM2011.eu).
From a wider prospective, the L4S Knowledge Community provides researchers, security managers and staff, crisis management experts and trainers with a comprehensive knowledge repository and a community space focusing on the challenges of security and crisis management in the transportation sector.
During the pilot rounds the Knowledge Community has been used to support trainers and facilitators with content, case studies and scenarios to be used for the design of effective learning experiences and training sessions.
In the other hand, the workshop participants and learners have in their disposal though the KC content, manuals and additional educational and academic material to enrich the learning experience.
In parallel, during and after the project lifetime, the KC is a tool, together with other dissemination channels, of promoting the project’s outcomes and to diffuse the research results to the international academic and research community.
Several types of users are targeted for using the knowledge community, for different purposes. These target groups are categorized as:
Academic user, researcher. This user could be interested in using the KC to retrieve relevant contents on the issue of security and crisis management that could be useful for publication aims. On one side, the KC is an important repository of knowledge and theories on security and crisis management. The KC can also support the academic user in discussing and sharing ideas and insights with colleagues. The academic user can be a facilitator or a simulation designer.
Decision makers in transportation organizations. This category of users can be considered as potential buyer/client of the L4S learning solutions. They can access knowledge assets, information and marketing material and also access videos, leaflet and promotional material of the project outputs.
Simulation developer/designer. This user could be interested in using the contents, i.e. theories and models, retrieved into the KC to build on these theoretical bases a simulation scenario he intends to create. The KC could then represent a good starting point to address those knowledge harvesting activities that represent the first step of a simulation development.
Facilitator/Teacher (academic or corporate sector). This user accesses the KC and uses its contents to structure a simulation-based workshop experience. This is a category of users that is supposed to do an intensive uptake of the KC.
Learner/student/participant to the L4S learning experiences. This user can find interesting material for his learning interests. Before and after the L4S learning experiences, the learners can access the KC to download material, open discussions, and participate to discussions.
The L4S Online Workshop Tool (OWL4S) is also developed during WP2 to support both the collaboration within the Knowledge Community and the simulation based learning experiences. The Online Workshop Tool is an enhanced environment designed to support L4S learning experiences. Its main interface is presented in
Figure 3: OWL4S participant’s interface (repository area)
(see attached document). Based on requirements and suggestions coming from some of the most relevant players in the Simulation Games learning providers and Industrial partners in the specific domains of security and transportation (L4S partners, their networks and the external experts involved in the Scientific Committee), the OWL4S has been designed including functionalities and features like:
- Flexibility and customization of all the elements (interface, contents, structure)
- Possibility to integrate any media (video, web pages, on-line applications) into the OWL4S pages
- Possibility to inject new real-time elements in the game (like an unexpected event) and add elements (like a brainstorming), while the participants are playing.
- Different ways to be delivered (co-located participants/distributed on-line, synchronous/asynchronous, individual/group, autonomously/with a facilitator)
- Different options of usage and application fields like:
o support brainstorming
o submit questionnaires
o voting pool
o presentation area
o repository to download pedagogical material
These functionalities and features allow the usage of this application as:
- Tool to support and integrate the different phases and elements of a learning experience
o providing documentation before the workshop to the learners
o supporting the briefing phase, integrating presentations with different media type
o encouraging the collaboration and ideas sharing, thanks to the brainstorming feature
o supporting the debriefing phase, integrating presentations with different media type
o collecting feedback and questionnaires
o distributing informative material after the workshop
o supporting a post workshop discussion and ideas sharing, thanks to the brainstorming feature
- Environment to design micro games (to focus on specific pedagogical dimensions)
- Dilemma-based simulation game, integrating videos, presentations and dilemmas in a pre-defined flow.
During the first year of the L4S project, the OWL4S has been used as a working tool for brainstorming and to collect ideas on the games design within the L4S consortium members. This usage has been very important in order to make improvements and changes on the interface and structure of the tool.
During the pilot rounds the OWL4S has been used as part of the L4S learning experience:
- To support phases of the learning experience (i.e. during briefing and debriefing phases, as brainstorming environment, as collaborative decision making support, etc.)
- Environment integrating all the elements of a simulation games learning experience (from the briefing phase to the game, from the interactive moments to the debriefing, from the questionnaires to the distribution of additional material to the participants)
- Flexible Space to design micro games (i.e. for the Reckon & Choose game to focus on specific pedagogical dimensions, so called “zoom in”)
- To expand the simulation possibilities, thanks to the exchange of data from the game to the OWL4S (like in the Reckon & Choose! simulation game).
After the L4S project conclusion the OWL4S will be exploited and promoted initially through the partners’ networks and clients, and, at a second stage proposed to the market, as part of the L4S learning package, following the exploitation plan.
The OWL4S has three main target users:
Learning experiences developer/designer: The Learning experiences developer/designer is the player who design the learning experience based on the client’s learning requests/needs. This user can also play the facilitator’s role or provide the learning experience to facilitators/teachers to be delivered to learners.
This user will find in the OWL4S an interesting platform, fully customizable (from the interface to structure and contents), to be used as an environment to design and deliver learning experiences. The OWL4S can also be used as part of an existing learning package, to support specific needs (i.e. brainstorming, voting, questionnaire submission).
Facilitator/Teacher (academic or corporate sector). The facilitator is the player who delivers the learning experience to the learners. He is the principal target user for the OWL4S because the system empowers him with several interesting features to better deliver the learning contents. The Facilitator can easily customize any content of the OWL4S depending on his needs.
Learner/student/participant to the L4S learning experiences. This user is the beneficiary of the learning experience. He will find in the OWL4S an interesting environment to better enjoy the leaning contents (i.e. documents, presentations with different media type) and interact with the teacher and the other learners.
And an indirect target user:
Client (after the L4S project conclusion). The client is an indirect target user of the OWL4S because he is not expected to use the platform. The client is the firm or institution that requires (and usually pays for) the learning package for his employees. The client expresses the learning goals to the facilitator/trainer in order to allow him to design a learning experience.
1.3.3 WP3: L4S Framework and Simulation Prototypes
Building on the information gathered from the industrial partners and referring to the management-related educational needs elicited in WP1, the consortium worked on the L4S Simulation Games Prototypes’ design and specification of scenarios, implemented by L4S sub-working teams addressing either the Maritime or the Air Transportation sector (“Sea Team” and “Air Team”). The insights gained in the extensive study of crisis management-related knowledge sources (summarized in the D1.1) has enabled the consortium to identify a number of factors determining the success or failure of crisis management in organizational and inter-organizational contexts, as well as a number of levels (individuals, teams/groups, organizations, inter-organizational entities) at which crisis management (or crisis management breakdowns) can be observed – and potentially influenced. In the WP3 deliverable on “L4S Framework, Simulation Scenarios and Design” (D3.1) the Design Framework is analyzed, i.e. the steps which led gradually to the design, specification and initial implementation of a first set of four prototypes of L4S Simulation-based Learning Experiences. In doing so, the attempt to deploy state-of-the-art pedagogical models and learning technologies (from multimedia to Web 2.0) was carried out. A first important design step was the way to structure the knowledge gathering process, conducted in the first phase of the project, through the integration of a variety of sources addressed from multiple perspectives. The second key step consisted in identifying three different L4S Simulation Levels according to the specific crisis management -related dynamics and breakdowns they focus on, namely:
(a) Organizational and Inter-Organizational Dynamics: Such Simulations and Learning Experiences focus particularly on factors determining crisis management dynamics in different organizational and inter-organizational contexts/cultures, their productive or dysfunctional manifestations in everyday organizational life, and the possibility to influence such dynamics through managerial action.
(b) Group/Team Dynamics: Such Simulations focus particularly on factors determining success or failure of crisis management groups or teams, their manifestation in the context of specific situations, and the possibility to influence such group dynamics in organizational and inter-organizational contexts, as well as in real-time or asynchronous crisis management situations.
(c) Individual/Interpersonal Dynamics: Such Simulations focus particularly on factors determining both motivation and capability to manage crisis situations at the individual level, their manifestation in inter-personal exchanges/crisis management contexts, and the possibility to influence them.
For each prototype, in the corresponding deliverable (D3.1) is presented the Pedagogical Foundations and Deployment Approach, the Underlying Models, as well as how its Implementation started and evolved. All prototypes were designed in a way to support their rapid development and incremental improvement, as well as the validation and deployment of the L4S Prototypes in the pilot settings made available by the Consortium partners.
The L4S prototypes designed and developed as well as their positioning matrix in relation with the organizational level that they address are depicted in the following figure. Apart from the designing and educational dimensions, this positioning will assist the effective promotion of the applications to the market as well as to cover the distinct needs of the various target segments and organizations Europe wide. For each prototype one-page summary is provided in the following pages. Figure 4: Positioning Matrix of the L4S Prototypes (see attached document).
Impact: Crisis Readiness Simulation
“IMPACT” is the name of the first of the Simulation Prototypes developed by the Sea Team, aimed to target key managers and decision makers in the Maritime Transportation sector EU-wide. The IMPACT prototype provides, through a set of connected, video- and simulation-based, online collaborative learning experiences, deployable both on-line and on-site or through a combination of both, an engaging experience based on a complex, multi-stage and multi-focus simulation game in which the users/players experience the challenges of crisis management at the individual, team and organizational level, and hence gain insights on how to best address these challenges in their own contexts. The game is designed to be played flexibly on-site (traditional workshop of 2 days) or online over a period of several weeks (from 4 to 9) and is articulated at 3 Levels, each of which addresses relevant competences and insights related to Crisis Management.
At Level 1 the players – operating in teams – are involved (through a first set of interactive video episodes) in 3 crisis situations that they will have to address by collaboration in distributed and diverse teams. The storyline features a company needing to address complex decisions in situations requiring immediate reaction through team discussions in order to identify the best way to react to the presented crisis situations and take difficult decisions that determine how the story unfolds and either succeed or fail in productively resolving the crisis situations.
At Level 2 the focus shifts from the team to the organizational perspective of effective crisis management. The level evolves though three basic steps. In the first step (“Maritime Transportation Crisis Analysis”), the teams review a video which describes a problem caused by a major accident due to a storm in the future (2016) and identify the key characteristics of the situation at hand as well as real cases and examples of similar real crisis situations and produce a first set of key managerial decisions to be taken to face the crisis situation and avoid its escalation. In the second step (“Organizational Crisis Immediate Reaction Analysis”) the teams are invited to review a video of how the simulated organization reacts to the crisis event through the activation of a number of key individuals distributed across Europe and their reaction to the crisis event. The team must compare the “ideal” behavior to the “actual” one that would take place in their organizations, and identify the top 3 Organizational Enablers of such behaviors (ideal and actual). In the third step (“Organizational Crisis Resolution Analysis”) the storyline underlying the simulation describes in detail how the simulated organizations succeed in productively resolving the crisis situation through the smooth collaboration of the key individuals involved in spite of their different roles and position within the organization. This part of the simulation provides again an “ideal” reference point for comparison of crisis resolution pattern featured in the simulation with the current reality of their organizations, collaboration and coordination analysis of the key players and which key competences and organizational enablers are involved.
Level 3 involves a more personal dimension through an analysis of the learners own, personal Crisis Readiness. This last phase aims at making learners aware of the leadership characteristics necessary to productively address crisis situations, and is directly linked to the current literature on leadership behavior by creating an intensive (stressful) individual online crisis management experience in which learners are exposed real-time to a continuous flow of information sources and requests for decisions and actions, facing them real-time to a set of overlapping complex crisis situations involving several people and issues which develop over time as a function of what the learner decides or not to do. The analysis of this intensive experience is then debriefed (in a traditional session or directly online, through a set of debriefing videos) based on the analysis of the learners decisions and behavior during the simulation and the data captured by the simulation real-time, providing a number of direct and personalized insights related to how to best address and improve the Crisis Readiness of the individual players.
Figure 5: Overview of User Interface at Level 3 of the IMPACT Simulation
(see attached document).
Reckon & Choose! – Tactical Emergency Collaboration
Reckon & Choose! simulation experience reflects the recognition of the particular situation of the current European landscape that is characterised by the lack of a consolidated body of knowledge and best practices regarding this coordinating tactical level in the air transportation industry. Today, there is high heterogeneity of approaches across nations, with additional legal and regional variations. Analysis of current performance in larger-scale emergencies exposes a general consensus that there is room for significant improvement and a specific need for standardisation.
The Reckon & Choose prototype was conceived to contribute to meeting this need by not only providing a learning environment for individual trainees, but by furthermore effectively serving as a knowledge elicitation and collection tool that gathers explicit and tacit knowledge from expert players and facilitators, producing a knowledge base that grows with individual deployments.
Throughout the game small teams of players interact with a model of reality, i.e. an abstracted excerpt that however preserves and concentrates key characteristics such as time pressure, partial information and dynamic (and not necessarily coherent, i.e. possibly contradicting or out-of-order) information flow. The players would not themselves be the decision makers in the simulated world, but rather be placed in observers roles. Setting out from the starting event of an airplane crashing into an airport building, they trace and follow one out of a set of time courses already taken. This trace is defined incrementally through specific choice points, at which the players are offered a very limited amount of different options (typically only two) to choose from: Each of these choice points should present a realistic, interesting, and relevant dilemma, selected out of a range of challenges derived from the Athens Airport Case document and informed by the feedback and complementary information provided by the Air Team partners.
In their answers, trainees’ are requested to specify explicitly the reasons (pros and cons) of each decision option as they see it and how they aggregated these pieces of information into their final decision for this challenge. Each of the possible paths from an opening scene to an ending scene would comply with the requirements described previously, e.g. of a coherent dramatic arc connecting the choice points to the first and the last scenes, and the decision options being offered in terms of outcomes, and not operational procedures.
According to this approach, trainees are expected to follow up on decisions made previously by tracking the pros and cons of those decisions and trying to optimise the path taken in the subsequent decisions– cf. the relevance of coherent behaviour and leadership. Generally, it is the utmost urgent and critical task for all leaders to communicate and make clear what their decision is, and which pros and cons they are seeing, in order to make the whole team follow this track of the decision taken — no matter whether some other decision would have been possible.
Along this line, the fact that trainees are forced to take a decision out of a finite set of options can be defended on the grounds that in facing a crisis there are no bad decisions, but rather it is important to try to maximise the outcome of any given path taken. Also, the choice dilemma formulations and the options offered are not to be selected based on considerations of actual best and worst practice, but should rather be defined along didactical criteria, such as e.g. making it difficult to come to a decision based on clear-cut arguments or a clear difference in overall assessment value.
Figure 6: The main display of the Reckon & Choose! Prototype, structured into the “view mode” (right hand side) and “action mode” (left hand side) (see attached document).
Crisis Team
The Crisis Team simulation is primarily targeted to non-experts in airport crisis management and aims to provide an experiential collaborative context through which they can engage with the subject matter. The learning goal is two-fold; first to engage participants with essential actions and the roles of key actors in crisis response and second to immerse participants in the distressed work environment of crisis situations where communication is patchy, the environment is volatile and there are multiple conflicting demands in compressed time.
In this simulation a team of players is asked to work with other teams on a particular airport crisis scenario for figuring out which actions to take and in what order. This approach allows for the emergence of collaborative behaviors and breakdowns originating actually from the players themselves. Moreover it forces participants to debate the content, objectives, resources and relative significance of different crisis response actions. There are no virtual characters; the players themselves are immersed in the collaborative experience with team dynamics and conflicts emerging out of this collaborative experience. Crisis Team is a short term learning experience of several hours. The online platform serves as the medium through which teams communicate and is designed to constrain communication possibilities so as to induce the difficulties and breakdowns that realistically arise in stressful collaboration contexts. The facilitator controls communication facilities, the time pressure and extraordinary events.
The Crisis Team scenario involves six teams of players each representing a different actor of the crisis management team of a major international. Whereas team members are collocated, teams are assumed to be distributed, communicating via the online Collaboration Platform which is design to induce artificial, yet realistic, stress and breakdowns. The mission is the operational implementation of the airport’s crisis management plan with the specific goal of selecting the most appropriate actions from a list of potential actions and of prioritizing them. Together the teams have to determine the actions to be implemented as well as the order in which to implement them. The collaboration platform is designed in a way that enables the players to experience how an initially collaborative environment can disintegrate into a competitive one in which teams begin to display non collaborative behaviors such as blame, accusation, frustration and because of the absence of face-to-face interactions, negative misattributions to the other teams’ choices, as well as refocusing of resources on competition rather than looking for reachable win-win solutions or a healthy compromise.
The simulation process unfolds in two main phases. Initially the actor is pre-assigned three actions representatives of the priority agenda of that actor. In the first phase, and after each team of participants has taken up an actor identity, they nominate their three actions, one at a time, each team in turn. Thus all teams have the opportunity to observe and to contemplate their own agenda and the actions nominated by other teams. In the second phase, teams take turns reassigning the priorities of all actions (nominated by all teams). An essential feature of the simulation is that teams are given mission objectives which are slightly differentiated and conflicting in the minor details. This recreates the diversity and knowledge integration challenge in which teams experience the difficulty of working with other teams that operate under different mindset(s), criteria and agendas.
By virtue of the way it is set up the Crisis Team simulation seeks to immerse players in the difficulties of collaborating in diverse teams that work under pressure. A second challenging feature of the simulation is the emergence of group and team breakdowns that can occur during the collaborative process. Through the Crisis Team simulation, players will experience how misunderstandings and miscommunications arise due to technological limitations and the absence of face-to-face interactions, mismatches between different objectives or agendas, the difficulties of knowledge integration in diverse teams, and other non-collaborative behaviors that can emerge in these situations. Another characteristic of the Crisis Team simulation is the opportunity to manage external, unplanned pressures and events that can occur during the collaborative process (and executed through the intervention agent). While teams might have established a common understanding and objective and may be working in a positive direction toward the accomplishment of their goals, external events might occur (as often do in reality), which might disrupt the momentum, pace and focus of collaboration. These events include external pressures such as political intervention, new related or unrelated emergency incidents, failure in resource mobilization, and others. Such events are under the discretionary control of the training facilitator.
In the standard setting, the collaboration platform allows extremely restricted communication, namely each team only sees the outcome of every other team’s decisions. There are no means of deliberation. In addition the facilitator controls whether teams can have access to a chat facility or not. Thus different collaboration phenomena can be produced. In addition, the facilitator controls an optional timer which can put even more time pressure to participants.
Figure 7: Operating Team’s Interface in CRISIS TEAM simulation
(see attached document).
CRISISTUBE Leadership Network
“CRISISTUBE” targets top managers and decision makers in the Maritime Transportation sector EU-wide. It provides collaborative inter-organizational Knowledge Exchange and development of enhanced quality standards & learning approaches related to Crisis Management through state-of-the-art Web 2.0 based Learning Networks technology to support effective exchange of Best Practices & experiences.
CRISISTUBE prototype provides an opportunity for key players addressing crisis management in the Maritime Transportation sector to join an active learning community operating continuously and over time, accessing information about other players (Networking Dimension), engaging in relevant learning-oriented exchanges related to crisis management fueled continuously by experts (Learning through the discussion of Cases) or by the users themselves (user-generated exchanges with sharing and discussion of experiences/best or worst practices). After having accessed the website through its Authentication Component users will typically see the overall interface of CRISISTUBE and start navigating through the Knowledge Network.
The Knowledge Network underlying CRISISTUBE is a highly dynamic environment in which Knowledge Assets of different types (videos, documents of different types, links, but also comments and discussions related to existing Knowledge Assets) can be browsed through, searched, as well as added flexibly
The Knowledge network is based upon three concepts:
- Challenges: users (typically experts) can describe Problem Areas or desirable Targets which are critical when it comes to effective crisis management in the Maritime Transportation sector and can be best addressed through inter-organizational collaboration.
- Experiences: all the users can describe their own experiences, particularly those related to facing one or more of the Challenges. Experiences can be both positive and negative, but are a means to contribute in addressing an existing Challenge, or to identify a new one. Experiences can also be directly connected to other Experiences, supporting best/worst practice sharing in critical areas.
- Ideas: all the users can describe their own ideas for inter-organizational collaboration, particularly those related to facing one or more of the Challenges. Ideas can also be directly connected to other Ideas, i.e. build on each other over time, supporting the collaborative identification of projects to be launched at the inter-organizational level.
Figure 8: Operating the Networks Visualization Pane in CRISISTUBE
(see attached document).
The L4S consortium during the 2nd period of L4S finalised the work carried on in the WP3 by actually further developing the simulation games and applications in order to make them effectively usable in the pilots by the L4S group and, more in general, by the L4S community first, and by all interested crisis-related communities eventually. Therefore, all implementations have been installed on the servers and made available to the pilots.
The tasks accomplished were the development and implementation of the server back-end and the implementation of the rich client front-end (regarding all of communication with the server, realisation of client-side game logic, and user interface/interaction management). Effort was put on creating various workshop formats, catering for both varying lengths and varying numbers of workshop participants to meet the variety of implementation possibilities the complex, multi-stage and multi-focus IMPACT simulation game can assume. This includes implementation testing of on-line and on-site collaborative learning experiences as well as a combination of both approaches. The implementation of the workshop format also concerns the design and implementation of interface formats, i.e. incorporation of briefing and debriefing aspects, linking results and workshop outcomes between all three levels of the IMPACT learning simulation to ensure a smooth learning transition for participants and aiding the sophistication of the simulation game end-product. For Reckon&Choose! the “air-team” worked on the facilitator console main contributions were made to the storyline. For the same game, partners collaborated regarding the media assets and technological solutions for user interface components, development of the “Country book” and of debriefing materials.
In addition, air and sea team interactively provided to each other content-related and technical feedback. All these research and development (RTD) efforts were interleaved with the WP4-related demonstration activities. Pilot partners provided detailed feedback and suggestions regarding the structure and content of the briefing and debriefing sessions of the learning experience by making best-practice transfer.
To sum up, all prototypes were successfully designed and developed in a way to support their rapid development and incremental improvement, as well as the validation and deployment of the L4S Prototypes in the pilot settings made available by the Consortium partners, as planned in the Work Package 4.
1.3.4 WP4: Pilots and Prototyping Cycles
Work Package 4 was dedicated to the running of Pilots and the progressive fine-tuning of the L4S Framework, the L4S Simulation Games (from WP3), the underlying L4S Knowledge Community (from WP2) and the Online Workshop tool, to progressively converge through cycles of evaluation-redesign-implementation towards final versions of these outputs.
1st Pilot Round August 2010 – January 2011
During the first pilot round, a total of 10 different pilots has been organized and carried out, involving a total of 246 pilot users, in which the four L4S Prototypes and their components have been deployed.
Table 1: Overview Table of Pilots conducted in the first round
L4S Prototype (see attached document)
With the help of all the Consortium members we were able to conduct the pilots during this phase already with participants belonging to different target groups (and not only with participants from the consortium members involved in the need analysis and the design and development of the L4S Prototypes, as initially planned. The advanced stage of the prototypes and their attractiveness for the target groups made it possible to organize sessions in which we were able to interact with university students (to test the deployment in educational settings) as well as with professional managers (to test the deployment in corporate training and life-long learning/executive development programs).
The pilots conducted have enabled us to gain new insights and to move one significant step forward in validating that the L4S Prototypes we have designed and developed during WP3:
- match the expectations of real users in terms of usability and learning value
- are built on software and media components which work as expected also in real deployment contexts (and not just under “lab” conditions)
- can be effectively deployed by other educators/trainers based on the pedagogical/ deployment guidelines provided
- are effective with a variety of audiences/target groups
Furthermore, a number of feedback collection methods (from questionnaires over brainstorming to focus groups) were experimented with and generated a variety of improvements which have emerged from the pilots. These insights were reviewed, integrated and implemented to produce new versions of the L4S Prototypes themselves, as well as of the related pedagogical material (teaching and deployment guides).
Overall, based on the experiences gathered during this first pilot round, the four L4S Prototypes appeared to meet well the expectations we set in our initial design document (D3.1) in particular in terms of:
- attractiveness (willingness-to-try)
- deployability (from technical and organizational points of view)
- innovativeness (compared to other approaches to address Crisis Mgmt)
- learning value (in terms of addressing effectively relevant competences)
This has been recognized in different contexts (from traditional, on-site workshops to online deployment contexts) by the very challenging audiences we were able to involve in the ten Pilots.
To conclude with the first pilot round of L4S, the pilots conducted have enabled us to gain new insights and to move one significant step forward in:
- Improving the reliability and quality of the software and media components
- Improving the debriefing approach (instructions for facilitators)
- Exploring how the L4S Prototypes can be “packaged” in order to fit the needs (in terms of crisis management competences) and be widely deployed with an even broader Target Group, namely top managers and decision makers in the Transportation Sector, but also in other relevant sectors like Energy, Industry or Services (like IT).
2nd Pilot Round February 2011 – June 2011
In the second piloting phase we have been able to organize an additional set of 12 different Pilots involving a total of additional 201 pilot users – with 850 additional committed users to conduct further pilots during Fall 2011 - in which the L4S Prototypes and their components have been deployed.
Table 2: Overview Table of Pilots conducted in the second round
L4S Prototype (see attached document)
Like in round one, we were able to conduct the Pilots during this phase already with participants belonging to different Target Groups. The advanced stage of the Prototypes and their attractiveness for the target groups made it possible to organize sessions in which we were able to interact with university students as well as with professional managers attending company training courses or Management School programs.
Through this second round of Pilots we have confirmed further that the L4S Prototypes:
- match the expectations of real users in terms of usability and learning value
- are built on software and media components which work as expected also in real deployment contexts (and not just under “lab” conditions)
- can be effectively deployed by other educators/trainers based on the pedagogical/ deployment guidelines provided
- are effective with a variety of audiences/target groups
The additional insights were reviewed, integrated and implemented to produce new versions of the L4S Prototypes themselves, as well as of the related pedagogical material (teaching and deployment guides). Effort was focused on thorough testing, documentation, third-party deployment, packaging, marketing, and production in order to prepare L4S prototypes becoming ready for deployment beyond the L4S project’s consortium members and pilot organizations
During the second pilot round, the Knowledge Community was also used mainly to support the consortium collaborative work and to enrich the dissemination actions to the target groups. The L4S facilitators used it as a knowledge repository to prepare their workshops and offered an additional knowledge repository environment to the participants. Following user’s reports, some technical bugs have been fixed, and some minor functionality has been improved. After the end of the L4S project the platform will be used in two ways:
- As a free Knowledge Community open to researchers, simulation games designers, facilitators and organizations interested in sharing contents and knowledge about crisis management in the transportation sector and beyond. The subscribed members beyond the L4S consortium will be using it as a knowledge repository and knowledge management tool.
- As an open platform to support collaborative knowledge sharing in different domains (lately is used to support the co-financed project Help4Mood – ICT-2009.5.1 – 248765)
During the ICT proposer’s day in Budapest (19 and 20 May 2011), the L4S Knowledge Community was presented to potential users. The environment has raised a good interest, mainly for enhanced learning applications.
For the second pilot round the OWL4S application was used as part of the L4S simulation-based learning experiences and prototypes. The following table lists the sessions that were supported by the OWL4S.
Table 3: Overview Table of Pilots supported by the OWL4S(see attached document)
Discussing with trainers and facilitators, some ideas on how to use the OWL4S in alternative domains and applications are the following:
- The OWL can be also used as a game environment to address specific learning goals and micro-scenarios. This can be done using a video and a related brainstorming session.
- It can also be used to fast draft and test a scenario before implementing a complex simulation game based on it.
- The OWL4S can be used to support zoom-in (or micro-games) for complementing another learning experience (this solution has been tested successfully with Reckon&Choose!).
As a conclusion for the two pilot rounds of L4S project, it is evident that a significant number of pilot trials was conducted, in a variety of corporate and academic environments, engaging a respective number of players (close to 500) and creating significant opportunities for the exploitation dimension of the L4S prototypes. Furthermore, a lot of effort was carried out in the refinement and fine tuning of all applications, in order to deliver learning experiences of high value.
1.3.5 WP6: Embedded, Continuous Evaluation
The objective of work package 6 was to generate through the evaluation process insights related to the technical features and educational, pedagogical and social values of a number of simulations that focused on assessing the extent to which it is possible to “learn”, (or increase ability) to cope with crisis situations using IT. As such, the project outputs have been evaluated from technical, technological and pedagogical points of view. The former aims to understand whether the knowledge community, the online workshop tool, and the simulation games work, are innovative and have the expected functionalities; the latter concentrates on the effects hat these tools have on learning processes, derived from the perceived value of the proposed solutions from users’ perspectives.
During the first 7 months of the project, the consortium has been working on the definition of the evaluation plan. The evaluation plan was dedicated to the continuous assessment of the outputs and applications developed throughout the project and to the documentation of the evaluation methodology and tools. More specifically, the objectives of the evaluation methodology adopted were:
- To assess the effectiveness of the technical solutions.
- To support the understanding of the user’s and market’s needs as well as the relevant functionalities provided by the technical solutions.
- To evaluate the strategic concept, learning value and potentialities of exploitation of L4S outputs.
The aim of the evaluation process was to:
- Develop guidelines for the development of the L4S simulation learning games,
- Provide guidelines and useful input from the users for improving the usability of the final prototypes and
- Investigate the impact that L4S simulation game based learning technology and the overall L4S learning experience has on end users.
Within the Evaluation Plan the need for both the technical/technological assessment and pedagogical evaluation of the L4S outputs has been addressed and justified by relevant frameworks mostly used in real life and diffused in literature. As for the technical evaluation, the L4S outputs were evaluated adopting the model from the ISO 9126 framework documented and unpacked by Chua and Dyson (2004).
As far as the pedagogical evaluation is concerned, the evaluation framework was based on Donald Kirkpatrick's Four Level Evaluation Model. According to the contribution of Kirkpatrick, each training session should be assessed at the following four-levels of evaluation:
- Reaction: how the learners react to the training experience.
- Learning: the extent to which the learners gain new knowledge, change attitudes and improve their skills.
- Behavior: the capability to perform the learnt skills while back on the job/working environment.
- Results: over the long-term, how the organization has improved its monetary results, efficiency, moral, etc. as a result of attending a specific training.
For each of the four levels, the relevance and practicability has been identified as well as the most appropriate assessment procedure in terms of both sample of analysis and tool of analysis.
Technical and Pedagogical Evaluation Criteria and Metrics report has presented the details of each framework and the specific questions to be addressed in each phase of the evaluation plan. The specific report was dedicated to the detailed investigation of the criteria and metrics that were used during the assessment analysis. While the technical perspective refers to the assessment of the technical and technological features of the outputs (technological quality of the tools in terms of usability and functionality), the pedagogical perspective analyses the impact of the tools on learning processes and dynamics identified by all partners using a number of learning objectives.
Following the evaluation plan and the related evaluation criteria and metrics documented, the consortium implemented the types and phase of assessments:
Internal Round Assessment: collection of the internal feedback and insights to the first version of the Knowledge Community and Simulation Games.
Evaluation of the First Prototype: collection of all users’ insights, feedback and suggestions for improvement during the first assessment round (Pilot A).
Evaluation of the Final Prototype: collection all the insights, feedback and suggestions for improvement from the second assessment round (Pilot B).
Project’s Assessment / Comparative Analysis: provision of insights on the potential use of the project’s outputs, their benefits and transferability to different areas of application
A brief summary of the principles and key findings of this extensive on-going evaluation are presented in the paragraphs below.
The L4S project planned two rounds of piloting evaluations to be conducted within the project lifecycle, involving overall a total number of 433 participants within the period August 2010 – June 2011, involving target users with different backgrounds.
Figure 9: Piloting users in Round A and B
(see attached document)
Round A involved participants which were mainly students. Round B focused mostly on crisis management experts from the sea and air domains, belonging to both partners’ organizations and their networks. Post-graduate students and academics that participated in the pilot trials of Round B were selected from disciplines closed to L4S project research interest.
The number of piloting sessions was more or less the same (10 sessions in Round A, 12 sessions in Round B), with a greater piloting of the IMPACT simulation based learning session, followed by the Reckon&Choose! and the Crisis Team ones.
A greater number of participants were involved in the piloting of the IMPACT based sessions both in Round A and B. The second most piloted simulation game was the Reckon&Choose!.
The figure below illustrates how the total number of users is distributed per simulation game-based session, within the project (with no distinction among Round A and Round B). 53% piloted the IMPACT game, followed by the Reckon&Choose! (29%), the Crisis Team (11%) and the Crisis Tube (7%).
Figure 10: Participants involved in the piloting of the different simulation games within the project (see attached document).
The expertise of the piloting users involved in Round B is different than the one of Round A. In particular, a greater number of experts of crisis management within the air and sea domains but not only were involved in Round B (66% of participants). This is much higher than the 18% involved in Round A, where the organized pilots involved mainly students and non experts of crisis management (73% vs. 18% of experts). The composition in terms of expertise of the two groups of participants (Round A and Round B) allowed the L4S Consortium and in particular the facilitators of the learning sessions to focus on different aims and objectives set up for the different sessions. On one hand, Round A focused much more on the technicalities of the games and usability, clarity of the games’ flow and their objectives. For those aims, the non experts were good piloting users to be involved. On the other hand, the involvement of experts within the field of crisis management in Round B allowed the facilitators to focus much more than within Round A on the pedagogical value and impact on learning of the games, discovering in greater details how to better exploit the value of the learning sessions.
Feedback on the learning experiences and technical quality of the simulation games was collected during both pilot rounds, mainly through structured questionnaires distributed to the participants both on paper and in electronic format through the OWL4S but also focus groups and interviews with participants were used. The assessment questionnaires submitted to the Workshop’s participants included the following subjects of analysis:
- Overall workshop assessment, including Likert-based and open questions on the quality of workshop’s contents, its comprehensiveness, role of the facilitator, schedule and overall workshop’s organization and degree of involvement of participants
- Realism and value of the simulation game with respect to the scenario, mission and complexity of the crisis management challenges and decisions the participants are required to take and implement through the simulation-based experience.
- Impact on knowledge, skills and attitudes, presenting Likert-based questions on the pedagogical value of the simulation-based learning experience (validation within educational and organizational contexts). According to the theoretical frame of this refers to Kirkpatrick’s learning level’s assessment. The decision to assess the simulations’ impact on knowledge, skills and attitudes is coherent with the methodological foundation presented within WP1 deliverable.
- Technical assessment of the simulation games, including Likert-based questions aiming at evaluating the games’ technical performance and features.
- User manual evaluation, including Likert-based and open questions for the assessment of manual’s clarity and comprehensiveness for the playing of the simulation game.
Additionally, an assessment form to be filled in by facilitators was produced.
Before submission to participants and facilitators, both assessment forms were revised and enriched with proposals, ideas and comments. We involved in the revision process also faculty members and pedagogical professionals external to the project and three full-time vocational professors for management learning and executive education, in their capacity as didactic workshop experts. Assessment forms were adapted to each simulation game, covering the perspectives on the assessment of specific technical features developed within and injected into the games.
Within Pilot A, ten (10) learning sessions were organized which involved the following partners:
- Partners organizing, leading and involved in the development of the simulation-based learning experiences workshops: OFAI, AlphaLabs, Deloitte, UCSC, Frequentis, AIA, WHL, and ALBA.
- Organizations and partner members/associates not involved in the development, invited to attend the simulation-based learning experiences evaluation workshops: CNS Solutions &Support GmbH, UCSC students, University of the Aegean (Chios) professors and students, J Lauritzen A/S Shipowners, Erasmus University of Rotterdam, INSEAD, Mercantile Marine Academy of Chios, Kreab Crisis Management Academy.
Pilot A, performed within August 2010 – January 2011, allowed us to collect directly from piloting users (professors and academics, experts of crisis management and students as well as non experts) a substantial number of suggestions and feedback on how to improve the first deployed versions of the L4S simulation games, the learning experiences based on their use and the OWL4S as a tool supporting the learning experience itself.
It was interesting to note that across different learning experiences, both facilitators and participants stressed the fact that:
- As far as the organization of simulation game-based learning experiences was concerned, the following aspects are of importance:
o Supporting material (briefing and debriefing slides, supporting readings, case studies etc.)
o Time allocation during the learning sessions (better structuring of the learning sessions, balancing the time dedicated to briefing, playing session and debriefing
o Preparation of a detailed – and interactive - introduction for non experts (as they are not familiar with the technical language, and the crisis management processes and procedures)
o Allocation of a more adequate time to allow the participants to get used to the simulation game user interface
- As far as the satisfaction of participants was concerned, we noted that:
o While playing, non expert users indeed complained about and expert players confirmed key characteristics of a crisis! (i.e. too stressful, too much time pressure, need to keep many variables under control, etc.)
o Especially true for non experts, they liked to play and play and play … emphasizing the importance of “pulling users out of the simulation game”, after having successfully immersed them into it.
- As far as the realism and complexity of the simulation game-based learning experiences was concerned, gathered feedback highlighted:
o The complexity of the technical language for non experts (which could be addressed e.g. by providing on-site glossaries for crisis management abbreviations, etc.)
o As suggested by experts, verbal communication should be allowed while playing, to the extent compatible with the abstractions and aims of the modeled scenario.
- Finally, as highlighted by several facilitators of the simulation game-based learning experiences based on the sessions they worked on and informal feedback gathered from participants, it would be desirable to work on:
o Translation of the simulation games, in order to allow a greater number of participants to have access to them (i.e. we especially refer to the operative level who does not have a good level of English)
o The possibility to zoom into decision making processes taking place during a crisis
o The possibility to inject (more complex) events depending on the expertise of the users
In the context of Pilot B round, and up to the end of June 2011, a total number of 12 sessions have been organized by the consortium partners, involving a total number of 201 final users. Round B pilots involved mainly and almost entirely experts of crisis management and executives of key organizations involved in the crisis management procedures, within the air and sea domain. This evidence allowed the consortium to gather relevant insights from those who are routinely and daily involved in crisis management practices and procedures within the sea and air domains nut also and beyond, collecting feedback not anymore merely on the technical features of the simulation games, which were quite consolidated after the end of Round A pilots, but especially and more important on the pedagogical value and learning impact of the simulation game-based learning Workshops.
This kind of feedback allowed the Consortium to strengthen the pedagogical material to be delivered together with the simulation games and illustrating the suggested ways of use of the games themselves to maximize learning gains.
As foreseen within the Evaluation Plan (D6.1) a final strategic evaluation was conducted in June 2011. This final evaluation aimed at analyzing the project’s impact in different fields and application areas and in particular it helped the L4S Consortium to forecast the benefits and transferability of the L4S solutions to different industries and across different European regions.
The assessment explored the following topics:
- Added-value and impact of simulation game-based trainings on organizational and individual learning;
- Linking the L4S outputs to the target user organizations: perceived value and strengths;
- Potential barriers to the use of the L4S simulation games in organizational contexts;
- Identified and suggested target users of the L4S simulation games and OWL4S;
Consortium partners were provided with an interview form and guidelines to conduct the interviews. Consortium partners were asked to target any decision maker who had an influence on training courses (i.e. HR managers, training directors if any, master program directors etc.), or any employed within the partner organization or its network able to give an idea on if and how the L4S outputs could be exploited after the end of the project. Not necessarily people to be interviewed had to be involved in piloting sessions. For this reasons, partners were asked to illustrate the value of the simulation game based learning sessions, the simulation games and the OWL4S through the dissemination material created throughout the project life (i.e. videos, leaflets, illustrating material, pictures etc.) and available on the project website and on the Knowledge Community.
To broaden the number of people involved in the strategic assessment, a strategic assessment form was also created on the OWL4S to be accessed by the participants attending the L4S final event, the International Symposium on Crisis Management. Participants of the Symposium were contacted by Deloitte after the conference and were sent the link and access information to enter the OWL4S and fill out the online questionnaire, composed by open ended and multiple choice questions. They were given the possibility to answer the questionnaire in anonymously.
The key finding of the strategic assessment highlights that the L4S package is relevant within the domain of crisis management, appreciated also by organizations not directly involved in crisis management within the air or sea domains and also by organizations which do not deal directly with crisis management practices. The issue of collaboration within unexpected and unforeseen situations and interaction under time pressure are in fact real ones for any type of business.
As a consequence, the L4S package has a number of identified fields of applications which include the following:
- National and international organizations dealing with crisis management in transportation including all management staff and first responders from the parties involved in the emergency operation (e.g. Rescue & Fire Fighting Services, Medical Services, Police, Civil Aviation etc.) having emergency roles/duties (strategic/tactical and also those at the operational level).
- Large and multinational companies interested in the L4S package as a training tool to manage collaboration in situations of stress, time pressure, unexpected flow of events (crisis management is linked to management and business continuity practices).
- Academic contexts, preferably MBA students and researchers.
As far as future exploitation of the L4S package is concerned, a number of stakeholders involved in the piloting cycles expressed their interest for future usage of the simulation games within their training sessions and courses, beyond the life of the project.
Potential Impact:
1.4.1 Potential impact
L4S project addresses the third functional group of multi-mission capabilities targeted by the ESRAB Report, namely Training and Exercises. These capabilities are specified as follows :
• Improve front line skill levels through training initiatives such as scenario and situation modelling, computer aided training, and simulation
• Improve Education of citizens on the manner in which to behave in case of crisis
• Test and audit of front line staff and facilities
The simulation-based learning experiences produced by this project target specifically behavioural and attitudinal skills, competencies and capabilities that are of critical significance in the context of managing crises. Prior experience with equivalent simulation-based training in “regular” management situations in both the private and public sectors has shown that it is the best way to solidify theoretical insight and transform it into actionable skills that managers and administrators deploy in their regular everyday work. The problem with established traditional methods of training (seminars, case studies) is that they remain purely cognitive exercises; they inform the participants’ espoused beliefs but they cannot alter the participants’ action when it matters. As a result we observe the ubiquitous knowing-doing gap: Everybody knows what should be done “in theory”, but, when putting the theory to practice, everybody reverts to entrenched and familiar habits. Simulation-based training is not only more realistic, it forces participants to engage in action under conditions of heightened pressure, intensified complexity, and forced collaboration breakdowns. In simulation-based learning experiences, participants engage in action, reflect on their actions, and then try again in order to solidify their lessons.
This kind of learning experience is particularly relevant in training and preparing for responding to crises, for at least two reasons. First, crisis management demands high-performance execution. Security crises do not afford the luxury of time to reflect extensively or to proceed with trial and error. The relevant skills and competencies must be fully assimilated in advance, both at individual as well as collective levels, so that appropriate actions and behaviours are seamless. Second, crises do not represent regular or commonly recurrent events. Therefore there is limited or no opportunity to practice in real world instances in order to establish effective work routines. Simulation of the proposed kind is just about the only available method for training and maintaining preparedness for responding to security crises.
More broadly, L4S simulation-based learning experiences can be used to incite and provoke behavioural responses to external crises or collaboration breakdowns, in a simulated manner, in a controlled and safe environment.
The theoretical models underlying both the simulation software as well as the logic of workshop structure and conduct were drawn from the relevant social sciences and sub-disciplines (psychology, leadership, complexity theory, organization theory, knowledge management, etc.). More specifically, the simulation experience exposes participants in a variety of intensely conflicting, paradoxical or generally problematic situations. These situations are based in real world scenarios and thus resonate with participants’ own experience. In dealing with such situations during the simulation, participants also fall into various behavioural, cognitive and attitudinal traps, which are also common in real world experience. The debriefing session that follows clarifies and organizes experience into actionable lessons that can be applied immediately in practice. The L4S simulation-based learning experiences have a critical impact in the ways in which key players in transportation security perceive, understand and act upon complex human and organizational actions and reactions. Much more so, since the deployment scenarios that were piloted and will distributed after the completion of the project, have direct relevance and applicability to the priority concerns of this class of industry users, namely agents, owners and operators of critical infrastructures in transportation.
Furthermore, the ESRAB Report explicitly recognizes the three pillars of effective security preparation and response, namely Technology, Organization, and People. The following diagram illustrates this point. Although information and communication infrastructures present tremendous promise both in prevention and in crisis response, their effectiveness is limited by the people responsible for managing and implementing security and crises are able to collaborate effectively in complex settings involving a multitude of private and public organizations, security agencies, critical infrastructure (CI) owners and operators, and others.
Figure 11: The three pillars of security (amended from the ESRAB Report) (see attached document)
Changes in one apex of the triangle affect the other two. Threat prevention and crisis response can only be as good as the weakest link in this triangle. L4S project addressed all three apexes of this security triangle in a balanced manner. It developed and deployed advanced simulation technologies, aimed at the cultivation and enhancement of “soft” competencies and skills for people in complex organizational and inter-organizational settings. More specifically, by “soft skills” in this case we refer to competences for effective collaboration in contexts where high performance in critical, resources (human, organizational, or technical) may be limited, overburdened or compromised, and at four distinct levels, namely individual, group, organizational and inter-organizational, and, more generally, in complex networks of people, organizations and technical resources.
The impact-generating outcomes of L4S project can be classified in three main categories. First, simulation-based learning experiences are integrated solutions combining technology (computer simulations) and educational processes and techniques. These workshops can be deployed in a variety of contexts (deployment scenarios), serving different objectives every time (e.g. training, experimenting, maintaining alertness, etc.) Second, the identification and development of collaboration skills, competencies and capabilities at individual, group, organizational and inter-organizational levels. Here the focus is on those collaboration and coordination competencies that serve specific requirements in crisis management. The L4S project offers a repertoire of theoretical and applied insights that can be effectively exploited to reflect on change current collaborative (or non-collaborative) behaviours. Third, the project presents an exploitation plan for the growth of a broader market of simulation products aimed at stimulating and exercising behavioural and attitudinal skills and competencies related to security prevention and response.
The further deployment of the L4S simulation-based learning experiences can make a significant contribution to the overall effectiveness and efficiency of security measures and crisis management training in critical infrastructures and in the broader European society. Greater preparedness and more effective collaboration during the management of a security incident will:
- minimize the damage to life and property
- reduce insurance claims from critical infrastructure operators
- restore order sooner
- maintain public confidence
- maintain economic and political stability
- minimize overall costs of security measures
Security measures represent a significant cost and they do not contribute directly to economic growth (with the exception of the value of the security industry per se, which can be significant). It follows that minimizing both the damages from a security incident and the costs of responding to that incident corresponds to a direct contribution to the overall net output of the European economy, its global competitiveness, and the well-being of European citizens. Appropriate investment to human and organizational capabilities is a critical determinant of the extent to which technology infrastructures and security planning perform as expected.
1.4.2 Dissemination activities
Work Package 5 was dedicated to the continuous dissemination of the project work throughout the duration of the project, and to the final packaging of the L4S simulation games, documentation and dissemination of the project outputs resulting from WP4 and from previous WPs in an easily deployable version. The project’s partners focused on the organization and implementation of dissemination activities and have established collaboration with the potential end users of the L4S Simulation Prototypes during the reporting period.
The L4S project approached two large groups of transportation and security actors. Figure 12: L4S Target Groups schematically presents the linkage between the two groups as well as the main actors that constitute each group (see attached document).
The target group with higher priority is titled as the “primary” target group. The “primary” target group consists of organisations that belong to the transportation sectors and mainly in maritime and aviation. These types of organisations are airports, ports, maritime companies, aviation and naval authorities, air couriers, cargo companies and transportation security authorities. The primary target group includes organisations that face similar security challenges as the project’s pilot partners. The pilot testing of the effectiveness of the learning solutions developed through the active participation and involvement in validation of the pilot partners has produced valuable feedback regarding the customization needs and the dissemination and exploitation of the L4S outputs towards the primary target group.
The second target group named after as the “secondary” group consists of educational and training institutions, as well as first responders and civil protection bodies. Professors, researchers and students from technical universities and business schools as well as specialized firms in corporate training for transportation industry players, change management companies, transportation consultants and relevant NGOs are defined as the L4S project’s outputs secondary target group. The group is complemented with executives, employees and human resources trainers from first response and civil protection bodies.
The main activities that took place in the context of the L4S project, addressing the two target groups, follow.
Organization of Dissemination Events and Activities
- Organization and run of the “International Symposium on Crisis Management 2011 – ISCM 2011” held in Athens for two days
The final international dissemination event of the L4S project, entitled “International Symposium on Crisis Management - ISCM 2011” took place the 9th and 10th of June 2011 at the Hotel Divani Apollon Palace and Spa in Vouliagmeni, Athens, Greece. The theme of the Symposium was “Competencies Development and Challenges in Transportation”. The organization of the event was leaded by the coordinator of the L4S project, Deloitte Business Solutions SA with the co-organization and support of the L4S partners. The Symposium attracted around 100 international participants from the air and sea transportation and the academic sector as well as policy makers, first responders and civil protection organizations.
The participants had the opportunity to attend the research results of the project and participate in special workshops experiencing the L4S Simulations and tools. Moreover, two parallel special workshops were conducted. In the first workshop, facilitated by Professor Albert Angehrn, the participants experienced the “Mayday Mayday" Simulation Game. Multimedia means (video) were used during the workshop which analyzed in a systemic and constructive way the organizational barriers that occur during crisis management in the transport and business, proposing ways for the improvement of the organization’s Crisis Readiness.
In the second workshop, facilitated by the Professor of Information Systems of the ALBA Graduate Business School Dr. Nicholas Mylonopoulos, the participants who came mainly from the air transportation experienced the “Crisis Team” Simulation Game. The “Crisis Team” immersed the participants in the distressed work environment of the management of an air accident highlighting the significance of cooperation during this situation. During the ISCM 2011, the participants attended the keynote speaker of the Symposium, Professor Sanjay Sarma from the Massachusetts Institute of Technology (MIT, USA) presenting the key challenges in the field of transport security and the modern technologies to be implemented in the near future (sensors, RFID, etc). Moreover, top level executives from leading organizations such as Mr. A. Aravanis, COO from the Athens International Airport and Mr. E. Vordonis, Executive Director of Thenamaris Ships Management, performed speeches on crisis management in the transportation sector.
In addition, two panel sessions were held on the development of competencies development in the air and sea transport. The panels involved distinguished guests from international organizations and institutions specializing in transport such as FREQUENTIS (Austria), the Catholic University of Milan (Italy), the Port of Barcelona (Spain), the University of Greenwich (England) etc. Research projects on a crisis management and security that are co-funded by the7th Framework Program of the European Commission (BEMOSA, PANDORA, BRIDGE, E-SPONDER and CAST) were also presented during the symposium.
- Organization of the “L4S Panel” during the “8th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2011)”
The L4S consortium was represented at the 8th International Conference on Information Systems for Crisis Response and Management - ISCRAM 2011 by organising and conducting the Panel Session “Experiential models for developing crisis management competencies in sea and air transport” on 10 May 2011 in Lisbon, Portugal. The session was held at the auditorium of LNEC´s Congress Center with the participation of 40 persons. The aim of the panel was to contribute to the learning effectiveness of crisis response training by sharing the L4S knowledge and experience by discussing alternative experiential models for developing advanced collaboration competences at the tactical and strategic levels of crisis management.
- Organization and of six National Dissemination Events in the following countries: Greece (two events), France, Germany, Italy and Austria, inviting key stakeholders of the security and corporate community
- Organization or participation in 65 workshops and meetings in order to raise awareness of the L4S project’s objectives and outputs in Europe and North America.
Creation of Dissemination and Marketing Material
- Creation and continuous update of the L4S website and the “International Symposium on Crisis Management -ISCM 2011” website
- Design of the project’s brochure and of five leaflets for the L4S applications. All these publications were made available in English, French, German, Spanish, Italian and Greek.
- Development of three editions of e-Newsletters which were disseminated to all interested and participating individuals and organizations during the second year of the project’s life time, quarterly
- Creation of the L4S poster
- Publication of eight press releases on the international and national dissemination events
- Publications of eleven articles and papers
- Communication of the L4S project objectives, outputs and events in more than seventeen websites
- Tutorials disseminating the L4S Simulation Prototypes
- Production of videos regarding L4S work progress
- Production of twelve video based interviews
- Shooting videos and photos of the events
1.4.3 Exploitation of results
The main activity implemented under this task is the development of the Deliverable 5.4 “Final Exploitation Plan and Distribution Strategy Report”. The deliverable was based on data collected regarding the identified market for the exploitation of the L4S applications in each participating country. Meetings, discussions, questionnaires and interviews were put in place by the L4S consortium in order to gain valuable feedback on the deployment of the exploitation plan of the L4S outcomes. Moreover, all partners used their network of collaborators in order to identify potential users of the L4S applications. The main conclusions of the “Final Exploitation Plan and Distribution Strategy Report” follow.
During the second year of the L4S project, there has been made constructive progress and development on the identification of L4S exploitation perspectives and diffusion strategy, starting from the first effort of the “Initial Exploitation and Dissemination Plan” (D5.1) to the Final Version of “Exploitation Plan and Distribution Strategy Report”.
L4S project exploits the reality and the needs of organizations to help their managers and personnel in order to develop key skills and competencies in crisis management area through an innovative, highly involving, effective and easy deployable life-long learning service.
More precisely, L4S simulation-based learning experiences on Crisis Management reflect in a very realistic way crisis management and collaboration dynamics in modelled disastrous situations in air and sea transportations. Players are involved in role-playing crisis incidents in which they can validate and develop specific competencies through learning-by-doing in realistically modelled scenarios.
All L4S learning experiences’ applications are designed and developed in order to improve:
- the crisis management and collaboration competencies of European corporate personnel, decision-makers and academic learners
- the crisis readiness of organizations, both in the private and public sector Europe-wide
The L4S Learning Experiences service is comprised by the simulation games and learning-networking tools developed during the L4S FP7 project; built upon the L4S framework, an experience-based learning framework that address the effective development of crisis management and collaboration competencies for corporate and academic learners.
L4S Learning Experiences Portfolio, includes the following simulation games on air and sea transport crisis management: i) “IMPACT – The Crisis Readiness Online Simulation Experience” a set of 3-leveled simulation games, ii) “RECKON&CHOOSE! Air Simulation” and iii) “CRISIS TEAM”. Apart from simulation games the portfolio contains the WEB 2.0 advanced networking and sharing tool “CRISIS TUBE Leadership Learning Network” as well as the supportive online workshop tool “OWL4S”.
Regarding the impact that the L4S games bring both to individuals and organizations, a set of value propositions of L4S service has been deduced, assisting the easier comprehension of its core competencies.
Value Proposition 1
L4S Simulation-based Learning Experiences provide an innovative Learning-by-Playing edutainment service of advanced pedagogical value for the domain of Crisis Management
L4S Learning Experiences particularly aim at the development of the competences critical to the management of a crisis and an emergency. This is achieved by involving corporate and academic learners in experiencing and debriefing a number of Simulated Crisis Situations that:
- are based on realistic and complex scenarios to make sure that the insights, knowledge and competencies acquired are linked to specific not-cognitive- only experience [Crisis Management “Knowing –Doing Gap” Trap]
- virtually replicate challenging role-playing situations, which require decisions making, coordination, collaboration and real time knowledge exchange competences under conditions of time pressure, emotions, complexity, diversity of actors involved and other constraints/complicating factors [Specific multi-level Crisis Management Traps]
- provide an experience of operation in teams (online and face-to-face) addressing a challenging task to review principles, enabling and supporting more productive collaboration particularly in teams which are often distributed and highly diverse [Crisis Management Team Collaboration Traps]
- address the link between the behaviour of individuals and teams in crisis situations and the overall organizational and inter- organizational contexts in which they operate , as this can be more or less conductive to effective crisis management [Crisis Management Organizational Collaboration Traps]
Value Proposition 2
L4S Learning Experiences promote the collaborative inter-organizational knowledge exchange and development of enhanced quality standards and learning approaches related to Crisis Management
L4S learners in parallel with the L4S simulations, have the opportunity to get involved in Collaborative Learning Networks and Applications (“Crisis Tube”, “OWL4S”, “L4S Knowledge Community”) based on Web 2.0 platforms with key functions to:
- support the continuous developments and updating of the insights gained and competencies acquired [Crisis Management Competencies Obsolescence Trap]
- enable and stimulate dynamic experience and best/worst practices exchange [Crisis Management Tacit Knowledge Loss Trap]
- assist and motivate the collaborative identification of new target areas and to further improve critical competencies /capabilities of effective Crisis Management at the individual , team, organizational and inter-organizational level [Crisis Management Parochial Perspective Trap]
Value Proposition 3
L4S service enhances the competencies, attitudes and the organizational factors required to improve the crisis readiness of individuals and organizations
An important concept introduced by the L4S games is the one of “Crisis Readiness”, a concept which stimulates the reflection of managers and decision makers on the readiness of their organizations, their teams as well as themselves as individuals/leaders when it comes to handle crisis situations. The L4S simulation games provide players with a challenging experience of collaborating and taking decisions in distributed and diverse (multi-functional, transnational) teams addressing complex crisis situations. Moreover, they motivate the players to assess and gradually improve their own competences related to crisis management at the individual, group and organizational levels. In more detail, they provide the opportunity, methodology and tools to enhance the following skills of corporate and academic learners:
- Situation Assessment Competences
- Stress & Panic Management Competences
- Collaboration & Coordination Competences
- Inter-cultural Decision Making & Negotiations Competences
- Organizational, inter-organizational & general Communication Competences
- Design & Management of org. and inter-org. Crisis Management Systems
Value Proposition 4
L4S service can be applied to different types of organizations, irrelevant of their size, sector, status or location addressing their specific needs
The simulation-based experience provided by L4S service can be easily tailored to the needs of various sectors and customer’s environment as its pedagogical content is vast and can be easily adapted and if needed enriched according to the demands of different audience (i.e. strategical, operational, or tactical level) and organizational structures (individual, teams/groups, organizational and inter-organizational). Therefore, it is apparent how such tools are appropriate not only in the L4S application fields where they have been firstly deployed (transport) but, practically, in all domains where a potential risk component holds. For this reason, all of these tools apply to all organisations due to the large variety of emergencies and crises they may be exposed. Organisation’s size may just determine the impact reach (e.g. by involving more or less stakeholders) a crisis may provoke.
Concerning the identification of the best way of diffusing this type of high-value learning experiences Europe-wide, the consortium has conducted a market analysis, which produced detailed and relevant market information focusing primarily on the countries that partners are originating from. As far as the exploitation perspectives are concerned, all L4S partners have verified their commitment to continue collaborating on L4S outcomes in areas such as:
- Scientific research and publications on L4S multi-disciplinary domains
- Simulation games research and development
- Enhanced teaching and training in security and crisis management using simulation games
- Exploring areas of security that simulation-based capability development can be researched and obtained, while pursuing and designing suitable simulation-based learning experiences
The post-L4S deployment phase will be primarily based on individual exploitation of the project outcomes by each L4S partner. The individual exploitation plans of the L4S partners are categorized in three distinct types, according to their entity nature, core business and professional expertise. The exploitation direction can be either:
- Internal: Organizations provide the L4S Learning Experiences internally to its employees and executives, adopting the L4S applications portfolio in its internal executive training programs.
- External: Commercial entities distribute the L4S Learning Experiences as a service to its customers in various industries. Moreover, the Learning Experiences may be bundled with other existing business products or services.
- Academic: Educational and Academic institutions integrate in their curricula the L4S training applications and they can also provide executive and vocational training to corporations and organizations.
The individual exploitation plan of each partner is provided in the respective report (D5.4).
Immediately after the project’s closure a “Deployment Assessment Committee” (DAC) will be established comprising from one representative per partner. The objective of this committee will be to monitor and assess the exploitation phase of the L4S Learning Experiences as well as to exchange best practices of applying the L4S outcomes mainly on pedagogical and technological dimensions. Furthermore, a “royalty system” will be after the completion of the project established that will reflect a revenue sharing policy of the consortium for the commercial exploitation activities of the partners. A Commercial Advisory Group, from the business partners is formed, in order to monitor the royalty system and consult the commercial partners.
Last but not least, a sustainability plan of the external deployment phase –concerning the four L4S commercial partners- has been carried out. There, a financial and viability analysis is present for each of the four entities, complementing with key hypotheses and assumptions.
The L4S consortium has clear evidence that L4S Learning Experiences have a high potential for diffusion and impact through various exploitation streams, that offer to the L4S partners distinct but valuable high quality assets in their internal and/or external operations and to transportation and security stakeholders an innovative portfolio of advanced learning experiences on crisis management.
From a business perspective, on the basis of the work conducted, we believe that both the diffusion and the financial targets set can be achieved, with value generated at all levels – from the personnel and decision makers experiencing the value of the proposed simulation-based learning sessions, to the facilitators, the organizations’ representatives, the network of companies diffusing the L4S Learning Experiences Europe-wide, the designers of advanced crisis management learning simulations and the academic staff and learners that will experience these learning environments. There is a real opportunity to make significant impact.
A multi-level exploitation plan has been thus defined, and its execution is commencing straightforward after the project’s completion. The work done clearly indicates that the currently available and immediately deployable L4S Simulation Games and Applications can provide impact and visibility, along with the generation of a strong stream of revenue to support this growth. The longer-term strategy and vision is to set up an efficient Europe-wide B2B Channel for the diffusion of Game-based Learning Experiences.
Such a channel would:
- Be positioned to run in parallel and synergistic to traditional publishing or software diffusion channels, including consulting
- Achieve the mission of efficiently diffusing new Game-based Learning Experiences (developed by a variety of players) which fulfil quality standards set for L4S Simulation Games
- Aim at rapid extension to other relevant educational/impact areas. The development plan is to expand the basic project results and customise them to the needs/ characteristics of other Critical Infrastructures (energy, financial services, food industry, etc.) and throughout Europe and beyond
- Support the growth of a variety of players , increasing our involvement in the design, production and delivery of simulation games and game-based learning experiences.
This is an ambitious but, we believe, achievable long-term objective and vision. Its focus extends over and beyond the horizon addressed in this FP7 project.
List of Websites:
L4S Knowledge Community (http://L4S.fvaweb.eu)
The L4S website (www.L4S-Project.info)
The ISCM website (www.ISCM2011.eu )
