Final Report Summary - SEAHORSE (Safety Enhancements in transport by Achieving Human Orientated Resilient Shipping Environment)
Executive Summary:
SEAHORSE is a leap forward towards shipping safety achieved through technology transfer from air transport to maritime transport focusing on human factors problems in an innovative, integrated and multidisciplinary manner towards safer and more resilient shipping operations.
Within the Aviation industry having safe and reliable operations is critical in preventing accidents and mistakes that can potentially cause a huge loss of life and destruction. The influence of human factors on operational safety is one area were the aviation industry has led the way in terms of understanding and implementing tools, methodologies and systems to combat human error within a system.
The SEAHORSE project brings together an experienced, diverse and committed consortium from air and maritime transport sectors and world leading expertise, with the overall goal of tackling the issue of ‘Human Factors and Shipping Safety’.
By taking a pioneering and original approach the SEAHORSE consortium aims to achieve meaningful improvements in shipping safety by addressing human and organisational factors with the introduction of the following innovative concepts:
• Enhanced understanding and identification of underlying reasons for marine accidents and execution of gap analysis compared to air transport industry
• Identify the best practices in air transport managing errors and non-standard practices including the implementation of resilience engineering principles and develop the framework to transfer these successful practices to maritime transport to enhance the safety through managing the human and organisational factors
• New and innovative ‘workaround Management’ methodology for the identification and assessment of the negative/positive outcomes of the practice of crew performing workarounds.
• Identification and measurement of the functional demands of ship operations and crew needs, limitations and capabilities.
• Development of Multilevel resilience approach covering individuals, teams, multi-teams and organisations linked and interactive to make sure interventions to improve resilience at one level should have positive impact on resilience at a different level.
• Innovative validation through real case studies involving operational ships, full bridge simulators and training institutions/companies
• Innovative SEAHORSE virtual platform, which can be utilised to tailor the multi-level resilience principles to individual company needs, develop targeted implementation plan and training materials
• Practical guidelines will be available for the whole industry.
Through the SEAHORSE concept and the utilisation of some of Europe’s leading companies and institutions, both in the consortium and advisory board, in various fields including human factors research, operation, engineering and training in air and maritime transport; SEAHORSE has the potential to create a significant impact, at not only a European level but also an International one, in making the ship operation a safe, resilient, attractive and efficient environment.
SEAHORSE project clearly demonstrated that different transport modes can and should work together to share the best practices with practical impact on transport safety by adopting the technology transfer framework similar to one developed by SEAHORSE project.
SEAHORSE project outcomes/tools are freely available to shipping companies and maritime education/training institutions towards improving the maritime safety.
Project Context and Objectives:
PROJECT CONTEXT AND MAIN OBJECTIVES
While an increase in the transportation of goods worldwide by sea has been recorded, there has been a general decrease in the number of crew on-board observed. Through system automation, operational streamlining and financial constraints extra demands have been placed on the crew which has the potential to lead to situations conducive to human error and accidents. The modern seaman is expected to be multi-disciplined with a high level of technical skills, have broader deck officer responsibilities, manoeuvre the vessel he/she is working on at short notice and be prepared to work long hours with very limited days off and outside social contact. These working conditions and this environment are proven not only to be making seafaring a less attractive career option but more worryingly causing accidents.
Traditionally safety has been addressed both by designers and regulatory bodies such as the International Maritime Organisation (IMO) through structural, mechanical, electrical and technological solutions with the aim of minimising damage and prevention of loss of life and ships/floating structures. Furthermore, it has been often ignored that the human element of the maritime system has not been evolving in the same way that technology is developing; with the physical capabilities and the limitations of the human being overlooked. Addressing the human element in practical ship design and operation is a challenging task due to traditions and lack of knowledge.
The air transport sector, which is in many ways similar to the marine sector have been facing similar human and organisational factors that affect operational safety. However the airline industry has been managing these issues by approaching the same problem systematically and developing much more advanced methodologies and techniques that can be adapted to the marine industry while utilising the experience of air transport.
The SEAHORSE project proposes to address human factors and safety in marine transport by transferring the well proven practices and methodologies from air transport to marine transport in an effective, collaborative and innovative manner by adapting and tailoring aviation practices to the unique needs of maritime transport.
This will be primarily achieved by introducing the principles of resilience engineering and smart procedures methodology in an integrated framework which will result in multi-level resilience that linking individuals, team, multi-party teams and organisations in ship operation that ultimately enhancing shipping safety.
The SEAHORSE project’s aim will be realised through the implementation of the following objectives:
1) Identify the key human/organisational factors, which lead to operational successes and failures in marine and air transport and perform gap analysis in marine practices in comparison to air industry;
2) Investigate how errors and non-standard practices were managed successfully in air transport and check the feasibility of applying best practices and resilience concept adopted in air transport for marine to improve human/organisational errors and safety
3) Develop the Technology Transfer Framework from air to marine for successful implementation
4) Introduce a smart procedure methodology in marine operations to identify and assess non-standard procedures carried out on board ships to quantify the positive/negative effects in order to enhance overall resilience.
5) Develop and validate a multi-level resilience model and virtual platform as well as guidelines for marine transport which encompasses individual, team, multi-party and organisational resilience that linked and integrated. The SEAHORSE Platform will address
a) The crew needs and limitations that may affect their resilience with regards to navigation/operation of the ships (competency, physical & cognitive limits, attitudes, team composition)
b) System design, equipment and procedures which promote/add/remove, by design or accident, resilient and/or non-standard procedural behaviour
c) Shared situational awareness, leadership, organisational drift, insufficient/non-existing safety culture, team work
d) Suitable training materials to implement the multilevel resilience
6) Implement and evaluate the benefits of the SEAHORSE multi-level resilience tools through the comparison with traditional maritime safety methods and operations in training, simulator and actual ship environment.
7) Educate stakeholders within the marine transport industry of the benefits of resilience engineering and the SEAHORSE concepts.
Project Results:
MAIN S&T RESULTS/FOREGROUNDS
1. State of the Art: The ‘as is’ Picture
From the four high-level topics of inquiry in the state of the art (SOTA) results can be summarised as follows:
Stakeholders
The review of stakeholders highlighted the fact that the two transport sectors are very different. Both have a strong emphasis on safety but there are differences in how safety is managed and negotiated by the main actors. The maritime sector, in the category of ‘secondary users’, contrasts strongly with the aviation sector. The maritime sector has a much more broad and varied category of secondary users. Clearly managing inputs from all these stakeholders and ensuring the safety of a ship’s operation in these circumstances is not an easy task. Negotiation safety with so many stakeholders with varied agendas, as happens in the maritime sector, is a task which is complicated by training and regulatory issues.
Training
One of the key differences in relation to training concerns oversight, assessment and currency of skills/knowledge. The aviation sector can be broadly said to have better systems and procedures in place to oversee, assess and ensure the currency of personnel training. One particular example is HF training. Human Factors Training in the maritime sector is only mandated for certain grades of staff. And while most of the same topics are covered in aviation HF training as in maritime the fact that only certain grades receive the training means that its potential impact on the system functioning is reduced.
Regulations
Both the maritime and aviation sectors are highly regulated industries but it is clear that oversight and the interpretation and implementation of regulations are an area where the maritime sector could learn from the aviation sector. The lack of a mandatory quality approval system for Flag States is creating a big safety discrepancy between the potential and actual safety of the maritime system. While the maritime sector seems to tolerate this systemic safety ‘workaround’ the aviation system is far less tolerant of safety ‘fudges’ such as these. Indeed the EU and US aviation safety regulators have the power to ban airlines that fail to meet their stringent safety requirement from even entering European airspace. This ‘blacklist’ is publicly available and regularly updated.
Challenges
Clearly the maritime sector is facing many safety challenges. And while the aviation sector has had greater success in, and a longer tradition of, tackling safety issues there are safety opportunities for each sector. In relation to current challenges there are areas of convergence and divergence between the two sectors. Both the maritime and aviation interviewees have stated that paperwork and task load are very important challenges for their operations. Similarly both sectors see challenges related to the organisational context – specifically the lack of resources in the aviation sector which has an impact on competence (insufficient training) or task performance (lack of equipment, inadequate facilities etc.), though the industry is trying to manage this through audit plans and regular reporting. In the maritime sector one issue is the compartmentalisation of safety where it is viewed as separate from the operation and not related to everyday work. This leads to the notion that safety is somebody else’s problem and results in a diffusion of responsibility for safety. Meanwhile there is convergence in relation to multicultural crews. The maritime sector cites this as a significant challenge whereas the aviation sector states that it is not a safety challenge as standard operating procedures constitute a good solution for avoiding possible issues that can come up due to multicultural issues.
2. Review of Occurrence Analysis
D2.2 provides, for the first time, a comparative analysis of accident occurrence data in both the aviation and maritime sectors highlighting the commonalities and differences across a number of topics ranging from mandatory reporting, taxonomy, and standardisation to causal factors.
The main results are threefold:
Occurrence Reporting procedures
Aviation: Aviation has a standardised mandatory occurrence Reporting system as International Civil Aviation Authority, ICAO Annex 13 (Chapter 8) requires States to establish mandatory incident reporting systems to gather information on actual or potential safety deficiencies. This requirement is supplemented by further ICAO requirements which require aviation companies to have in place formal processes to collect, analyse and act upon feedback on hazards and risks in operations in order to support the implementation of mandated safety management systems.
Maritime: The maritime sector has not reached the maturity level of aviation despite the efforts within the IMO, and different authorities use different taxonomies to guide the collection of data. Additionally there are significant variations in the standards of reporting. This creates a significant problem with regards to the creation of a single maritime occurrence reporting taxonomy as well as a centralised database.
Availability and similarity of Data
In aviation reports of single accidents are publicly available in Agency databases (e.g. NTSB-AAR, European Central Repository). However, more in depth information on the contributing factors is classified and only accessible by authorized national agencies to ensure anonymity and confidentiality.
The SEAHORSE Consortium managed to obtain a comprehensive accident review from the UK CAA containing important information on human and organisational factors which could be considered comparable to the MAIB database used for the maritime analysis. A direct like-for-like comparison across categories and/or levels of human and organisational factors was not possible due to the fact that the CAA and the MAIB use different frameworks to categorise accidents and incidents.
Aviation and maritime are two transport sectors with different levels of safety maturity and it stands to reason that they have emerged using different classification schemes for accident and incident data. So, while a direct comparison cannot be made, the data were sufficiently rich to give us insights into the incidence of human and organisational issues in the maritime sector as compared to aviation.
Occurrence Analysis
Maritime
According to the European Transport safety Council, passenger fatality rates in shipping are 14.8 passengers per 100 million passenger hours and 0.46 per 100 million passenger kilometres. The data obtained from the MAIB (Maritime Accident Investigation Branch) was analysed under three levels: Underlying accident factor category, Underlying accident factor and Underlying accident sub-factor. It was found that the majority of the factors leading to accident is related to the human factor (74% of the underlying main factors) while 26% are listed under the technical category.
Aviation
The Global Fatal Accident Review 2002 to 2011 was carried out to provide a ten-year overview of worldwide fatal accidents involving large jet and turboprop aeroplanes engaged in passenger, cargo and ferry/ positioning flights (CAA, 2013). The corresponding on-board fatality rate for the same period was 22.0 fatalities per million flights flown or 12.7 when expressed as per million hours flown. According to European Transport safety Council, passenger fatality rate in aviation is 0.035 per 100 million passenger kilometres.
3. ‘Gap Analysis Report’
A total of 44 gaps were identified which impact safety in the maritime sector. This is the first such study clearly providing the gaps between the air and maritime transport with regards to safety. They vary widely and differ in terms of focus, and level of granularity. Below is the list of these 44 gaps grouped according to topic.
In total, 9 gaps defined under the “Stakeholders”, 7 gaps were identified under the “Functional Demands”, 5 gaps were identified under the “Training”, 7 gaps were identified under the “Regulations”, 4 gaps were identified under the “Current Challenges”, 1 gap was identified under the “Advanced tackling of HOF issues”, 5 gaps were identified under the “Operational Demands” and 6 gaps were identified under the “Socio-Economic Issues”.
4. Identified Solutions and Scope of Transfer
On the basis of the outcomes of the above process the solutions contained in the database were classified and divided into six categories, each one defined by specific criteria:
1. Most promising solutions: this category includes solutions not available (or not working) that are ranked as the most promising for transfer by at least 30% of experts, very much beneficial for maritime safety by at least 50% with a high impact on safety.
2. Promising solutions: this category includes solutions not widely available (or not working) that are ranked as the most promising for transfer by at least 20% of experts, very much beneficial for maritime safety by at least 50% with a moderate impact on safety.
3. Promising solutions to be further investigated: this category includes the solutions evaluated as promising in terms of safety impact but already existing in the maritime domain.
4. “Nice to have” solutions: those with less expected benefits for maritime safety;
5. Less beneficial solutions: those with no clear benefits for maritime safety
6. To be excluded from the transferability analysis all those solutions perceived as too specific for the aviation domain or useless for maritime safety.
Out of 166 solutions included in the database 73 were classified in the first four categories while the others were excluded. Within the first 4 categories, 9 solutions were classified as most promising, 6 as promising, 10 as promising to be further investigated and 48 as “nice to have”.
5. Database of Solutions (resilience resources)
The main outcome of this task is the Database of Resilience Resources, an Excel file that hosts all the identified resilience resources (solutions) implemented in air transport. The Database represents a first attempt of organising in a systematic way the aviation safety knowledge according to resilience engineering principles. All the solutions have been classified according to the “Area of resilience improvement” parameter, which could help the database users in the selection of all the solutions that, combined together in a proper manner, could support the system/organisation in facing a certain safety problem.
6. Feasibility Report for the Identified Solutions
The main outcome of the feasibility analysis is that all the shortlisted solutions can be considered ‘feasible’ in that they match at least four of the seven criteria identified. These seven criteria are: precision of terms of reference, similarity of terms of reference, costs/benefits evaluation, benefits, development costs, application level, processes impacted by the adoption
The main added value of the work carried out in task is that the methodology identified and followed proved an effective and efficient way of evaluating the feasibility of transferring solutions from one domain to another. This methodology can be used for any domain – it is not specific to maritime or aviation and can easily be adopted to drive the transfer of innovation (safety or otherwise) from one domain to another.
7. SEAHORSE Implementation Strategy
The methodology developed in this task is proven to be an effective and efficient way of evaluating the feasibility of transferring solutions from aviation to maritime. The in depth feasibility analysis clearly reveals that successful resilience solutions/best practices can not be utilised directly in maritime domain as ‘Off the Shelf’ approach does not work in this case. Therefore, particular attention has to be paid to the customisation process to ensure an effective adaptation of the solution to the new domain. With this respect, the experience related to the transfer to the maritime sector suggested some key elements that need to be taken into account such as:
• Local culture: this element can play a crucial role since it can affect the way in which the solution is implemented, especially in large organizations where different cultures may coexist.
• Decision chain: it is relevant to identify from the beginning the decision chain and the people responsible for the solution implementation.
• Target of the solution: People or areas of the organization that are the target of the solution should be clearly identified to take into account their specific needs in terms of effort and availability.
This strategy, although tested on specific cases, can be used for any safety solution the maritime sector would like to import from other domains. In fact the steps defined to drive the transfer process are independent of the vagaries of either the ‘departure’ or ‘destination’ sectors implicated in the transfer of innovation.
8. Database of Identified Workarounds on Board Ships
The deviation from SOPs is a phenomenon that affects the entire shipping industry and represents a threat to the safety of shipping. Due to a wide range of reasons, seafarers often deviate from SOPs in order to accomplish their tasks and complete their duties. In this report this phenomena is defined “workaround”. The consortium managed to capture responses in a very sensitive and controversial area and 259 workarounds and all these data are captured in Deliverable 4.1 database. Initial scan of the survey indicated that the most common workarounds are located in the areas of reporting paperwork, navigational rules and standards, and personal protective equipment.
The dominant reason of performing workaround is time constraints. When deviating from SOPs, crewmembers develop their own way to overcome the given difficulty, which will be defined in this report as a workaround. It is clear that the alternative devised way, the workaround, can be safe and provide a more optimal means of working; however, these alternatives can also bring risks.
9. Risk benefit analysis of performing workarounds
Detailed analyses of the SEAHORSE smart survey questionnaire data collected as part of the SEAHORSE project has been presented with the primary aim of establishing risks and benefits of workarounds as compared to relevant SOPs..
Survey also captured the workarounds in terms of commonalities, type of operation, onboard locations, reasons of opting workarounds, benefits and risks of workarounds as deemed by the reporting personnel. Most of the SOPs that were altered fell under the categories of ‘pre-requisites’ (defined as the formalities to be completed before commencing any work) and ‘recommended practice’ (procedures outlined in the documentations to undertake a given task). Avoidance of ‘personal protective equipment’ was also very common. Interestingly, more than 77% reported workarounds were being followed by other seafarers as well. Though, engineering cadre was found to be a little stricter on following the SOPs on as is basis. Workarounds have been adopted by the respondents, practically, at all onboard locations.
As such, ‘time constraints’ was the most wide spread reason identified by the seafarers. ‘Insufficient manpower’ and ‘crew fatigue’, appeared to be other variants of the lack of time, were also very common. ‘Physical constraints’ was the second most common category that could be linked with the design aspects of the ship and its equipment. As far workarounds benefits go, convenience was considered by respondents as the dominant factor followed by the time efficiency. The two other dominant reasons for altering the SOPs were the potential improvements in performance and safety aspects of SOPs. It was also interesting to note that in a few cases, the respondents themselves found no specific benefits of following workarounds.
10. SEAHORSE Smart Procedures Methodology
The SEAHORSE Smart Procedures Methodology, developed within the SEAHORSE Project, with the ambitious aim of providing the maritime industry with a robust method for the collection and assessment of workarounds practised on-board vessels.
The unique contribution of this methodology is the development of a method to capture workarounds practiced on-board vessels.
A questionnaire was developed to provide a structured means of capturing information about workarounds, which has not previously been done. The assessment of the alternatives is performed based on a pre-existing method for ranking and requires the involvement of a group of experts. A set of attributes is provided in order to support each reviewer in the assessment of the workaround. The assessments carried out by each expert are aggregated by factoring in the importance of attributes and experts through weightings. Crucially, the methodology closes the review loop by providing feedback of the outcome of the review to the original seafarer who submitted the workaround.
The study clearly highlights that the maritime industry still has significant issues concerning safety. Good practices should be formalised within existing procedures while bad practices should be prohibited. Furthermore, in many cases the reason why they perform this workaround is more important than the workaround itself.
Automated Web–based tool was developed to utilise the procedural improvement methodology allowing the suggested workarounds to be assessed very quickly while maintaining the anonymity of the proposer.
11. Results of the initial validation study and modifications
In order to validate the methodology, a one-day workshop was held in Rotterdam in 2015. The workshop consisted of 40 expert participants. A total of 7 workarounds had been chosen to be assessed by 4 different groups
Paper-based booklets were distributed containing the review forms and each participant reviewed the workarounds. All data was manually entered into the computer system at a later date and analysed. The participants also had a chance to test an initial version of the online SEAHORSE platform; each participant had 15 minutes to trial the system. Feedback was collected from the participants throughout the workshop and a final survey was administered to them at the end of the reviews.
Key Findings
The result of a workaround varies between 0 and 1. In total, 35% of the workarounds were found to be better than the SOPs. This creates further room for developing SOPs in order to enhance maritime safety. Treating the bad practices as a learning opportunity will lead companies to understand the reasons behind performing risky workarounds.
Everybody can access and use the tool either submit a suggestion or perform the assessment without gathering around the same table. The review of the workaround is calculated and the results are automatically disseminated to all the employees of the company electronically. Initial mock-ups were drafted to build up the architecture of the software. This methodology, embedded within the software, is positioned to improve SOPs in organisations working in the maritime industry and therefore lead to improved shipping safety. A user guide is available together with the tool to assist the users to utilise the tool effectively.
Such a system is required to allow the identification of unstandardized practices on-board vessels and to allow these to be objectively scrutinised by expert reviewers. The system allows opportunities to improve safety on-board vessels not only by making persons aware of unsafe practices but to also by making improvements to the overall design of the SOPs, improving the way in which they are executed.
12. Report on Multi-level Resilience Model
The aim of Task T5.1 was to define what human oriented resilience is and how it is assessed within an organization, so that the effects of resilience solutions or tools developed within the consortium can be assessed.
Based on the fact that existing accident analysis methods do not possess all the desired characteristics to measure the resilience of a maritime socio-technical system (STS), the Resilience Assessment Method (RAM) was developed in WP 5 of the Seahorse project
The assessment tool consists of a resilience questionnaire, a multi-level resilience matrix, and a set of resilience solutions that organizations may adopt to improve their resilience. Any maritime organization seeking to improve its resilience to unforeseen disturbances or challenges should start by identifying which safety critical operations/ functions (in different operational phases) should be analysed with the resilience assessment tool. Examples of safety-critical functions are mooring, navigation, lifting, unloading, etc. After this functional decomposition has been completed, the resilience questionnaire is used by the organization to describe the function’s operational demands, outcomes, resources and coping strategies. The questionnaire assesses the quality of the available resources and strategies in order to achieve the required outcomes, given the operational demands. The output is shown in a multi-level resilience matrix, with rows depicting the different levels in an organization (individual, team, organizational), and the columns showing the scores on the four essential resilience abilities: anticipation, monitoring, responding, and learning. An organization scoring low on, for instance, team learning, may then be advised to implement a human factors training program in order to strengthen this aspect.
The method was applied retrospectively to six maritime incident scenarios. The questionnaire was applied to all scenarios. Initial results show that the RAM is a promising method. The questionnaire offers the possibility to gather more information in a structured way. The validation was, however, done by a single person. This means the inter-rater reliability has not been tested yet. To overcome this, the current study should be repeated in future studies and a comparison of the results should indicate whether different researchers will get the same results or not.
One of the goals of this study is to develop a tool that is able to analyse accidents, the current (resilience) state of an STS and to help design new STSs. However, the RAM has not yet been tested in its ability to measure the current state of an STS and design new STSs. In order to investigate if the RAM is able to analyse current STSs, it should be tested whether the method is able to identify latent variability within a system. Similarly, for designing an STS, trying whether the method has this ability is a good way to test it.
The multi-level resilience assessment tool was further developed and validated in an interactive workshop with stakeholders during the public SEAHORSE meeting in Rotterdam on October 13, 2015.
More than 40 maritime partners participated by providing suggestions of currently-used resilient solutions to particular challenges the maritime industry is faced with, for instance, language problems or reduced crew sizes. The workshop was highly successful in generating these solutions, which in the course of the project will be implemented in a Virtual Platform and made available, together with the resilience questionnaire, to the maritime industry, as well as any other interested parties.
In order for the operational procedures on board to enable successful performance of safety critical operations in harsh environments such as the maritime system, resilience is required. Therefore the SEAHORSE project’s approach will be to implement resilience engineering principles in an integrated and innovative manner, taking into account the knowledge generated and experience gained by the aeronautical industry. By utilizing the SEAHORSE project’s multi-level resilience assessment tool, RAM, the user will be provided with new resilience resources to prevent a decrease in system performance, allowing the system to return to baseline performance much more quickly and display greater resilient behaviour.
13. Report on Multi-level Resilience Assessment Methods
The objective of T5.2 was to support the application of the Multi-Level Resilience Model through the development and specification of measures to assess the impact of resilience solutions on safety, efficiency and the operation. The strongest link with earlier work packages is that with WP3, in which solutions from the aviation sector were identified to improve the resilience of the maritime sector. The task for T5.2 then was to systematically develop measures to assess the impact of their implementation as specified in the D3.3 “Feasibility Reports for the Identified Solutions”.
The practical objective of T5.2 was to provide end-users and operators with practical tools to be able to evaluate the effectiveness of the implementation of resilience resources and understand if the intended safety goals are actually achieved. Task 5.2 developed the metrics to assess the impact of applying resilience resources: what and how to measure the performance outcome for a maritime transport system as a result of a resilience intervention. The focus of the task was tempered by the Health, Safety & Environment Agency (HSE) guidance on the use and deployment of safety indicators – stating that they should not be overly complex, and should not overburden safety managers who already gather safety data (HSE, 2006). As such the SEAHORSE Resilience Impact Assessment metrics focus on “quality not quantity” to ensure, insofar as possible, that busy management teams have the correct, critical indicators to measure the impact of implementing a resilience solution, rather than having too many unnecessary indicators – adding to an already high workload.
The aim of this task was to identify the “metrics to assess the impact of applying resilience resources” and “thresholds” for these metrics so that the implementation of the resilience solutions identified in T3.4 can be measured and evaluated. The solutions were evaluated as part of WP3 in order to integrate them into the existing safety management system and all the organisational aspects, and characteristics required for the implementation of the solution were identified and taken into account for the implementation strategy definition. The task focused on the solutions selected within the project but also provides generic impact assessment measures that can be used for any resilience solution implementation – thereby ensuring that the SEAHORSE methodology can be fully adopted in the future. The four solutions selected were as follows:
1. HF Training
2. Pre-flight Checklist
3. Just Culture
4. Mandatory Safety Reporting
For each one of these solutions, a maritime-version is being developed and implemented addressing the specifics of the maritime sector’s operational and safety needs. Important factors such as management support and feedback, the degree to which users can perform with the newly acquired competences and a focus on the actual work in its context are important transfer enablers and are included in the implementation plans developed for each solution.
Task 5.2 ‘Multi-level Resilience Assessment Methods’ had to produce something which companies could easily understand and use. Safety managers already have sufficient workload so the output of the task should not have been overly-complicated or requiring additional training as highlighted by the HSE (2006). If substantive training were needed to get the knowledge across to safety managers and personnel it would fail in its objective. Nonetheless, the task of evaluating the impact of a safety solution is not a facile one and any framework should not be so simple as to be devoid of meaning or power. This resulted in a requirement for a comprehensive evaluation framework which had an easily-understood organising principle. Kirkpatrick’s model was chosen as the model to inform the assessment framework.
1. Reaction - Assessing impact through the elicitation of perceptions and attitudes of those impacted directly by the resilience solution
2. Learning - Assessing impact through the elicitation of new knowledge and learning of those impacted directly by the resilience solution
3. Behaviour - Assessing impact on the behaviour of those impacted directly by the resilience solution
4. Results - Assessing impact on the results that the organisation uses to evaluate its performance
Implicit in each category of analysis is an appropriate enquiry method. For example ‘Reaction’ and ‘Learning’ are best assessed by talking to people and getting their thoughts – are they happy with this solution, does it make their job easier, have they learned anything through using the solution? Assessing impact of ‘Behaviour’ means you want to see if people do things differently so you would need to observe behaviour in a structured way. Finally assessing the impact on ‘Results’ would have a different focus – more on KPIs (Key Performance Indicators) and organisational data. There is also an implicit focus of each category. ‘Reaction’ and ‘Learning’ focus on the individual, ‘Behaviour’ can be the individual or the organisation, whereas ‘Results’ is a more organisational, macro-level category. As such the following three-pronged approach was developed:
• Assessing impact through the elicitation of perceptions and attitudes of those impacted directly by the resilience solution (via a questionnaire),
• Pairing these attitudes with behaviours (either physical real-time behaviours) or artefacts that demonstrate a past behaviour (via targeted observations), and
• Collating (and supplementing) organisational data on accidents, incidents and KPIs which focus on the ‘hard’ outcomes achieved (via document analysis)
The Resilience Impact Assessment Framework has three analysis methods and four levels of analysis. The questionnaire (targeting perceptions and attitudes) focuses on ‘Reaction’ (what do you think of this solution), ‘Learning’ (what have you learned from this solution), ‘Behaviour’ (what do you do differently as a result of the solution), and ‘Results’ (what has this meant in terms of your job/role). Observations and audits (linking attitudes and learning to behaviour) target just the ‘Behaviour’ category from Kirkpatrick’s framework (See Error! Reference source not found. ).
This category is looking at collective or individual behaviour by observing it directly (to see how people are working, are they doing things correctly) or by observing indirect artefacts (or evidence) of behaviours – such as did the company print off the Just Culture statement and display it in a prominent place? Organisational data is targeting the accident and incident data that the organisation will already collect and supplementing this with some additional metrics which focus on results.
In order for organisations to be able to plan, implement and sustain the Impact Assessment process it is necessary to set out a roadmap – detailing who manages the process, how the process runs, who is involved and in what order the assessments take place. The process starts with the Questionnaire. This process is managed by the Safety Manager or equivalent (such as DPA - Designated Person Ashore) who should distribute the questionnaire(s) and ensure that all staff should complete the questionnaire confidentially. Ideally this will cover all levels of the organisation – not just those that are immediately impacted by the solution in question. This is to target any unintended consequences or oversights for other roles not directly impacted. Then the Safety Manager should begin the process of the Observations, targeting real-time behaviour of those using the new resilience solution, and looking at documents or evidence of past behaviour (such as processes and systems being put in place and managed effectively). Once this is complete the focus shifts to the analysis of Organisational Data which is again managed by the Safety Manager and relates to the interrogation of existing safety data in the organisation and the supplementation of this with additional analyses targeting safety, productivity and costs.
These three assessment methods are applied to each of the four selected SEAHORSE resilience solutions: HF Training, Pre-flight Checklist, Just Culture, Mandatory Safety Reporting - and will be tested in WP6 and WP7 through the implementation of solutions. The focus is on these solutions and there are specific questions targeting the specific impact of each solution – given that each solution is different from the other. Having said that, there are ‘generic’ questions and topics which apply to all solutions. This is intended to allow for the assessment of impact of future resilience solutions which may be developed once the SEAHORSE project is finished.
The metrics, the thresholds, and the methodology for managing the impact assessment process specified herein ensures that maritime operators can fully evaluate the impact on their operations as a result of the implementation of the SEAHORSE resilience solution (on safety, on efficiency and on the overall operation). The inclusion of generic metrics ensures that the SEAHORSE transfer methodology is fully sustainable in that it can be carried out in the future for other resilience solutions and the impact of their future implementation can also be assessed – thereby ensuring the legacy of the SEAHORSE project.
14. Results of the Initial Validation Study and Modifications
One of the tools incorporated in the Virtual Platform is the Resilience Assessment Tool. This is a questionnaire assessing a company’s resilience. In order to provide recommendations (‘solutions’) based on the questionnaire results, it was necessary to validate and amplify the solutions developed in WP3 with maritime solutions for particular challenges faced by a range of maritime companies. This was accomplished in the Public Workshop held in Rotterdam on October 13, 2015. The resulting solutions were incorporated in the Resilience Assessment Tool and became part of the Virtual Platform.
The SEAHORSE Virtual Platform was subject to an Initial Validation during the 8th Technical Meeting (TM8) held from the 7th of March until the 9th of March 2016 at the Istanbul Technical University/Maritime Faculty. The first step of the validation design process was the identification of the maturity level of the SEAHORSE Virtual Platform. Once it was identified, the validation objectives were defined and the validation approach was chosen. The overall validation design process was based on the European Operational Concept Validation Methodology (E-OCVM) [1] which is a framework widely used in the aviation domain.
The initial validation consisted of three parts: (1) validation questionnaire; (2) open discussion; (3) industry stakeholder workshop. As part of the validation questionnaire, individual project members of the SEAHORSE consortium including maritime experts answered questions on the clarity, feasibility, usefulness, and completeness of the virtual platform. In the open discussion session, facilitated discussions were carried out, aimed at identifying strong and weak points of the virtual platform. In the industry stakeholder workshop, an in-depth session was conducted during which the entire Resilience Assessment Tool was completed by three participating maritime companies.
The Virtual Platform, in its first version, was viewed by most respondents as fairly complete, logical, useful, and feasible. The Resilience Assessment Tool was seen by many as a powerful way of getting insight into the current state of a company’s resilience, as well as a good way to convince management to invest in safety.
Of course, there were also suggestions for improvement as well as some sceptical remarks about the feasibility of the tools incorporated in the Virtual Platform. These suggestions and remarks have been dealt with in subsequent versions of the Virtual Platform. Many comments and suggestions were aimed at the way individual items were phrased; these items were subsequently modified accordingly or altogether deleted. Other suggestions dealt with the way the tools were structured overall or the way they had to be filled in. Most of these comments could also be addressed.
An important issue that arose during the Technical Meeting in Istanbul was how to incorporate the various databases into the Virtual Platform, for instance, the crew database, ship database, and company database. After a long deliberation and weighing of various options, and considering the scope and feasibility of the development of the Virtual Platform, we decided not to link directly the Virtual Platform with existing databases, as this would, first, be too prohibitive in terms of development effort, and, second, not yield any additional insights into the resilience of a company.
We therefore took a two-pronged approach: first, by looking bottom-up at what currently available databases offer in terms of information, and, second, by looking top-down at what additional database information needed to be obtained in order to strengthen the items in the Resilience Assessment Tool.
The resulting set of questions was added as an additional layer to the Resilience Assessment Tool. These questions should be answered by respondents prior to filling in the other items. Answering the questions requires consulting various company and ship databases and thus provides a baseline for answering the subsequent items.
15. SEAHORSE Virtual Platform for Resilience Design & Assessment in Maritime Transport
The SEAHORSE Virtual Platform is a functioning prototype which gathers all the research and models developed within the SEAHORSE Project. It represents a virtual space where maritime companies can find different types and levels of assistance for maintaining and improving their resilience structure and enhancing safety performance. With its multi-dimensional and multi-level operational concept, it seeks to achieve the ambitious goal to support maritime companies from high level decisions on resilience matters to more practical and specific operational aspects. In particular, its aim is to provide shipping companies with a comprehensive support in their path towards safety and resilience, where “comprehensive support” stands for:
• Resilience assessment and grading;
• Identification of gaps in the resilience structure;
• Identification of weaknesses and/or non-compliances that may lead to errors and accidents;
• Identification of resilience resources and solutions and decision support in order to address the identified gaps, weaknesses and/or non-compliances;
• Provision of practical guidelines for the implementation of the selected solutions;
• Provision of structured assistance during the implementation process;
• Ex-post assessment of initiatives undertaken;
• Standard Operating Procedures assessment and re-design;
• Collection and assessment of workarounds practised on board vessels;
• Comparison of assessed workarounds and SOPs and identification of the most effective and safe ways of working;
• Audit of the crew quality at both individual and team levels;
• Identification of strengths and weaknesses of the crew, training needs and suitability for specific types of ship.
In practical terms, the Platform accommodates four tools, each of which is made up of three layers connected and integrated to each-other (input, database and output layers):
1. Resilience Assessment Tool (RAT): this tool aims to support end-users in the assessment of the company level of resilience and in the identification of gaps/weaknesses/non-compliance that may lead to errors and accidents. It also provides tailor-made safety and resilience solutions together with some practical guidelines for their implementation;
2. Impact Assessment Tool (IAT): this tool intends to provide the end-users with a practical support for monitoring the implementation process of safety/resilience solutions and assessing the impact of the initiatives undertaken;
3. Procedure Improvement System (PIS): this tool aims to provide the maritime industry with a concrete support for the collection and assessment of workarounds practised on-board vessels. It allows users to capture workarounds performed by seafarers within a company, assess them and compare them to SOPs in order to find the most effective and safe way of working;
4. Individual Crew Quality Audit Tool (ICQAT): this tool intends to support end-users in auditing the crew quality at both individual and team levels and thus in identifying strengths and weaknesses of the crew, training needs and suitability for specific types of ship.
From a technical standpoint, the SEAHORSE Virtual Platform is modelled with a web-based solution exploiting the WordPress framework and can be deployed in the existing IT infrastructure of the end users or, alternatively, outsourced to external services. Indeed, these are capable of running either together or as standalone instruments.
The SEAHORSE Virtual Platform can be considered as a functioning prototype positioned at the end of the pre-industrialisation phase. Indeed, with the typical limitations of a system at this maturity level, the Platform represents a software ready to be installed into a maritime company. Furthermore, this prototype has an open architecture which means that it is capable of accommodating other tools and guidelines as the interests of end-users require.
16. SEAHORSE Multi-level Resilience Tool
This chapter’s aim is to provide a comprehensive overview on the concept of the Multi-Level Resilience Software (MLRS) and to bring to light the added value that the SEAHORSE Project has generated for the entire maritime industry through the development of this software.
The MLRS represents a substantial part of the broader SEAHORSE Virtual Platform where all the research and models developed within the Project are gathered. This software is the part of this prototype developed on the purpose of providing maritime companies with a comprehensive support in their path toward resilience. Its operational concept comes and develops within the new research area of resilience engineering where resilience is conceived as the ability of system to cope with and manage demands, with the help of available resources, such that performance is sustained or even improved (Kamphuis et al., 2013). The main goals of this software are summarised in the below points:
• Resilience assessment and grading;
• Identification of gaps in the resilience structure;
• Identification of weaknesses and/or non-compliances that may lead to errors and accidents;
• Identification of resilience resources, solutions and decision support in order to address the identified gaps, weaknesses and/or non-compliances;
• Provision of practical guidelines for the implementation of the selected solutions;
• Provision of structured assistance during the implementation process.
These six points bring to light the capability of the MLRS of assisting end users in developing, assessing and maintaining a resilience structure that is tailored to their particular needs and operational requirements. It is a virtual space where maritime users can embark an assisted restructuring process of their companies’ resilience structure. This process, metaphorically called ‘resilience journey’ in this document and involves everybody in the organisation who is somehow related to the subject of the analysis, regardless of the roles played in their company. Workers are usually asked to report their perceptions about the safety needs of the organisation and/or to provide their feedback after the implementation of a new safety intervention. Broader is the role of managers who are also supposed to further analyse the feedback provided by workers, decide the interventions to implement and assess the impact of the initiative undertaken over time.
Three are the main sections of the software which are in turn composed of different sub-sections.
1. ‘Resilience Assessment’ – It represents the starting point of the resilience journey and includes:
a. The Resilience Assessment Tool (RAT) which is a diagnostic instrument for the determination of organisational strengths and weaknesses with respect to resilience;
b. The Resilience Assessment Results which is the sub-section where an overview of the status of the resilience structure is provided and a tailored set of implementable solutions is advised.
2. ‘Solution Implementation’ – It is a sort of a Control Panel where users (managers only) can enable the passage from the resilience assessment phase to the impact assessment one.
3. ‘Impact Assessment’ – It represents the final step of the assessment process and includes:
a. The Impact Assessment Tool which is an instrument for gathering information in a structured way about what the workers think has changed as a result of the implementation of a specific resilience solution;
b. The Impact Assessment Tool Results which is the sub-section dedicated to the review of the results of the Impact Assessment Questionnaire;
c. The Impact Assessment Metrics which intends to provide the user with a practical support for the quantitative analysis of the impact produced by the implementation of one or more of the proposed resilience solutions.
The resilience journey so far described might appear as a strict linear process. However, the fact that it is composed by consecutive steps does not mean that the MLRS offers a rigid and one-way path. In fact, the MLRS is a flexible system capable to provide a dynamic assistance. All the sections of the software are conceptually designed to run simultaneously, thus the user could work at the same time on different steps of the resilience journey.
The implementation of the conceptual Framework into an automatic software represents a significant leap forward for the provision to marine companies of a practical support for the enhancement of resilience. The major benefits that the MLRS would bring to a maritime company are summarised in the below points:
• Safety Management would be able to perform a resilience assessment whenever and wherever desired;
• The software allows the involvement of important stakeholders/contractors in the assessment process;
• It provides a realistic reconstruction of the resilience structure since it starts from worker’s perceptions and concrete operational needs;
• It gives an immediate understanding of the wellness of the organisation;
• It provides safety personnel with justifications (quantifiable data) for their safety initiatives and proposals of new investment and it represents a support for decision-making;
• It would bring improvements in the transparency of the management activities;
• Indirectly, the usage of the MLRS would generate positive impacts on the organisational safety culture.
The MLRS is a software ready to be installed into a maritime company. However, as any prototype, it has some limitations on both the technical and conceptual sides. The possibility of benchmarking within the maritime industry seems to be the capability of most interest to end-users. The non-implementation of this feature was due to the fact that it requires higher maturity level, hence it goes beyond the scope of the SEAHORSE Project. However, since the benchmark functionality would significantly increase the applicability and usefulness of the software, this improvement is advised as one of the priorities for further development activities.
17. Validation scenarios
The preceding work packages to the present one, Work Package 6, develop a number of outputs. The purpose of this document is to present 12 validation scenarios to be used to test these outputs to ensure the validity and quality of these solutions. The validation scenarios are fundamental to the project. Well-designed validation scenarios will lead to a higher level of credibility and trust in the solutions, as well as certainty in the quality and applicability to maritime organisation.
The validation scenarios are as follows:
Safety Climate Assessment in Shipping Companies: This validation scenario’s purpose is to test the surveys developed under the safety culture assessment framework. The surveys were carefully developed with an interdisciplinary group to ensure that they capture the right information to conduct a comprehensive analysis. A detailed literature review was performed not only in the maritime industry but also in other industries to identify the requirements of an appropriate safety culture questionnaires and assessment methods. The surveys will be distributed to three different shipping organisations, of varying sizes and specialities and conducting extensive analysis on the survey results.
These companies were chosen from different operation types in order to address all aspects of different operation types. The sizes of the companies are given below:
• Container Shipping Company – 300 Employees
• Bulk Carrier Company – 87 Employees
• Tanker Shipping Company – 2220 Employees
The questionnaires were designed to be accessible online for a month. The total of 4 month is required to validate the safety climate assessment and improvement framework is from scratch. The required time to finalize validation can be broken down as below:
• Information about the planned safety climate surveys, preparations within the company – 1 month
• Distribution and collection stages of the survey – 1 month
• Analysis of the results and validation 2 month
SEAHORSE Procedure Improvement System in Calmac: This validation scenario’s purpose is to test the Procedure Improvement System. Implementing the system in the organisation, the reviewing and results aspects of the system are to be validated. This will be done through the processing of 20 existing cases.
SEAHORSE Procedure Improvement System in Danoas: This validation scenario is to test the Procedure Improvement System, it is implemented within the shipping organisation Danoas.
Human Factors Training in Jumbo: Human factor training is to be tested in Jumbo. The training is to be delivered through training modules at Jumbo’s office.
Standard Operating Procedures Development for Jumbo: This validation scenario is designed to test the Standard Operating Procedures Best Practiced Guidelines. It is done by using the guide to turn an existing SOP into the correct format.
Standard Operating Procedures Development for Sener Shipping: The SOP Best Practices Guide is to be tested in a simulator setting.
Pre-Departure and Arrival Checklists: The checklist is to be validated in Calmac who will implement it within their organisation.
Mandatory Safety Reporting: The validation scenario tests the Mandatory Safety Reporting in a tanker company.
Resilience Assessment: The resilience assessment is tested through expert workshops.
Impact Assessment: The Impact Assessment tool is to be validated through expert workshops.
Just Culture: The Just Culture solution is to be tested in through workshops. The workshops will comprise of a set of experts.
Observation Study in Calmac: This validation scenario is to test the methodology developed for conducting observation studies. The methodology is to be validation by conducting an observation study on-board two vessels of a ferry operator.
18. Task Based Validation Report
Achievements
Checklists were validated both in operational settings (CALMAC, Chapter 2) as well as simulator settings (ITU, Chapter 3, SAMK, Chapter 7). Standard operating procedures were validated on the basis of a SOP Best Practices Guide (SU/KAHN, Chapter 6). Human Factors training was given by ESM to KAHN and described in chapter 4. Mandatory safety reports and safety bulletins were validated by means of consulting end users and maritime experts (APA, Chapter 5).
Overall, the conclusion is that methods and strategies from the airline industry can be successfully implemented in the maritime sector. Given existing regulations, some of these methods have been mandatory for a long time. However, we also discovered that the maritime sector has its own characteristics that sometimes impede the absorption of methods and strategies from the airline industry. For instance, the fact that bridge layouts are not as standardized as they are in cockpits, may lead to less uniformity in the design of checklists. It should also be remembered that proper design of checklists (or SOPs) is a demanding exercise and that the maritime sector can learn a lot from the aviation sector in this respect (CALMAC enlisted the help of an experienced Airbus captain to help in the design of the checklists). As another example, human factors training may be provided to maritime personnel, but the contents of the training should be geared to the maritime sector and should not be copied from the aviation sector.
Value added to SEAHORSE
This task contributed to the overall goal of the SEAHORSE project by demonstrating the value of several solutions that have been found to enhance safety in the aviation sector and by customizing them for the maritime sector.
19. Validation report: SEAHORSE “Procedure Improvement Methodology”
The SEAHORSE project has developed a wide range of outputs that can be used to support shipping organisations to better improve the safety and resilience of shipping. One of the outputs is the Procedure Improvement System, a tool that supports the collection, assessment and review of SOP suggestions provided by employees of a maritime organisation. At present, the maritime industry lacks a rigours and robust means to collect and review unstandardized procedures. The developed Procedure Improvement System provides a robust and repeatable means to review unstandardized procedures.
The purpose of this report is to present the validation of the SEAHORSE Procedure Improvement System; the validation is conducted in two distinct ways. The tool was tested by Danaos and by CalMac. Danaos tested through a full-scale validation, with the Procedure Improvement System augmented such that it can be used offline. CalMac validated part of the Procedure Improvement System through the testing of a number of cases.
First, the tool was tested in Danaos for a period of four months on 33 vessels. Over the four month period, the Procedure Improvement System was validated based on a number of indicators: 1) The Reaction Indicator, 2) The Learn Indicator (LI), 3) The Behavior Indicator (BI) and 4) The Efficient Indicator (EI).
The Procedure Improvement System was tested in 33 vessels in the fleet of Danaos (container ships). Crewmembers submitted 47 alternative controls to be evaluated to the tool. In total, 30 out of 47 had to do with operation on the bridge of the ship and the rest 17 for other.
Secondly, the tool was tested on 20 cases in CalMac, training was given to the team in CalMac on how to use the system. Initial engagement with CalMac for the tool began in June 2016 and the final feedback was collected in October 2016. The team selected 20 cases to be reviewed, these were a mixture of cases taken from their current system and some developed based on experience of the team. Reviewers were appointed and accounts were created for each of the reviewers. All four reviewers were selected to review each of the cases.
Within CalMac, the 20 cases were successfully reviewed, each case was reviewed within two weeks. The system was shown to be able to be used to successfully review a set of cases. In comparison with expected results, four of the 20 cases were identified as not being correct in the opinion of the head of the team. While the tests returned results for all of the cases, it was noted that for some there was only a marginal difference between the suggestion and the SOP and it was expected and desired to have more significant differences between the suggestion and the SOP. Overall, the suggestions were reviewed far quicker than their current means. The system was felt a good way to meet organisation aims of improving safety and meeting requirements of the regulatory bodies. Unfortunately, some key issues were identified as limiting the organisation from fully implementing the system. These include: offline functionality, issues with non-electronic Safety Management System and the time needed to make it fully electronic and lastly developing a ‘bank’ of experts to call up as needed for reviewing.
20. Validation report: Multi-Level Resilience Model for Maritime
The overall goal of the EU FP7 SEAHORSE Project is to improve the safety and resilience in the maritime sector by focusing on Human Factors. To achieve this, the project used a resilience engineering approach. To this end, a Multi-Level Resilience and Impact Assessment Models were developed, and, based on those models, the Resilience Assessment Tool (RAT) and Impact Assessment Tool (IAT) . These two tools were then integrated into the Multi-Level Resilience Assessment Software. This deliverable describes the validation activities of the overall resilience Framework which took place during the 9thTechnical Meeting in Athens and in a successive dedicated session with captains of industrial partner KAHN.
The RAT is a questionnaire that assesses a maritime company’s resilience at four levels: multi-party, organizational, team, and individual level. At each level, four cornerstones of resilience are assessed: monitoring, responding, anticipating, and learning. Each of the cornerstones, at each level, is assessed by multiple self-report questions. The questionnaire has been validated on a number of incident scenarios and during several workshops with maritime stakeholders (as described in earlier WP5 deliverables). The RAT results in a diagnosis of a company’s resilience structure. The diagnosis results in the recommendation of particular resilient solutions that maritime companies may adopt in order to enhance their resilience.
The IAT aims to enable maritime companies to assess the impact of implementing resilience solutions after having implemented such solutions.
The first Task Based validation activity of the SEAHORSE Multi-Level Resilience Framework was carried out during the 9th Technical Meeting (TM9) held from the 30th of May until the 1st of June 2016 at the office of DANAOS Shipping in Athens. The workshop contained a task-based validation of the Resilience Assessment Tool (RAT) and Impact Assessment Tool (IAT) by means of collecting end-users’ feedback on three dimensions: content, usability and applicability (referred to from now on as the ‘Validation Workshop’). It consisted in a walk-through exercise in a workshop setting where both questionnaires and group discussions where used. The workshop was attended by 17 experts with different areas of expertise.
The second Task Based validation activity was organised with two Jumbo captains after the workshop in Athens. During this session all items of the RAT were discussed based on comments received during the Athens session. The final version of the RAT contains 142 items in total.
The final results are summarized in the below points:
• Content: indicate that most respondents think thought that the content ‘as is’ was clear to the respondents in terms of its purpose, meaning of questions and different categories. A change in terminology was suggested to make the questions in both instruments more suitable for use in maritime industry. Certain wordings or concepts were not familiar in shipping and needed to be changed. Suggestions were made to use already existing guidelines for maritime vocabulary.
• Usability: almost all respondents indicated that ‘it is clear how they must fill in the questions’ and that ‘In general, the tool is easy to use’. Furthermore, certain specific design issues should be fixed to improve the usability of the tool. Several technical improvement suggestions were processed, like: implement 'go back buttons’, ‘freeze of display/screen sections’ and ‘better indication of which resilience levels the respondent is filling out the survey’ have been implemented accordingly. Including a download options making it possible to download the questions including the scores as .csv files (not Excel, no charts). Also, some recommendations to improve privacy of users and the data collected on the VP was taken care of with state of art technical (ICT) measures and in line with currently applicable EU privacy and cybersecurity regulation requirements. Finally, the Virtual Platform (of which the RAT is part of) will be available in an online version or can be installed on maritime end-users’ own local IT systems.
Applicability: most participants acknowledged that the RAT provided insight into how to assess and improve resilience aspects in their organization. Some suggestions for improvements were: further customization towards maritime, improvements in the design and testing the tool in companies to make it more applicable for their organization.
21. Validation report: SEAHORSE Virtual Platform
The main aim of the SEAHORSE Project is to enhance safety and resilience in maritime operations. The results of the SEAHORSE Project were presented in numerous conferences and workshops to promote these concepts within the maritime industry (See Deliverable 7.4). In order to support this aim, a platform titled the SEAHORSE Virtual Platform (SVP) was specified and developed. The SEAHORSE Virtual Platform hosts all SEAHORSE outputs and allows shipping companies to access the deliverables and utilize the developed tools to enhance the resilience of their organisations
The four main tools of the SEAHORSE Virtual Platform are:
1. Resilience Assessment Tool
2. Impact Assessment Tool
3. Procedure Improvement Tool
4. Crew Quality Audit Tool
The SEAHORSE Virtual Platform is designed to accommodate all SEAHORSE outputs, tools and deliverables. All outputs designed and developed under the SEAHORSE Project can be accessed on this platform by clicking here (http://seahorse.smt.strath.ac.uk/). The Credentials below can be utilized to access to the SVP:
Username: test1 Password:1234
This deliverable outlines the validation activities which were performed to finalise the development of the SEAHORSE Virtual Platform. The user experiences of 30 respondents were collected by distributing a widely-used “system usability scale questionnaire” developed by Brooke (1986). The results provided interesting insights into the user experiences of the SEAHORSE Virtual Platform users. It is evident form Error! Reference source not found. that resilience assessment tool, impact assessment tool and procedure improvement tool were found to be better than the industry wide threshold (68) according to SUS assessment. Only Crew Analysis Tool of the SVP got slightly lower score than the threshold amongst them. This has been already raised as an issue and efforts were made to enhance the user friendliness of the Crew Analysis Tool. The Procedure Improvement Tool is already in use by two shipping companies and negotiations are taking place with three more shipping companies to establish SVP for them. The Procedure Improvement Tool obtained the highest score within the SVP due to the prevalent usage of the tool in the maritime industry.
Administrator rights were distributed to 6 users (profile of system administrator) and their opinions were also collected via the same SUS questionnaire. Administrators dealt with different issues such as assigning new users, setting the appropriate weightings for the assessments while utilising the tools. The user experience score of the administrators of the SEAHORSE Virtual Platform is 78.33. The higher score was obtained due to the fact that administrator users utilised the system more frequently and they had a more comprehensive knowledge of the SVP.
Individual comments were also provided by the users on typos, functional integrity and IT problems faced. These were taken into account by the SEAHORSE Consortium and all the major changes have been accomplished.
Online platforms and websites are unfortunately prone to security issues. Any sort of weaknesses within the system can harm the developed assets quite severely. Therefore, significant importance is given the system security to keep the platform running. Two types of parameters were considered as vital:
1. Dos Attack
2. Brute Force Attack
Both these types of attacks were all generated and tested artificially within the system. The SEAHORSE Virtual Platform was resilient and robust enough to work sufficiently while the system was under attack. The SEAHORSE Virtual Platform is currently stored on the University of Strathclyde’s server, however, if a company would like to host the tool on their own server for some other security or policy reason this is feasible. All SEAHORSE Virtual Platform Tools have a copyright statement to secure the originality of the system and protect it from legal issues.
22. Report on the impact achieved in the task base and full scale validation studies
The aim of the SEAHORSE project is to improve safety and resilience in the maritime sector through transfer of knowledge between the aviation and maritime industries.
WP6 is the work package where the tools and solutions developed throughout the project for this purpose are validated and their impact assessed, confirming their applicability and implement ability for maritime companies.
The deliverables describe the different validation techniques covered in the work package including scenarios, use of simulators, interviews, assessments and by comparison within fleets of similar vessels. The validation methods involve consortium maritime partners as well as maritime advisory board members and selected external maritime stakeholders.
The tools and solutions developed and presented within WP6 present a compliment of implementable solutions for ship operators and shipping companies as validated by aviation and maritime experts. These range from software solutions including the SEAHORSE platform, a multi-tool application that includes the Procedure Improvement System (PIS), Maritime Resilience Model and Crew Quality Audit Tool to training and operational proposals including Human Factor Training programmes, Mandatory Safety Reporting and Checklist recommendations for the maritime industry, influenced by analysis of best practices in the aviation and maritime industries.
Furthermore, as evidenced by feedback from aviation consortium partners and the advisory board, many of the developed solutions could also be readily adapted and utilised by the aviation industry, emphasizing the cross transfer of best practices between the two transport sectors.
Deliverable 6.6 describes the impact that the developed solutions have had on the maritime companies within which they were validated. In summary these are;
Human Factors Training in Jumbo Shipping
A training plan designed specifically for Jumbo Shipping was implemented by Human Factors training experts(TNO and ESM) with adaptation to the maritime operator’s needs. Similar customisation would be required for participant maritime companies to ensure the applicability and effectiveness of the proposed Human Factors training provided. The impact of the training was then assessed by in company workshops and interviews. Finally, the participants were asked to complete the SEAHORSE impact assessment questionnaire. The feedback shows potential for significant impact of the solution within the company, especially when tailored to the company requirements. Useful information gathered was helped to inform and update the solution, with additional emphasis on “Train the trainer” sessions and appropriate adaptation to meet the seniority and skills levels of the trainees.
Standard Operating Procedure Development for Jumbo Shipping
The recommended process for Standard Operating Procedure Development includes assessment of the procedure for improvement, review of the proposed improvement and update, development of an SOP manual and communication to the company, implementation and feedback. This process was utilized in Jumbo Shipping with focus on SOPs relevant to their requirements.
The guidelines were found to be implementable and were followed with the approach broadly validated. Feedback confirmed that impact within the company requires adjustments and customisation of the procedure to the operator, its needs and structure of the company. Additional SOPs from Jumbo Shipping will be developed by the same process and their impact assessed by crew feedback questionnaires after project close.
Checklist Impact assessment by ITU
Checklist proposals for ‘Arrival in Port’ and ‘Preparation for Sea’ were conducted using simulators at the Full Mission Ship Handling Simulator at Istanbul Technical University’s Maritime Faculty. A series of questionnaires was then carried out to capture the qualitative feedback which then led to the checklists being re-structured.
The checklists were seen as having significant time saving impact on ship operators. An additional constructive proposal was that self-test functions should be included in electronic systems to save time during bridge checklists.
Checklists and Observation Study in CalMac Ferries
Checklist solutions were implemented across the CALMAC fleet allowing for fleet wide validation via similar vessels operating in similar environments. The checklists provide standardisation whereas different systems previously operated on different vessels.
Through questionnaires and feedback, the solution was deemed to be well perceived by crew and office staff with further refinement potential and additional reference to ‘information sheets’ to be included. Furthermore, the checklists were regarded as delivering benefits for CALMAC vessels in operation and during manoeuvring including for standardising bridge protocols. Safety improvements to be viewed over time are anticipated to include the elimination of single points of failure and the avoidance of high-potential safety incidents.
Mandatory Safety Reports and Safety Bulletins
Solutions developed regarding Taxonomy, Safety Report and Safety Bulletin templates were validated in a range of shipping companies and maritime partners.
Taxonomy validation and methodology was validated in a Chemical tanker company and by Sener Shipping. The taxonomy was viewed as implementable and useful and updated following feedback of missing criteria during validation. A standardised taxonomy was viewed not just as useful but essential to provide the most effective capture of incidents, analysis of reporting data and for accurate comparison of data.
Safety Report templates and Safety Bulletins were developed with consultation from maritime consortium partners, external maritime companies and by an external crewing company. Safety Bulletins were then further validated within AP&A and initial impact feedback gathered was positive, enhancing perceptions of management’s positive attitude to safety improvements and professionalising and standardising company safety communications to staff and crew. Further impact assessments are also to be carried out after 3 months via questionnaires in order to provide a sufficient period for regular bulletin circulation and awareness within the company.
Resilience Assessment Tool and Impact Assessment
The Resilience Assessment Tool includes a resilience questionnaire, a multi-level resilience matrix, a set of resilience solutions to be readily implemented by ship operators. Accompanying the Resilience Assessment Tool an impact assessment tool was developed to practically evaluate the impact of implemented resilience solutions.
Feedback on the tools was gathered via varied validation activities, involving different workshops with end-user stakeholders in the maritime industry. Most participants acknowledged that the tools provided insight into how to assess and improve resilience aspects in their organizations and could provide constructive guidance to help enhance safety through resilience in maritime companies. Specific feedback was also integrated into upgrades and revisions of the assessment tool, especially items regarding usability.
Just Culture
The solution for Just Culture was validated within a European shipping company. Stages of implementation include an initial Just Culture Survey, development of Safety Performance Indicators, company documentation review and review of application of just culture process in incidents or occurrences.
Initial impact of the solution was viewed positively with clear benefits for the shipping company, however due to the nature of Just Culture and the time required to embed the solution within the company, periodic annual reviews are recommended to fully assess impact on company culture.
Crew Quality Audit Tool in Sener Shipping
The Crew Quality Audit Tool, designed to improve employment quality through quantitative measurement of employee performance, was validated in Sener Shipping. Seafarers were assessed and evaluated against different criteria A-) Knowledge and skill base, B-) Adaptation to safety rules, C-) Professional behavior, D-) Leadership and social behavior, E-) Personal criteria, F-) Adaptation to Sea and ship life.
The Crew Quality Audit Tool was measured against an existing tool in the company and the benefits and potential future impact on safety was notable. The new tool allows the company to reliably measure ability at the job application stage which can enhance onboard safety via improved recruitment of seafarers and their compatability with the rest of the onboard team. Other improvements for the operator include the ability to compare seafarers on an equal basis and monitoring of seafarer’s development and training on deployment and across subsequent contracts.
Electronic Checklists
The electronic checklist validation by another shipping company includes implementation in over 30 vessels. The solution allows users to more effectively manage checklists onboard in a consistent and reliable way. It was also adapted to accommodate the limited internet connectivity on board with the software synchronizing data at intervals with the office.
Through questionnaires and impact assessment, the tool was viewed to be helpful in a number of ways including in time saving for seafarers (improved by auto-fill checklists that can be updated at intervals). The Electronic Checklist was also found to enhance reliability, data analysis and optimize communication between the vessel and office.
Safety Culture Assessment and Improvement
The aim of the safety culture assessment to identify the safety related problems of any shipping company, address the vulnerabilities and develop improvement strategies to enhance maritime safety. Two surveys, one for crew members and one for shore personnel, were developed in order to perform safety culture assessment in shipping companies. The questionnaires were distributed to different shipping companies in order to cover different operation types and create a benchmark for each safety climate dimension.
The results shed light on the strength and weaknesses of the each company. Crew members of the tanker shipping company achieved higher scores amongst the companies. However, the shore staff of the Bulk Carrier Company achieved the higher scores amongst the companies.
The results significantly vary between different vessels of the companies, even different results were observed between sister ships of the same company. These outcomes were thoroughly discussed to gain insight into underlying reasons of these discrepancies.
The safety climate assessment results generated a substantial short term impact within the each company. The companies gained insight into their current safety climate level on ten important dimensions amongst other companies. All of them have already started taking actions to improve the areas where they scored lower. The company managers and researchers from the University of Strathclyde had discussions on the results generated to address the gaps identified. Long term impact will be visible when researchers have enough data to compare the current situation of safety climates in each company and their maturity levels after the implementation of all given action plans.
The confidential assessment reports were provided to each company together with the recommendations and action plans. The majority of the recommendations have been already taken into account in all companies. All the results were discussed with the HSEQ departments in each shipping company and the Strathclyde will continue to support each company at the implementation of action plans.
There is a requirement to make these assessments annually in order to see the trends within the organizations. The safety climate framework will be implemented in a cruise liner and a ferry company beyond the project time to enhance the accuracy of the benchmark. The framework will be promoted to other types shipping companies in order to capture problems of different shipping operations. Ultimate impact will be achieved when numerous shipping companies start assessing their safety climate with the given methodology. In addition to the survey results, in each company, a thorough study will be conducted to investigate all the available KPIs and their positive impact of safety performance. After capturing the attitudes and perceptions of the employee’s and validating those subjective results with objective safety related key performance indicators, it is also of paramount importance to arrange on board observational studies to identify crucial parameters which affect seafarers’ adherence to safety management rules. Existing blame culture is the biggest challenge for the overall methodology. An effective just culture approach should be implemented in order to ensure the quality of the reporting and data collection methodologies.
23. Guidelines and standards
In this deliverable, the aim is to develop best practices guidelines for the shipping companies to enhance their safety maturity levels. Aviation industry’s successful solutions were chosen to be transferred after the feasibility analysis conducted in WP3 and these solutions were tailored for the shipping companies. Best practice documents were developed in order to guide organizations to implement those selected solutions. Each best practices document provides step by step instructions for the shipping companies to implement the solution with their organizations. There are five best practice guidelines given in this document, namely:
1. Standard Operating Procedure Development
2. Just Culture
3. Pre-departure and Arrival Checklists
4. Human Factors Training
5. Mandatory Safety Reporting
6. Safety Bulletins
Each best practices guide describes:
1. how to implement the given solution,
2. the prerequisites to implement the solution,
3. resources needed (time, personnel),
4. the duration of the implementation, and
5. How to assess impact.
All these best practice guidelines were implemented and validated by the end-users. The SOP Development Guide was utilized in a shipping company and they converted their Lifting procedures into a more structured, concise and clear format. The new SOP has been tested in some heavy lifting operation vessels and the other SOPs are now being converted by utilizing best practices guidelines.
The Just Culture solution was tested and validated via workshops and different experts were presented in these workshops. Regarding the Just Culture implementation, it takes a high amount of time to build the trust and increase all the reporting numbers. The results will be seen in a long term basis.
Arrival and departure checklists were developed by utilizing the knowledge gained from aviation. The best practice guideline was implemented to standardize the arrival and departure checklists for a fleet of 10 RoRo Passenger vessels. The developed checklist has the “check and verify” system, this removed the single point of failure within the system. The new checklists were validated in a shipping company and now it is in use in all RO-RO vessels.
Human Factors Training guidelines were also tested and validated in a shipping company. The guidelines were converted into training modules and delivered at the main office of the company. Relevant Human Factors issues were discussed that created a better awareness for the participating officers. The training was rated well by the participants.
Mandatory safety reporting guide aimed to standardize the reporting system and transfer the most relevant knowledge between the parties in a consistent manner. The designers within the consortium finalized the template for each mandatory safety reporting scheme in order to include all the compulsory information. These reporting templates were tested and validated in a tanker company.
24. Training modules
Safety is the common value for all maritime stakeholders and they are looking for means to enhance safety in the maritime industry. The majority of accidents (70-80%) are attributed to human element. A highly skilled and resilient crew is seen as a way forward to decrease human error related accidents and avoid reoccurrences of catastrophes such as the Herald of Free Enterprise, Deep Water Horizon and Costa Concordia. Shipping companies continuously invest in training to enhance their crew members’ knowledge, competency, skills and resilience. The Standard of Training, Certification and Watch-keeping Convention was proposed by IMO to define basic standards of training and create more resilient crew members for the maritime industry.
This deliverable aims to enhance crew members’ resilience levels by providing a set of training modules. These training modules were developed by utilizing the best practices knowledge available in the aviation industry. All of the modules were develop by maritime and aviation experts. The aviation experts checked the modules thoroughly to ensure all the required information is given and the implementation methodology is applied correctly. The maritime experts were responsible to tailor the training modules based on the needs of the maritime industry. Six training modules were generated for the shipping organizations. These training modules are:
1. SOP Development Training Module
2. Just Culture Training Module
3. Checklists Training Module
4. Human Factors Training Module
5. Mandatory Safety Reports Training Module
6. Safety Bulletins Training Module
The developed training modules were uploaded to the SEAHORE Virtual Platform and they are publicly available for the shipping companies. All the trainings were developed in PowerPoint format and are available at the website. This deliverable provides assistance for the readers on how to access each training module at SEAHORSE Virtual Platform. The deliverable was constructed as a reference document in order to direct readers to access the relevant information. All training modules were developed in a self-explanatory way, therefore no information regarding the details of the training modules is provided in this report. It is suggested to use training modules and the best practice document (Deliverable 7.1) together to enhance the outcome of the modules.
All training modules can be downloaded from the SEAHORSE Public Deliverables website. The website can be accessed under the SEAHORSE Virtual Platform by clicking here (http://seahorse.smt.strath.ac.uk/PublicDeliverables/).
Potential Impact:
1-Potential Impact
The main expected impact that SEAHORSE project would like to achieve is to clearly demonstrate that different transport modes can and should work together to share the best practices with practical impact on safety. The SEAHORSE project proposes to address human factors and safety in maritime transport by transferring the well proven practices and methodologies from air transport to maritime transport in an effective, collaborative and innovative manner.
Project clearly concluded that best practices in one transport mode can be utilised in another transport mode through establishing a transfer framework as solutions in one transport mode needs to be tailored to other transport mode. SEAHORSE was the first project under technology transfer and due to the proven success of the project, it is expected that more multi transport research project will be developed and more interactions between different transport modes will be encouraged and devised leading to safety enhancements in all transport modes in a cost and time effective manner.
EC DG research and Innovation –Transport Directorate is already working on establishing a Transport Safety Research Cluster covering all transport modes and encouraged by the SEAHORSE project’s results more multi-transport research calls can be expected in future work programmes.
The SEAHORSE project’s primary impact is an improved overall human factors and safety in maritime transport achieved through the introduction of innovative “SEAHORSE Procedure Improvement Methodology” and “SEAHORSE Multi-Level Resilience Model” supplemented by training and guidelines for the efficient implementation. Whole shipping industry is likely to benefit from the innovative outputs of the project as it will support the shipping companies and the maritime training institutions to understand the importance of Human and organisational factors to enhance the maritime safety and integrate them to their ship operations.
SEAHORSE created significant awareness within maritime community that, safety can be enhanced beyond compliance of rules and regulation through human and organisational factors through THE NEW SAFETY APPROACH.
While SEAHORSE developed novel, practical methodologies and tools to enhance maritime safety through transfer of best practices in human/organisational factors, in order to create industry wide take up and impact, it is necessary to have to turn these methodologies and tools to industry standards.
Through creating the training and guidance, the SEAHORSE project expects not only to impact the awareness and knowledge of current seafarers but also influence the future ones through integration in the education curriculum of the maritime schools. This will lead to skilled workforce creating more attractive environment for potential future cadets.
2- Exploitable Results
SEAHORSE consortium produced a number of outputs that can be utilised by the shipping companies, Maritime Education and training centres. As the aim of the project is to enhance the maritime safety by taking into account human and organisational factors in a structured manner, the tools are available to Shipping companies and educational establishments through free licencing. This is expected to influence the industry to take into account human element and organisational factors towards enhancing maritime safety. The consortium partners are available to assist the industry during the utilisation of SEAHORSE tools and methodologies. Ultimately, these tools and methodologies developed by the SEAHORSE consortium will be utilised to support IMO’s activities on Human element.
The SEAHORSE exploitable outputs are briefly introduced below. All the tools are available through WEB-based SEAHORSE VIRTUAL PLATFORM via free licencing.
Comparative State Of The Art & Gap Analysis
Comparative state of the art study has been conducted and the SEAHORSE Project identified 46 gaps between Aviation and Maritime under different headings including stakeholders; Functional Demands; Training; Regulations; Current Challenges; Human and Organisational Factors; Operational Demands; Socio-Economic Issues. Such a gap analysis has been reported first time between two transport modes.
Resilience Solutions and Transfer Methodology
Resilience solutions available in airlines are for the first time compiled into a database with an aim of transferring the solutions from air transport to maritime sector with the involvement of practising maritime experts as well as safety managers. The SEAHORSE transfer methodology was employed to shortlist and ranks the resilience resources based on a subjective evaluation made by maritime professionals and experts. The evaluation criteria were defined in order to assess clearly the potential applicability of the proposed solutions to the maritime domain
Procedure Improvement Methodology and Tool
SEAHORSE developed, first time in the maritime sector, a methodology and automated tool which provides a robust method for the collection and assessment of workarounds practised on-board vessels within the maritime industry. The collected workaround data are utilised as an opportunity to improve existing standard operating procedures for a given ship or a company.
Resilience and Impact Assessment Tool (Software):
Seahorse developed, first time in maritime sector, a multi-level resilience and impact assessment framework and tool to allow the safety managers to assess the company in terms of resilience compliance and identify the gaps/weaknesses/noncompliance that may lead to errors and accidents. Furthermore, the SEAHORSE Impact Assessment Tool is intended for workers who have been directly impacted by the implementation of one of the SEAHORSE solutions. It gathers information in a structured way about what workers think has changed as a result of the implementation of the SEAHORSE solutions. It also gathers behavioural data on changes to the way things are performed following the implementation of the SEAHORSE solution. Additionally, for safety and managerial staff, it provides a method for collating existing safety metrics and supplementing them with new measures to assess organisational safety impacts.
Crew Quality Audit Tool (Software): SEAHORSE developed a tool which allows companies, by utilising the databases, to audit the crew quality at individual as well as at team levels and identifies the strengths weaknesses of the crew and identifies the training needs or suitability for type of ships for operation.
Best Practise Guidelines
Best practice guidelines were developed for shipping companies to enhance their safety maturity levels based on the aviation industry’s successful solutions. Each best practices document provides step by step instructions for the shipping companies to implement the solution with their organizations. These solutions address safety culture, just culture, sop development, human factors training, arrival and departure checklist, mandatory safety reporting, safety bulletin.
Training modules of the solutions
These training modules were developed by utilizing the best practices knowledge available in the aviation industry. All of the modules were develop by maritime and aviation experts. These training modules are:
1. SOP Development Training Module
2. Just Culture Training Module
3. Checklists Training Module
4. Human Factors Training Module
5. Mandatory Safety Reports Training Module
6. Safety Bulletins Training Module
Safety Culture Assessment and Improvement Framework
A safety culture assessment and improvement framework developed to identify the existing problems in shipping companies and further enhance safety. It has been utilized in three different shipping companies to create an industry wide benchmark and currently more shipping companies are liaising with SEAHORSE project to utilise the framework.
SEAHORSE Virtual Platform
A WEB-BASED tool hosts all the tools developed within the project and allows companies to developed their databases and utilise the tools towards safety enhancement of a company. Free licencing can be provided to Shipping companies and Maritime Education/training institutions
( http://seahorse.smt.strath.ac.uk/bl.html)
3 Dissemination Activities
SEAHORSE consortium has adapted a coherent approach to make sure the resources among the consortium are applied efficiently to disseminate as widely as possible. Depending on the status of the project and the content of the results, the SEAHORSE performed a number of dissemination activities as listed below.
• Setting up project website: http://seahorseproject.eu/
• Setting up accounts in social media
o Linkedin: https://www.linkedin.com/company/seahorse-project
o Facebook: https://www.facebook.com/seahorseproject
o Twitter: https://twitter.com/SeahorseFP7
• Producing project brochure and banner
• Issuing newsletters
• Hosting public workshops and regional events
• Giving presentations in targeted scientific or commercial conferences
• Providing presentation to shipping companies or associations.
• Publications
• Press release and others
3.1 Workshops and conference organised by Seahorse Project
Seahorse project organised 7 different planned public events in 6 different locations and 262 people took part in these events.
3.1.1 Main Annual Workshops: Workshop 1, Workshop 2, and Workshop 3
SEAHORSE organised three flag ship SEAHORSE public workshops. These workshops are mainly set up to disseminate the project concept and progress. Meanwhile, these workshops also serve as a good platform to engage the local maritime stakeholders in supporting the validation of the solutions and tools developed throughout the project duration. The presentations of all the annual workshops are publicly available via the project website (http://seahorseproject.eu).
• SEAHORSE Workshop 1: 12 November 2014 (Month 13), Lloyd’s Register’s headquarters, London, United Kingdom. It was hosted by Lloyd’s Register and AP&A.
On the 12th November 2014, the first SEAHORSE workshop was held at Lloyd’s Register’s London office. London was chosen as the first venue due to its leadership in maritime business so that we can raise SEAHORSE’s profile in the industry at the beginning of the project.
It is a successful event, attracting 49 carefully selected participants. 77% of the participants came from the industry. We found that ship operators and owners (19%) are the industrial groups that tend to be most interested in this subject area. We have also secured a first keynote speaker from Cyprus Airways giving a presentation prepared with another advisory board member from Airbus. Other keynote speakers include visitors from Rolls-Royce and EasyJet. Through their in-depth knowledge in the aviation human factors, some health discussions have been raised throughout that day.
• SEAHORSE Workshop 2: 13 October 2015 (Month 24), Rotterdam, Netherlands. It was hosted by KAHN and TNO.
The Rotterdam workshop was successfully held in the Shipping and Transport College on the Tuesday 13th October 2015, in total more than 60 people participated. This workshop features a key note speaker – Markku Mylly, the Director of European Maritime Safety Agency (EMSA), and Arthur Dijkstra, the Captain of Boeing 777 of the KLM. The multi-level resilience assessment tool was further developed and validated in an interactive SEAHORSE workshop in Rotterdam with interested stakeholders. More than 40 maritime participants from industry provided suggestions of currently-used resilient solutions developed for particular challenges confronted in the maritime industry. Such challenges could be language problems or reduced crew sizes. This second SEAHORSE workshop successfully promoted brainstorming for new solutions, which will be implemented in a Virtual Platform in the course of the project. Such tools, together with the resilience questionnaire, will be made available to the maritime industry as well as any other interested parties. It also attracted the attention of fishing industry.
• SEAHORSE Workshop 3: 31 May 2016 (Month 31), Danaos’ headquarters, Piraeus/Athens, Greece. It was hosted by DANAOS and AP&A.
During the Piraeus workshop held on the 31th May 2016, it was highlighted that the work on safety improvement usually focuses on technical issues. However according the data collected from the Marine Accident Investigation Branch (MAIB), more than 80% of incidents are due to human factors. Together with the seafarers shortage, challenges lied ahead of the marine industry is clear.
Adopting the principles of resilience engineering in marine operation is one of the approach the project SEAHORSE is proposing. As highlighted during the workshop, learning, responding, monitoring and anticipating are the four key capabilities should monitor. The ownership of improving the weakness of ship operation should not be solely relied on the crew on board but also shared with marine professionals working onshore.
One of the SEAHORSE’s key advisory board members from aviation reminded the audience an important lesson learned from the aviation industry. It is common to see that near misses and accident rates going up at the beginning when similar safety solutions are implemented. It may appear as if the safety level is going down at the first glance, but this is actually a healthy exercise because an organisation can have a better visibility in where the problems are. Appropriate actions can then be introduced accordingly. Companies should be expected to observe such trends at first and should not be turned away from the continuation of the implementation process.
3.1.2 Local Workshops
In addition to 3 Annual Workshops which aim to disseminate the seahorse results internationally, three additional local workshops were organised in Glasgow, UK, Istanbul, Turkey and Madrid, Spain to disseminate the results to local maritime industries
Local Workshop 1: Glasgow, UK, 3 March 2015
The local workshop was attended by 40 people including people from different shipping companies including TEEKAY Shipping, Anglo-Eastern Ship Management, V-Ships, Calmac, Gaslog, Hellenic tankers; Aviation companies such as Airbus and Logan air, training companies and maritime colleges as well as representative of EU project CyCLades. A number of presentations made by Airbus, Loganair, CyCLades project as well as members of Seahorse consortium to inform wider audience about the objectives and achievements of the project. In the afternoon, all the attendees took part in a structured-interactive workshop to gather expert opinions of the safety managers about risk of workarounds to assess the methodology develop within SEAHORSE about managing the workarounds
Local Workshop 2: Istanbul, Turkey, 8 March 2016.
Istanbul local workshop, which was organised by Istanbul technical University, was attended by around 100 People from Shipping companies, maritime authorities, maritime schools, Cadets. Presentations were made to introduce seahorse aim, objectives and methodologies, Seahorse Tools and Virtual platform as well as industrial implementations of seahorse methodologies including implementation of pre-departure and arrival checklist approach to Calmac ferries. Workaround tools was also demonstrated in an interactive way involving industry representatives. Local shop was very successful as the Local maritime industry was made aware of the project outcomes and how they can utilise these tools.
Local Workshop 3: Madrid, Spain, 19 October 2016
Third local workshop was organised by ESM in Madrid and attended by 35 people from Aviation Authorities, Shipping Companies, Spanish Maritime authorities including search and rescue department as well as Road Transport Sector.
Antonio Castellano from the ESM opened the day. Prof. Osman Turan from the University of Strathclyde outlined the pressing issues regarding the human and organisational factors in the maritime operations. The Spanish Safety Investigation Authority (CIAIAC – Ministry of Transport of Spain) then delivered a thought-provoking presentation offering an example of difficulties in an investigation of a helicopter accident. Dr. Paul Liston from the Trinity College Dublin (TCD) elaborated the technology transfer methodology developed by the project SEAHORSE. 46 gaps have been identified. 3 scenarios and 73 solutions were shortlisted. F.A. (Dolf) van der Beek led the floor to the introduction of the resilience engineering paradigm. The procedure improvement tool developed by the project SEAHORSE was presented by Dr. Rafet Emek Kurt from the University of Strathclyde. The individual crew quality audit tool was jointly presented by Ozcan Arslan from the Istanbul Technical University and Volkan Arslan from the University of Strathclyde. The experience of testing some solutions in the industrial environment was presented by Ben Wood from CalMac, Fotis Oikonomou from Danaos Shipping. Other industrial issues were discussed during a panel session chaired by the University of Strathclyde. It is recognised that it will be important to continue seeking better solutions in improving ship safety. It is especially beneficial to have a new framework in addressing the current issues regarding the safety management system
3.1.3 International Seahorse Conference, 21-23 September 2016, Glasgow UK.
27 papers have been accepted and published in this International Conference held in Glasgow, between 21-23 September 2016. The conference, which was attended by 60 delegates, began with a welcome speech delivered by Prof Osman Turan, the co-ordinator of project SEAHORSE from the University of Strathclyde. There were 4 keynote speakers from the industry, including Harry Nelson from Airbus, Evangelos A. Tsoumpos from A.E.Nomikos Shipping, Martin Hernquist from the Swedish Club, and Captain Sanjay Patil from MISC Berhad. They shared their valuable experience from aviation to maritime industry and raised questions such as whether accidents can be prevented and human and organisational factors for the maritime safety. Practical issues are in heavy discussion among seafarers, safety managers and safety experts from other industries.
Prof Jan Maarten of TNO then introduced Resilience Assessment Tool (RAT. Prof Maarten also stated that RAT is the first tool available in time to check resilience of an organization. Furthermore, it was emphasised by different speakers that safety and just culture should be promoted into maritime along the conference programme. Blame culture affects the confidence of employees gravely and in return real feedback from employees cannot be retrieved. Fair approaches and fair treatments of behaviour are recommended.
During a small discussion session, the question ‘Can different transport modes work together in achieving common safety goals?’ was raised to the floor. Barriers to prevent action in achieving higher safety level have been discussed:
• Money
• Attitude
• Government
Impact assessment methodology was presented by Dr. Paul Liston of Trinity College Dublin and Vanessa Arrigoni of Deep Blue. They presented a transfer methodology which may be useful for others who is seeking to conduct similar transfer between different industries or practices.
The Seahorse Virtual Platform, resilience assessment software, was then introduced to the conference delegates and helped to ascertain the resilience of an organisation.
Conference proceeding are freely available to public through the website.
List of Websites:
http://www.seahorseproject.eu/
for further communication: info@seahorseproject.eu
contact details of the co-ordinator: o.turan@strath.ac.uk
SEAHORSE is a leap forward towards shipping safety achieved through technology transfer from air transport to maritime transport focusing on human factors problems in an innovative, integrated and multidisciplinary manner towards safer and more resilient shipping operations.
Within the Aviation industry having safe and reliable operations is critical in preventing accidents and mistakes that can potentially cause a huge loss of life and destruction. The influence of human factors on operational safety is one area were the aviation industry has led the way in terms of understanding and implementing tools, methodologies and systems to combat human error within a system.
The SEAHORSE project brings together an experienced, diverse and committed consortium from air and maritime transport sectors and world leading expertise, with the overall goal of tackling the issue of ‘Human Factors and Shipping Safety’.
By taking a pioneering and original approach the SEAHORSE consortium aims to achieve meaningful improvements in shipping safety by addressing human and organisational factors with the introduction of the following innovative concepts:
• Enhanced understanding and identification of underlying reasons for marine accidents and execution of gap analysis compared to air transport industry
• Identify the best practices in air transport managing errors and non-standard practices including the implementation of resilience engineering principles and develop the framework to transfer these successful practices to maritime transport to enhance the safety through managing the human and organisational factors
• New and innovative ‘workaround Management’ methodology for the identification and assessment of the negative/positive outcomes of the practice of crew performing workarounds.
• Identification and measurement of the functional demands of ship operations and crew needs, limitations and capabilities.
• Development of Multilevel resilience approach covering individuals, teams, multi-teams and organisations linked and interactive to make sure interventions to improve resilience at one level should have positive impact on resilience at a different level.
• Innovative validation through real case studies involving operational ships, full bridge simulators and training institutions/companies
• Innovative SEAHORSE virtual platform, which can be utilised to tailor the multi-level resilience principles to individual company needs, develop targeted implementation plan and training materials
• Practical guidelines will be available for the whole industry.
Through the SEAHORSE concept and the utilisation of some of Europe’s leading companies and institutions, both in the consortium and advisory board, in various fields including human factors research, operation, engineering and training in air and maritime transport; SEAHORSE has the potential to create a significant impact, at not only a European level but also an International one, in making the ship operation a safe, resilient, attractive and efficient environment.
SEAHORSE project clearly demonstrated that different transport modes can and should work together to share the best practices with practical impact on transport safety by adopting the technology transfer framework similar to one developed by SEAHORSE project.
SEAHORSE project outcomes/tools are freely available to shipping companies and maritime education/training institutions towards improving the maritime safety.
Project Context and Objectives:
PROJECT CONTEXT AND MAIN OBJECTIVES
While an increase in the transportation of goods worldwide by sea has been recorded, there has been a general decrease in the number of crew on-board observed. Through system automation, operational streamlining and financial constraints extra demands have been placed on the crew which has the potential to lead to situations conducive to human error and accidents. The modern seaman is expected to be multi-disciplined with a high level of technical skills, have broader deck officer responsibilities, manoeuvre the vessel he/she is working on at short notice and be prepared to work long hours with very limited days off and outside social contact. These working conditions and this environment are proven not only to be making seafaring a less attractive career option but more worryingly causing accidents.
Traditionally safety has been addressed both by designers and regulatory bodies such as the International Maritime Organisation (IMO) through structural, mechanical, electrical and technological solutions with the aim of minimising damage and prevention of loss of life and ships/floating structures. Furthermore, it has been often ignored that the human element of the maritime system has not been evolving in the same way that technology is developing; with the physical capabilities and the limitations of the human being overlooked. Addressing the human element in practical ship design and operation is a challenging task due to traditions and lack of knowledge.
The air transport sector, which is in many ways similar to the marine sector have been facing similar human and organisational factors that affect operational safety. However the airline industry has been managing these issues by approaching the same problem systematically and developing much more advanced methodologies and techniques that can be adapted to the marine industry while utilising the experience of air transport.
The SEAHORSE project proposes to address human factors and safety in marine transport by transferring the well proven practices and methodologies from air transport to marine transport in an effective, collaborative and innovative manner by adapting and tailoring aviation practices to the unique needs of maritime transport.
This will be primarily achieved by introducing the principles of resilience engineering and smart procedures methodology in an integrated framework which will result in multi-level resilience that linking individuals, team, multi-party teams and organisations in ship operation that ultimately enhancing shipping safety.
The SEAHORSE project’s aim will be realised through the implementation of the following objectives:
1) Identify the key human/organisational factors, which lead to operational successes and failures in marine and air transport and perform gap analysis in marine practices in comparison to air industry;
2) Investigate how errors and non-standard practices were managed successfully in air transport and check the feasibility of applying best practices and resilience concept adopted in air transport for marine to improve human/organisational errors and safety
3) Develop the Technology Transfer Framework from air to marine for successful implementation
4) Introduce a smart procedure methodology in marine operations to identify and assess non-standard procedures carried out on board ships to quantify the positive/negative effects in order to enhance overall resilience.
5) Develop and validate a multi-level resilience model and virtual platform as well as guidelines for marine transport which encompasses individual, team, multi-party and organisational resilience that linked and integrated. The SEAHORSE Platform will address
a) The crew needs and limitations that may affect their resilience with regards to navigation/operation of the ships (competency, physical & cognitive limits, attitudes, team composition)
b) System design, equipment and procedures which promote/add/remove, by design or accident, resilient and/or non-standard procedural behaviour
c) Shared situational awareness, leadership, organisational drift, insufficient/non-existing safety culture, team work
d) Suitable training materials to implement the multilevel resilience
6) Implement and evaluate the benefits of the SEAHORSE multi-level resilience tools through the comparison with traditional maritime safety methods and operations in training, simulator and actual ship environment.
7) Educate stakeholders within the marine transport industry of the benefits of resilience engineering and the SEAHORSE concepts.
Project Results:
MAIN S&T RESULTS/FOREGROUNDS
1. State of the Art: The ‘as is’ Picture
From the four high-level topics of inquiry in the state of the art (SOTA) results can be summarised as follows:
Stakeholders
The review of stakeholders highlighted the fact that the two transport sectors are very different. Both have a strong emphasis on safety but there are differences in how safety is managed and negotiated by the main actors. The maritime sector, in the category of ‘secondary users’, contrasts strongly with the aviation sector. The maritime sector has a much more broad and varied category of secondary users. Clearly managing inputs from all these stakeholders and ensuring the safety of a ship’s operation in these circumstances is not an easy task. Negotiation safety with so many stakeholders with varied agendas, as happens in the maritime sector, is a task which is complicated by training and regulatory issues.
Training
One of the key differences in relation to training concerns oversight, assessment and currency of skills/knowledge. The aviation sector can be broadly said to have better systems and procedures in place to oversee, assess and ensure the currency of personnel training. One particular example is HF training. Human Factors Training in the maritime sector is only mandated for certain grades of staff. And while most of the same topics are covered in aviation HF training as in maritime the fact that only certain grades receive the training means that its potential impact on the system functioning is reduced.
Regulations
Both the maritime and aviation sectors are highly regulated industries but it is clear that oversight and the interpretation and implementation of regulations are an area where the maritime sector could learn from the aviation sector. The lack of a mandatory quality approval system for Flag States is creating a big safety discrepancy between the potential and actual safety of the maritime system. While the maritime sector seems to tolerate this systemic safety ‘workaround’ the aviation system is far less tolerant of safety ‘fudges’ such as these. Indeed the EU and US aviation safety regulators have the power to ban airlines that fail to meet their stringent safety requirement from even entering European airspace. This ‘blacklist’ is publicly available and regularly updated.
Challenges
Clearly the maritime sector is facing many safety challenges. And while the aviation sector has had greater success in, and a longer tradition of, tackling safety issues there are safety opportunities for each sector. In relation to current challenges there are areas of convergence and divergence between the two sectors. Both the maritime and aviation interviewees have stated that paperwork and task load are very important challenges for their operations. Similarly both sectors see challenges related to the organisational context – specifically the lack of resources in the aviation sector which has an impact on competence (insufficient training) or task performance (lack of equipment, inadequate facilities etc.), though the industry is trying to manage this through audit plans and regular reporting. In the maritime sector one issue is the compartmentalisation of safety where it is viewed as separate from the operation and not related to everyday work. This leads to the notion that safety is somebody else’s problem and results in a diffusion of responsibility for safety. Meanwhile there is convergence in relation to multicultural crews. The maritime sector cites this as a significant challenge whereas the aviation sector states that it is not a safety challenge as standard operating procedures constitute a good solution for avoiding possible issues that can come up due to multicultural issues.
2. Review of Occurrence Analysis
D2.2 provides, for the first time, a comparative analysis of accident occurrence data in both the aviation and maritime sectors highlighting the commonalities and differences across a number of topics ranging from mandatory reporting, taxonomy, and standardisation to causal factors.
The main results are threefold:
Occurrence Reporting procedures
Aviation: Aviation has a standardised mandatory occurrence Reporting system as International Civil Aviation Authority, ICAO Annex 13 (Chapter 8) requires States to establish mandatory incident reporting systems to gather information on actual or potential safety deficiencies. This requirement is supplemented by further ICAO requirements which require aviation companies to have in place formal processes to collect, analyse and act upon feedback on hazards and risks in operations in order to support the implementation of mandated safety management systems.
Maritime: The maritime sector has not reached the maturity level of aviation despite the efforts within the IMO, and different authorities use different taxonomies to guide the collection of data. Additionally there are significant variations in the standards of reporting. This creates a significant problem with regards to the creation of a single maritime occurrence reporting taxonomy as well as a centralised database.
Availability and similarity of Data
In aviation reports of single accidents are publicly available in Agency databases (e.g. NTSB-AAR, European Central Repository). However, more in depth information on the contributing factors is classified and only accessible by authorized national agencies to ensure anonymity and confidentiality.
The SEAHORSE Consortium managed to obtain a comprehensive accident review from the UK CAA containing important information on human and organisational factors which could be considered comparable to the MAIB database used for the maritime analysis. A direct like-for-like comparison across categories and/or levels of human and organisational factors was not possible due to the fact that the CAA and the MAIB use different frameworks to categorise accidents and incidents.
Aviation and maritime are two transport sectors with different levels of safety maturity and it stands to reason that they have emerged using different classification schemes for accident and incident data. So, while a direct comparison cannot be made, the data were sufficiently rich to give us insights into the incidence of human and organisational issues in the maritime sector as compared to aviation.
Occurrence Analysis
Maritime
According to the European Transport safety Council, passenger fatality rates in shipping are 14.8 passengers per 100 million passenger hours and 0.46 per 100 million passenger kilometres. The data obtained from the MAIB (Maritime Accident Investigation Branch) was analysed under three levels: Underlying accident factor category, Underlying accident factor and Underlying accident sub-factor. It was found that the majority of the factors leading to accident is related to the human factor (74% of the underlying main factors) while 26% are listed under the technical category.
Aviation
The Global Fatal Accident Review 2002 to 2011 was carried out to provide a ten-year overview of worldwide fatal accidents involving large jet and turboprop aeroplanes engaged in passenger, cargo and ferry/ positioning flights (CAA, 2013). The corresponding on-board fatality rate for the same period was 22.0 fatalities per million flights flown or 12.7 when expressed as per million hours flown. According to European Transport safety Council, passenger fatality rate in aviation is 0.035 per 100 million passenger kilometres.
3. ‘Gap Analysis Report’
A total of 44 gaps were identified which impact safety in the maritime sector. This is the first such study clearly providing the gaps between the air and maritime transport with regards to safety. They vary widely and differ in terms of focus, and level of granularity. Below is the list of these 44 gaps grouped according to topic.
In total, 9 gaps defined under the “Stakeholders”, 7 gaps were identified under the “Functional Demands”, 5 gaps were identified under the “Training”, 7 gaps were identified under the “Regulations”, 4 gaps were identified under the “Current Challenges”, 1 gap was identified under the “Advanced tackling of HOF issues”, 5 gaps were identified under the “Operational Demands” and 6 gaps were identified under the “Socio-Economic Issues”.
4. Identified Solutions and Scope of Transfer
On the basis of the outcomes of the above process the solutions contained in the database were classified and divided into six categories, each one defined by specific criteria:
1. Most promising solutions: this category includes solutions not available (or not working) that are ranked as the most promising for transfer by at least 30% of experts, very much beneficial for maritime safety by at least 50% with a high impact on safety.
2. Promising solutions: this category includes solutions not widely available (or not working) that are ranked as the most promising for transfer by at least 20% of experts, very much beneficial for maritime safety by at least 50% with a moderate impact on safety.
3. Promising solutions to be further investigated: this category includes the solutions evaluated as promising in terms of safety impact but already existing in the maritime domain.
4. “Nice to have” solutions: those with less expected benefits for maritime safety;
5. Less beneficial solutions: those with no clear benefits for maritime safety
6. To be excluded from the transferability analysis all those solutions perceived as too specific for the aviation domain or useless for maritime safety.
Out of 166 solutions included in the database 73 were classified in the first four categories while the others were excluded. Within the first 4 categories, 9 solutions were classified as most promising, 6 as promising, 10 as promising to be further investigated and 48 as “nice to have”.
5. Database of Solutions (resilience resources)
The main outcome of this task is the Database of Resilience Resources, an Excel file that hosts all the identified resilience resources (solutions) implemented in air transport. The Database represents a first attempt of organising in a systematic way the aviation safety knowledge according to resilience engineering principles. All the solutions have been classified according to the “Area of resilience improvement” parameter, which could help the database users in the selection of all the solutions that, combined together in a proper manner, could support the system/organisation in facing a certain safety problem.
6. Feasibility Report for the Identified Solutions
The main outcome of the feasibility analysis is that all the shortlisted solutions can be considered ‘feasible’ in that they match at least four of the seven criteria identified. These seven criteria are: precision of terms of reference, similarity of terms of reference, costs/benefits evaluation, benefits, development costs, application level, processes impacted by the adoption
The main added value of the work carried out in task is that the methodology identified and followed proved an effective and efficient way of evaluating the feasibility of transferring solutions from one domain to another. This methodology can be used for any domain – it is not specific to maritime or aviation and can easily be adopted to drive the transfer of innovation (safety or otherwise) from one domain to another.
7. SEAHORSE Implementation Strategy
The methodology developed in this task is proven to be an effective and efficient way of evaluating the feasibility of transferring solutions from aviation to maritime. The in depth feasibility analysis clearly reveals that successful resilience solutions/best practices can not be utilised directly in maritime domain as ‘Off the Shelf’ approach does not work in this case. Therefore, particular attention has to be paid to the customisation process to ensure an effective adaptation of the solution to the new domain. With this respect, the experience related to the transfer to the maritime sector suggested some key elements that need to be taken into account such as:
• Local culture: this element can play a crucial role since it can affect the way in which the solution is implemented, especially in large organizations where different cultures may coexist.
• Decision chain: it is relevant to identify from the beginning the decision chain and the people responsible for the solution implementation.
• Target of the solution: People or areas of the organization that are the target of the solution should be clearly identified to take into account their specific needs in terms of effort and availability.
This strategy, although tested on specific cases, can be used for any safety solution the maritime sector would like to import from other domains. In fact the steps defined to drive the transfer process are independent of the vagaries of either the ‘departure’ or ‘destination’ sectors implicated in the transfer of innovation.
8. Database of Identified Workarounds on Board Ships
The deviation from SOPs is a phenomenon that affects the entire shipping industry and represents a threat to the safety of shipping. Due to a wide range of reasons, seafarers often deviate from SOPs in order to accomplish their tasks and complete their duties. In this report this phenomena is defined “workaround”. The consortium managed to capture responses in a very sensitive and controversial area and 259 workarounds and all these data are captured in Deliverable 4.1 database. Initial scan of the survey indicated that the most common workarounds are located in the areas of reporting paperwork, navigational rules and standards, and personal protective equipment.
The dominant reason of performing workaround is time constraints. When deviating from SOPs, crewmembers develop their own way to overcome the given difficulty, which will be defined in this report as a workaround. It is clear that the alternative devised way, the workaround, can be safe and provide a more optimal means of working; however, these alternatives can also bring risks.
9. Risk benefit analysis of performing workarounds
Detailed analyses of the SEAHORSE smart survey questionnaire data collected as part of the SEAHORSE project has been presented with the primary aim of establishing risks and benefits of workarounds as compared to relevant SOPs..
Survey also captured the workarounds in terms of commonalities, type of operation, onboard locations, reasons of opting workarounds, benefits and risks of workarounds as deemed by the reporting personnel. Most of the SOPs that were altered fell under the categories of ‘pre-requisites’ (defined as the formalities to be completed before commencing any work) and ‘recommended practice’ (procedures outlined in the documentations to undertake a given task). Avoidance of ‘personal protective equipment’ was also very common. Interestingly, more than 77% reported workarounds were being followed by other seafarers as well. Though, engineering cadre was found to be a little stricter on following the SOPs on as is basis. Workarounds have been adopted by the respondents, practically, at all onboard locations.
As such, ‘time constraints’ was the most wide spread reason identified by the seafarers. ‘Insufficient manpower’ and ‘crew fatigue’, appeared to be other variants of the lack of time, were also very common. ‘Physical constraints’ was the second most common category that could be linked with the design aspects of the ship and its equipment. As far workarounds benefits go, convenience was considered by respondents as the dominant factor followed by the time efficiency. The two other dominant reasons for altering the SOPs were the potential improvements in performance and safety aspects of SOPs. It was also interesting to note that in a few cases, the respondents themselves found no specific benefits of following workarounds.
10. SEAHORSE Smart Procedures Methodology
The SEAHORSE Smart Procedures Methodology, developed within the SEAHORSE Project, with the ambitious aim of providing the maritime industry with a robust method for the collection and assessment of workarounds practised on-board vessels.
The unique contribution of this methodology is the development of a method to capture workarounds practiced on-board vessels.
A questionnaire was developed to provide a structured means of capturing information about workarounds, which has not previously been done. The assessment of the alternatives is performed based on a pre-existing method for ranking and requires the involvement of a group of experts. A set of attributes is provided in order to support each reviewer in the assessment of the workaround. The assessments carried out by each expert are aggregated by factoring in the importance of attributes and experts through weightings. Crucially, the methodology closes the review loop by providing feedback of the outcome of the review to the original seafarer who submitted the workaround.
The study clearly highlights that the maritime industry still has significant issues concerning safety. Good practices should be formalised within existing procedures while bad practices should be prohibited. Furthermore, in many cases the reason why they perform this workaround is more important than the workaround itself.
Automated Web–based tool was developed to utilise the procedural improvement methodology allowing the suggested workarounds to be assessed very quickly while maintaining the anonymity of the proposer.
11. Results of the initial validation study and modifications
In order to validate the methodology, a one-day workshop was held in Rotterdam in 2015. The workshop consisted of 40 expert participants. A total of 7 workarounds had been chosen to be assessed by 4 different groups
Paper-based booklets were distributed containing the review forms and each participant reviewed the workarounds. All data was manually entered into the computer system at a later date and analysed. The participants also had a chance to test an initial version of the online SEAHORSE platform; each participant had 15 minutes to trial the system. Feedback was collected from the participants throughout the workshop and a final survey was administered to them at the end of the reviews.
Key Findings
The result of a workaround varies between 0 and 1. In total, 35% of the workarounds were found to be better than the SOPs. This creates further room for developing SOPs in order to enhance maritime safety. Treating the bad practices as a learning opportunity will lead companies to understand the reasons behind performing risky workarounds.
Everybody can access and use the tool either submit a suggestion or perform the assessment without gathering around the same table. The review of the workaround is calculated and the results are automatically disseminated to all the employees of the company electronically. Initial mock-ups were drafted to build up the architecture of the software. This methodology, embedded within the software, is positioned to improve SOPs in organisations working in the maritime industry and therefore lead to improved shipping safety. A user guide is available together with the tool to assist the users to utilise the tool effectively.
Such a system is required to allow the identification of unstandardized practices on-board vessels and to allow these to be objectively scrutinised by expert reviewers. The system allows opportunities to improve safety on-board vessels not only by making persons aware of unsafe practices but to also by making improvements to the overall design of the SOPs, improving the way in which they are executed.
12. Report on Multi-level Resilience Model
The aim of Task T5.1 was to define what human oriented resilience is and how it is assessed within an organization, so that the effects of resilience solutions or tools developed within the consortium can be assessed.
Based on the fact that existing accident analysis methods do not possess all the desired characteristics to measure the resilience of a maritime socio-technical system (STS), the Resilience Assessment Method (RAM) was developed in WP 5 of the Seahorse project
The assessment tool consists of a resilience questionnaire, a multi-level resilience matrix, and a set of resilience solutions that organizations may adopt to improve their resilience. Any maritime organization seeking to improve its resilience to unforeseen disturbances or challenges should start by identifying which safety critical operations/ functions (in different operational phases) should be analysed with the resilience assessment tool. Examples of safety-critical functions are mooring, navigation, lifting, unloading, etc. After this functional decomposition has been completed, the resilience questionnaire is used by the organization to describe the function’s operational demands, outcomes, resources and coping strategies. The questionnaire assesses the quality of the available resources and strategies in order to achieve the required outcomes, given the operational demands. The output is shown in a multi-level resilience matrix, with rows depicting the different levels in an organization (individual, team, organizational), and the columns showing the scores on the four essential resilience abilities: anticipation, monitoring, responding, and learning. An organization scoring low on, for instance, team learning, may then be advised to implement a human factors training program in order to strengthen this aspect.
The method was applied retrospectively to six maritime incident scenarios. The questionnaire was applied to all scenarios. Initial results show that the RAM is a promising method. The questionnaire offers the possibility to gather more information in a structured way. The validation was, however, done by a single person. This means the inter-rater reliability has not been tested yet. To overcome this, the current study should be repeated in future studies and a comparison of the results should indicate whether different researchers will get the same results or not.
One of the goals of this study is to develop a tool that is able to analyse accidents, the current (resilience) state of an STS and to help design new STSs. However, the RAM has not yet been tested in its ability to measure the current state of an STS and design new STSs. In order to investigate if the RAM is able to analyse current STSs, it should be tested whether the method is able to identify latent variability within a system. Similarly, for designing an STS, trying whether the method has this ability is a good way to test it.
The multi-level resilience assessment tool was further developed and validated in an interactive workshop with stakeholders during the public SEAHORSE meeting in Rotterdam on October 13, 2015.
More than 40 maritime partners participated by providing suggestions of currently-used resilient solutions to particular challenges the maritime industry is faced with, for instance, language problems or reduced crew sizes. The workshop was highly successful in generating these solutions, which in the course of the project will be implemented in a Virtual Platform and made available, together with the resilience questionnaire, to the maritime industry, as well as any other interested parties.
In order for the operational procedures on board to enable successful performance of safety critical operations in harsh environments such as the maritime system, resilience is required. Therefore the SEAHORSE project’s approach will be to implement resilience engineering principles in an integrated and innovative manner, taking into account the knowledge generated and experience gained by the aeronautical industry. By utilizing the SEAHORSE project’s multi-level resilience assessment tool, RAM, the user will be provided with new resilience resources to prevent a decrease in system performance, allowing the system to return to baseline performance much more quickly and display greater resilient behaviour.
13. Report on Multi-level Resilience Assessment Methods
The objective of T5.2 was to support the application of the Multi-Level Resilience Model through the development and specification of measures to assess the impact of resilience solutions on safety, efficiency and the operation. The strongest link with earlier work packages is that with WP3, in which solutions from the aviation sector were identified to improve the resilience of the maritime sector. The task for T5.2 then was to systematically develop measures to assess the impact of their implementation as specified in the D3.3 “Feasibility Reports for the Identified Solutions”.
The practical objective of T5.2 was to provide end-users and operators with practical tools to be able to evaluate the effectiveness of the implementation of resilience resources and understand if the intended safety goals are actually achieved. Task 5.2 developed the metrics to assess the impact of applying resilience resources: what and how to measure the performance outcome for a maritime transport system as a result of a resilience intervention. The focus of the task was tempered by the Health, Safety & Environment Agency (HSE) guidance on the use and deployment of safety indicators – stating that they should not be overly complex, and should not overburden safety managers who already gather safety data (HSE, 2006). As such the SEAHORSE Resilience Impact Assessment metrics focus on “quality not quantity” to ensure, insofar as possible, that busy management teams have the correct, critical indicators to measure the impact of implementing a resilience solution, rather than having too many unnecessary indicators – adding to an already high workload.
The aim of this task was to identify the “metrics to assess the impact of applying resilience resources” and “thresholds” for these metrics so that the implementation of the resilience solutions identified in T3.4 can be measured and evaluated. The solutions were evaluated as part of WP3 in order to integrate them into the existing safety management system and all the organisational aspects, and characteristics required for the implementation of the solution were identified and taken into account for the implementation strategy definition. The task focused on the solutions selected within the project but also provides generic impact assessment measures that can be used for any resilience solution implementation – thereby ensuring that the SEAHORSE methodology can be fully adopted in the future. The four solutions selected were as follows:
1. HF Training
2. Pre-flight Checklist
3. Just Culture
4. Mandatory Safety Reporting
For each one of these solutions, a maritime-version is being developed and implemented addressing the specifics of the maritime sector’s operational and safety needs. Important factors such as management support and feedback, the degree to which users can perform with the newly acquired competences and a focus on the actual work in its context are important transfer enablers and are included in the implementation plans developed for each solution.
Task 5.2 ‘Multi-level Resilience Assessment Methods’ had to produce something which companies could easily understand and use. Safety managers already have sufficient workload so the output of the task should not have been overly-complicated or requiring additional training as highlighted by the HSE (2006). If substantive training were needed to get the knowledge across to safety managers and personnel it would fail in its objective. Nonetheless, the task of evaluating the impact of a safety solution is not a facile one and any framework should not be so simple as to be devoid of meaning or power. This resulted in a requirement for a comprehensive evaluation framework which had an easily-understood organising principle. Kirkpatrick’s model was chosen as the model to inform the assessment framework.
1. Reaction - Assessing impact through the elicitation of perceptions and attitudes of those impacted directly by the resilience solution
2. Learning - Assessing impact through the elicitation of new knowledge and learning of those impacted directly by the resilience solution
3. Behaviour - Assessing impact on the behaviour of those impacted directly by the resilience solution
4. Results - Assessing impact on the results that the organisation uses to evaluate its performance
Implicit in each category of analysis is an appropriate enquiry method. For example ‘Reaction’ and ‘Learning’ are best assessed by talking to people and getting their thoughts – are they happy with this solution, does it make their job easier, have they learned anything through using the solution? Assessing impact of ‘Behaviour’ means you want to see if people do things differently so you would need to observe behaviour in a structured way. Finally assessing the impact on ‘Results’ would have a different focus – more on KPIs (Key Performance Indicators) and organisational data. There is also an implicit focus of each category. ‘Reaction’ and ‘Learning’ focus on the individual, ‘Behaviour’ can be the individual or the organisation, whereas ‘Results’ is a more organisational, macro-level category. As such the following three-pronged approach was developed:
• Assessing impact through the elicitation of perceptions and attitudes of those impacted directly by the resilience solution (via a questionnaire),
• Pairing these attitudes with behaviours (either physical real-time behaviours) or artefacts that demonstrate a past behaviour (via targeted observations), and
• Collating (and supplementing) organisational data on accidents, incidents and KPIs which focus on the ‘hard’ outcomes achieved (via document analysis)
The Resilience Impact Assessment Framework has three analysis methods and four levels of analysis. The questionnaire (targeting perceptions and attitudes) focuses on ‘Reaction’ (what do you think of this solution), ‘Learning’ (what have you learned from this solution), ‘Behaviour’ (what do you do differently as a result of the solution), and ‘Results’ (what has this meant in terms of your job/role). Observations and audits (linking attitudes and learning to behaviour) target just the ‘Behaviour’ category from Kirkpatrick’s framework (See Error! Reference source not found. ).
This category is looking at collective or individual behaviour by observing it directly (to see how people are working, are they doing things correctly) or by observing indirect artefacts (or evidence) of behaviours – such as did the company print off the Just Culture statement and display it in a prominent place? Organisational data is targeting the accident and incident data that the organisation will already collect and supplementing this with some additional metrics which focus on results.
In order for organisations to be able to plan, implement and sustain the Impact Assessment process it is necessary to set out a roadmap – detailing who manages the process, how the process runs, who is involved and in what order the assessments take place. The process starts with the Questionnaire. This process is managed by the Safety Manager or equivalent (such as DPA - Designated Person Ashore) who should distribute the questionnaire(s) and ensure that all staff should complete the questionnaire confidentially. Ideally this will cover all levels of the organisation – not just those that are immediately impacted by the solution in question. This is to target any unintended consequences or oversights for other roles not directly impacted. Then the Safety Manager should begin the process of the Observations, targeting real-time behaviour of those using the new resilience solution, and looking at documents or evidence of past behaviour (such as processes and systems being put in place and managed effectively). Once this is complete the focus shifts to the analysis of Organisational Data which is again managed by the Safety Manager and relates to the interrogation of existing safety data in the organisation and the supplementation of this with additional analyses targeting safety, productivity and costs.
These three assessment methods are applied to each of the four selected SEAHORSE resilience solutions: HF Training, Pre-flight Checklist, Just Culture, Mandatory Safety Reporting - and will be tested in WP6 and WP7 through the implementation of solutions. The focus is on these solutions and there are specific questions targeting the specific impact of each solution – given that each solution is different from the other. Having said that, there are ‘generic’ questions and topics which apply to all solutions. This is intended to allow for the assessment of impact of future resilience solutions which may be developed once the SEAHORSE project is finished.
The metrics, the thresholds, and the methodology for managing the impact assessment process specified herein ensures that maritime operators can fully evaluate the impact on their operations as a result of the implementation of the SEAHORSE resilience solution (on safety, on efficiency and on the overall operation). The inclusion of generic metrics ensures that the SEAHORSE transfer methodology is fully sustainable in that it can be carried out in the future for other resilience solutions and the impact of their future implementation can also be assessed – thereby ensuring the legacy of the SEAHORSE project.
14. Results of the Initial Validation Study and Modifications
One of the tools incorporated in the Virtual Platform is the Resilience Assessment Tool. This is a questionnaire assessing a company’s resilience. In order to provide recommendations (‘solutions’) based on the questionnaire results, it was necessary to validate and amplify the solutions developed in WP3 with maritime solutions for particular challenges faced by a range of maritime companies. This was accomplished in the Public Workshop held in Rotterdam on October 13, 2015. The resulting solutions were incorporated in the Resilience Assessment Tool and became part of the Virtual Platform.
The SEAHORSE Virtual Platform was subject to an Initial Validation during the 8th Technical Meeting (TM8) held from the 7th of March until the 9th of March 2016 at the Istanbul Technical University/Maritime Faculty. The first step of the validation design process was the identification of the maturity level of the SEAHORSE Virtual Platform. Once it was identified, the validation objectives were defined and the validation approach was chosen. The overall validation design process was based on the European Operational Concept Validation Methodology (E-OCVM) [1] which is a framework widely used in the aviation domain.
The initial validation consisted of three parts: (1) validation questionnaire; (2) open discussion; (3) industry stakeholder workshop. As part of the validation questionnaire, individual project members of the SEAHORSE consortium including maritime experts answered questions on the clarity, feasibility, usefulness, and completeness of the virtual platform. In the open discussion session, facilitated discussions were carried out, aimed at identifying strong and weak points of the virtual platform. In the industry stakeholder workshop, an in-depth session was conducted during which the entire Resilience Assessment Tool was completed by three participating maritime companies.
The Virtual Platform, in its first version, was viewed by most respondents as fairly complete, logical, useful, and feasible. The Resilience Assessment Tool was seen by many as a powerful way of getting insight into the current state of a company’s resilience, as well as a good way to convince management to invest in safety.
Of course, there were also suggestions for improvement as well as some sceptical remarks about the feasibility of the tools incorporated in the Virtual Platform. These suggestions and remarks have been dealt with in subsequent versions of the Virtual Platform. Many comments and suggestions were aimed at the way individual items were phrased; these items were subsequently modified accordingly or altogether deleted. Other suggestions dealt with the way the tools were structured overall or the way they had to be filled in. Most of these comments could also be addressed.
An important issue that arose during the Technical Meeting in Istanbul was how to incorporate the various databases into the Virtual Platform, for instance, the crew database, ship database, and company database. After a long deliberation and weighing of various options, and considering the scope and feasibility of the development of the Virtual Platform, we decided not to link directly the Virtual Platform with existing databases, as this would, first, be too prohibitive in terms of development effort, and, second, not yield any additional insights into the resilience of a company.
We therefore took a two-pronged approach: first, by looking bottom-up at what currently available databases offer in terms of information, and, second, by looking top-down at what additional database information needed to be obtained in order to strengthen the items in the Resilience Assessment Tool.
The resulting set of questions was added as an additional layer to the Resilience Assessment Tool. These questions should be answered by respondents prior to filling in the other items. Answering the questions requires consulting various company and ship databases and thus provides a baseline for answering the subsequent items.
15. SEAHORSE Virtual Platform for Resilience Design & Assessment in Maritime Transport
The SEAHORSE Virtual Platform is a functioning prototype which gathers all the research and models developed within the SEAHORSE Project. It represents a virtual space where maritime companies can find different types and levels of assistance for maintaining and improving their resilience structure and enhancing safety performance. With its multi-dimensional and multi-level operational concept, it seeks to achieve the ambitious goal to support maritime companies from high level decisions on resilience matters to more practical and specific operational aspects. In particular, its aim is to provide shipping companies with a comprehensive support in their path towards safety and resilience, where “comprehensive support” stands for:
• Resilience assessment and grading;
• Identification of gaps in the resilience structure;
• Identification of weaknesses and/or non-compliances that may lead to errors and accidents;
• Identification of resilience resources and solutions and decision support in order to address the identified gaps, weaknesses and/or non-compliances;
• Provision of practical guidelines for the implementation of the selected solutions;
• Provision of structured assistance during the implementation process;
• Ex-post assessment of initiatives undertaken;
• Standard Operating Procedures assessment and re-design;
• Collection and assessment of workarounds practised on board vessels;
• Comparison of assessed workarounds and SOPs and identification of the most effective and safe ways of working;
• Audit of the crew quality at both individual and team levels;
• Identification of strengths and weaknesses of the crew, training needs and suitability for specific types of ship.
In practical terms, the Platform accommodates four tools, each of which is made up of three layers connected and integrated to each-other (input, database and output layers):
1. Resilience Assessment Tool (RAT): this tool aims to support end-users in the assessment of the company level of resilience and in the identification of gaps/weaknesses/non-compliance that may lead to errors and accidents. It also provides tailor-made safety and resilience solutions together with some practical guidelines for their implementation;
2. Impact Assessment Tool (IAT): this tool intends to provide the end-users with a practical support for monitoring the implementation process of safety/resilience solutions and assessing the impact of the initiatives undertaken;
3. Procedure Improvement System (PIS): this tool aims to provide the maritime industry with a concrete support for the collection and assessment of workarounds practised on-board vessels. It allows users to capture workarounds performed by seafarers within a company, assess them and compare them to SOPs in order to find the most effective and safe way of working;
4. Individual Crew Quality Audit Tool (ICQAT): this tool intends to support end-users in auditing the crew quality at both individual and team levels and thus in identifying strengths and weaknesses of the crew, training needs and suitability for specific types of ship.
From a technical standpoint, the SEAHORSE Virtual Platform is modelled with a web-based solution exploiting the WordPress framework and can be deployed in the existing IT infrastructure of the end users or, alternatively, outsourced to external services. Indeed, these are capable of running either together or as standalone instruments.
The SEAHORSE Virtual Platform can be considered as a functioning prototype positioned at the end of the pre-industrialisation phase. Indeed, with the typical limitations of a system at this maturity level, the Platform represents a software ready to be installed into a maritime company. Furthermore, this prototype has an open architecture which means that it is capable of accommodating other tools and guidelines as the interests of end-users require.
16. SEAHORSE Multi-level Resilience Tool
This chapter’s aim is to provide a comprehensive overview on the concept of the Multi-Level Resilience Software (MLRS) and to bring to light the added value that the SEAHORSE Project has generated for the entire maritime industry through the development of this software.
The MLRS represents a substantial part of the broader SEAHORSE Virtual Platform where all the research and models developed within the Project are gathered. This software is the part of this prototype developed on the purpose of providing maritime companies with a comprehensive support in their path toward resilience. Its operational concept comes and develops within the new research area of resilience engineering where resilience is conceived as the ability of system to cope with and manage demands, with the help of available resources, such that performance is sustained or even improved (Kamphuis et al., 2013). The main goals of this software are summarised in the below points:
• Resilience assessment and grading;
• Identification of gaps in the resilience structure;
• Identification of weaknesses and/or non-compliances that may lead to errors and accidents;
• Identification of resilience resources, solutions and decision support in order to address the identified gaps, weaknesses and/or non-compliances;
• Provision of practical guidelines for the implementation of the selected solutions;
• Provision of structured assistance during the implementation process.
These six points bring to light the capability of the MLRS of assisting end users in developing, assessing and maintaining a resilience structure that is tailored to their particular needs and operational requirements. It is a virtual space where maritime users can embark an assisted restructuring process of their companies’ resilience structure. This process, metaphorically called ‘resilience journey’ in this document and involves everybody in the organisation who is somehow related to the subject of the analysis, regardless of the roles played in their company. Workers are usually asked to report their perceptions about the safety needs of the organisation and/or to provide their feedback after the implementation of a new safety intervention. Broader is the role of managers who are also supposed to further analyse the feedback provided by workers, decide the interventions to implement and assess the impact of the initiative undertaken over time.
Three are the main sections of the software which are in turn composed of different sub-sections.
1. ‘Resilience Assessment’ – It represents the starting point of the resilience journey and includes:
a. The Resilience Assessment Tool (RAT) which is a diagnostic instrument for the determination of organisational strengths and weaknesses with respect to resilience;
b. The Resilience Assessment Results which is the sub-section where an overview of the status of the resilience structure is provided and a tailored set of implementable solutions is advised.
2. ‘Solution Implementation’ – It is a sort of a Control Panel where users (managers only) can enable the passage from the resilience assessment phase to the impact assessment one.
3. ‘Impact Assessment’ – It represents the final step of the assessment process and includes:
a. The Impact Assessment Tool which is an instrument for gathering information in a structured way about what the workers think has changed as a result of the implementation of a specific resilience solution;
b. The Impact Assessment Tool Results which is the sub-section dedicated to the review of the results of the Impact Assessment Questionnaire;
c. The Impact Assessment Metrics which intends to provide the user with a practical support for the quantitative analysis of the impact produced by the implementation of one or more of the proposed resilience solutions.
The resilience journey so far described might appear as a strict linear process. However, the fact that it is composed by consecutive steps does not mean that the MLRS offers a rigid and one-way path. In fact, the MLRS is a flexible system capable to provide a dynamic assistance. All the sections of the software are conceptually designed to run simultaneously, thus the user could work at the same time on different steps of the resilience journey.
The implementation of the conceptual Framework into an automatic software represents a significant leap forward for the provision to marine companies of a practical support for the enhancement of resilience. The major benefits that the MLRS would bring to a maritime company are summarised in the below points:
• Safety Management would be able to perform a resilience assessment whenever and wherever desired;
• The software allows the involvement of important stakeholders/contractors in the assessment process;
• It provides a realistic reconstruction of the resilience structure since it starts from worker’s perceptions and concrete operational needs;
• It gives an immediate understanding of the wellness of the organisation;
• It provides safety personnel with justifications (quantifiable data) for their safety initiatives and proposals of new investment and it represents a support for decision-making;
• It would bring improvements in the transparency of the management activities;
• Indirectly, the usage of the MLRS would generate positive impacts on the organisational safety culture.
The MLRS is a software ready to be installed into a maritime company. However, as any prototype, it has some limitations on both the technical and conceptual sides. The possibility of benchmarking within the maritime industry seems to be the capability of most interest to end-users. The non-implementation of this feature was due to the fact that it requires higher maturity level, hence it goes beyond the scope of the SEAHORSE Project. However, since the benchmark functionality would significantly increase the applicability and usefulness of the software, this improvement is advised as one of the priorities for further development activities.
17. Validation scenarios
The preceding work packages to the present one, Work Package 6, develop a number of outputs. The purpose of this document is to present 12 validation scenarios to be used to test these outputs to ensure the validity and quality of these solutions. The validation scenarios are fundamental to the project. Well-designed validation scenarios will lead to a higher level of credibility and trust in the solutions, as well as certainty in the quality and applicability to maritime organisation.
The validation scenarios are as follows:
Safety Climate Assessment in Shipping Companies: This validation scenario’s purpose is to test the surveys developed under the safety culture assessment framework. The surveys were carefully developed with an interdisciplinary group to ensure that they capture the right information to conduct a comprehensive analysis. A detailed literature review was performed not only in the maritime industry but also in other industries to identify the requirements of an appropriate safety culture questionnaires and assessment methods. The surveys will be distributed to three different shipping organisations, of varying sizes and specialities and conducting extensive analysis on the survey results.
These companies were chosen from different operation types in order to address all aspects of different operation types. The sizes of the companies are given below:
• Container Shipping Company – 300 Employees
• Bulk Carrier Company – 87 Employees
• Tanker Shipping Company – 2220 Employees
The questionnaires were designed to be accessible online for a month. The total of 4 month is required to validate the safety climate assessment and improvement framework is from scratch. The required time to finalize validation can be broken down as below:
• Information about the planned safety climate surveys, preparations within the company – 1 month
• Distribution and collection stages of the survey – 1 month
• Analysis of the results and validation 2 month
SEAHORSE Procedure Improvement System in Calmac: This validation scenario’s purpose is to test the Procedure Improvement System. Implementing the system in the organisation, the reviewing and results aspects of the system are to be validated. This will be done through the processing of 20 existing cases.
SEAHORSE Procedure Improvement System in Danoas: This validation scenario is to test the Procedure Improvement System, it is implemented within the shipping organisation Danoas.
Human Factors Training in Jumbo: Human factor training is to be tested in Jumbo. The training is to be delivered through training modules at Jumbo’s office.
Standard Operating Procedures Development for Jumbo: This validation scenario is designed to test the Standard Operating Procedures Best Practiced Guidelines. It is done by using the guide to turn an existing SOP into the correct format.
Standard Operating Procedures Development for Sener Shipping: The SOP Best Practices Guide is to be tested in a simulator setting.
Pre-Departure and Arrival Checklists: The checklist is to be validated in Calmac who will implement it within their organisation.
Mandatory Safety Reporting: The validation scenario tests the Mandatory Safety Reporting in a tanker company.
Resilience Assessment: The resilience assessment is tested through expert workshops.
Impact Assessment: The Impact Assessment tool is to be validated through expert workshops.
Just Culture: The Just Culture solution is to be tested in through workshops. The workshops will comprise of a set of experts.
Observation Study in Calmac: This validation scenario is to test the methodology developed for conducting observation studies. The methodology is to be validation by conducting an observation study on-board two vessels of a ferry operator.
18. Task Based Validation Report
Achievements
Checklists were validated both in operational settings (CALMAC, Chapter 2) as well as simulator settings (ITU, Chapter 3, SAMK, Chapter 7). Standard operating procedures were validated on the basis of a SOP Best Practices Guide (SU/KAHN, Chapter 6). Human Factors training was given by ESM to KAHN and described in chapter 4. Mandatory safety reports and safety bulletins were validated by means of consulting end users and maritime experts (APA, Chapter 5).
Overall, the conclusion is that methods and strategies from the airline industry can be successfully implemented in the maritime sector. Given existing regulations, some of these methods have been mandatory for a long time. However, we also discovered that the maritime sector has its own characteristics that sometimes impede the absorption of methods and strategies from the airline industry. For instance, the fact that bridge layouts are not as standardized as they are in cockpits, may lead to less uniformity in the design of checklists. It should also be remembered that proper design of checklists (or SOPs) is a demanding exercise and that the maritime sector can learn a lot from the aviation sector in this respect (CALMAC enlisted the help of an experienced Airbus captain to help in the design of the checklists). As another example, human factors training may be provided to maritime personnel, but the contents of the training should be geared to the maritime sector and should not be copied from the aviation sector.
Value added to SEAHORSE
This task contributed to the overall goal of the SEAHORSE project by demonstrating the value of several solutions that have been found to enhance safety in the aviation sector and by customizing them for the maritime sector.
19. Validation report: SEAHORSE “Procedure Improvement Methodology”
The SEAHORSE project has developed a wide range of outputs that can be used to support shipping organisations to better improve the safety and resilience of shipping. One of the outputs is the Procedure Improvement System, a tool that supports the collection, assessment and review of SOP suggestions provided by employees of a maritime organisation. At present, the maritime industry lacks a rigours and robust means to collect and review unstandardized procedures. The developed Procedure Improvement System provides a robust and repeatable means to review unstandardized procedures.
The purpose of this report is to present the validation of the SEAHORSE Procedure Improvement System; the validation is conducted in two distinct ways. The tool was tested by Danaos and by CalMac. Danaos tested through a full-scale validation, with the Procedure Improvement System augmented such that it can be used offline. CalMac validated part of the Procedure Improvement System through the testing of a number of cases.
First, the tool was tested in Danaos for a period of four months on 33 vessels. Over the four month period, the Procedure Improvement System was validated based on a number of indicators: 1) The Reaction Indicator, 2) The Learn Indicator (LI), 3) The Behavior Indicator (BI) and 4) The Efficient Indicator (EI).
The Procedure Improvement System was tested in 33 vessels in the fleet of Danaos (container ships). Crewmembers submitted 47 alternative controls to be evaluated to the tool. In total, 30 out of 47 had to do with operation on the bridge of the ship and the rest 17 for other.
Secondly, the tool was tested on 20 cases in CalMac, training was given to the team in CalMac on how to use the system. Initial engagement with CalMac for the tool began in June 2016 and the final feedback was collected in October 2016. The team selected 20 cases to be reviewed, these were a mixture of cases taken from their current system and some developed based on experience of the team. Reviewers were appointed and accounts were created for each of the reviewers. All four reviewers were selected to review each of the cases.
Within CalMac, the 20 cases were successfully reviewed, each case was reviewed within two weeks. The system was shown to be able to be used to successfully review a set of cases. In comparison with expected results, four of the 20 cases were identified as not being correct in the opinion of the head of the team. While the tests returned results for all of the cases, it was noted that for some there was only a marginal difference between the suggestion and the SOP and it was expected and desired to have more significant differences between the suggestion and the SOP. Overall, the suggestions were reviewed far quicker than their current means. The system was felt a good way to meet organisation aims of improving safety and meeting requirements of the regulatory bodies. Unfortunately, some key issues were identified as limiting the organisation from fully implementing the system. These include: offline functionality, issues with non-electronic Safety Management System and the time needed to make it fully electronic and lastly developing a ‘bank’ of experts to call up as needed for reviewing.
20. Validation report: Multi-Level Resilience Model for Maritime
The overall goal of the EU FP7 SEAHORSE Project is to improve the safety and resilience in the maritime sector by focusing on Human Factors. To achieve this, the project used a resilience engineering approach. To this end, a Multi-Level Resilience and Impact Assessment Models were developed, and, based on those models, the Resilience Assessment Tool (RAT) and Impact Assessment Tool (IAT) . These two tools were then integrated into the Multi-Level Resilience Assessment Software. This deliverable describes the validation activities of the overall resilience Framework which took place during the 9thTechnical Meeting in Athens and in a successive dedicated session with captains of industrial partner KAHN.
The RAT is a questionnaire that assesses a maritime company’s resilience at four levels: multi-party, organizational, team, and individual level. At each level, four cornerstones of resilience are assessed: monitoring, responding, anticipating, and learning. Each of the cornerstones, at each level, is assessed by multiple self-report questions. The questionnaire has been validated on a number of incident scenarios and during several workshops with maritime stakeholders (as described in earlier WP5 deliverables). The RAT results in a diagnosis of a company’s resilience structure. The diagnosis results in the recommendation of particular resilient solutions that maritime companies may adopt in order to enhance their resilience.
The IAT aims to enable maritime companies to assess the impact of implementing resilience solutions after having implemented such solutions.
The first Task Based validation activity of the SEAHORSE Multi-Level Resilience Framework was carried out during the 9th Technical Meeting (TM9) held from the 30th of May until the 1st of June 2016 at the office of DANAOS Shipping in Athens. The workshop contained a task-based validation of the Resilience Assessment Tool (RAT) and Impact Assessment Tool (IAT) by means of collecting end-users’ feedback on three dimensions: content, usability and applicability (referred to from now on as the ‘Validation Workshop’). It consisted in a walk-through exercise in a workshop setting where both questionnaires and group discussions where used. The workshop was attended by 17 experts with different areas of expertise.
The second Task Based validation activity was organised with two Jumbo captains after the workshop in Athens. During this session all items of the RAT were discussed based on comments received during the Athens session. The final version of the RAT contains 142 items in total.
The final results are summarized in the below points:
• Content: indicate that most respondents think thought that the content ‘as is’ was clear to the respondents in terms of its purpose, meaning of questions and different categories. A change in terminology was suggested to make the questions in both instruments more suitable for use in maritime industry. Certain wordings or concepts were not familiar in shipping and needed to be changed. Suggestions were made to use already existing guidelines for maritime vocabulary.
• Usability: almost all respondents indicated that ‘it is clear how they must fill in the questions’ and that ‘In general, the tool is easy to use’. Furthermore, certain specific design issues should be fixed to improve the usability of the tool. Several technical improvement suggestions were processed, like: implement 'go back buttons’, ‘freeze of display/screen sections’ and ‘better indication of which resilience levels the respondent is filling out the survey’ have been implemented accordingly. Including a download options making it possible to download the questions including the scores as .csv files (not Excel, no charts). Also, some recommendations to improve privacy of users and the data collected on the VP was taken care of with state of art technical (ICT) measures and in line with currently applicable EU privacy and cybersecurity regulation requirements. Finally, the Virtual Platform (of which the RAT is part of) will be available in an online version or can be installed on maritime end-users’ own local IT systems.
Applicability: most participants acknowledged that the RAT provided insight into how to assess and improve resilience aspects in their organization. Some suggestions for improvements were: further customization towards maritime, improvements in the design and testing the tool in companies to make it more applicable for their organization.
21. Validation report: SEAHORSE Virtual Platform
The main aim of the SEAHORSE Project is to enhance safety and resilience in maritime operations. The results of the SEAHORSE Project were presented in numerous conferences and workshops to promote these concepts within the maritime industry (See Deliverable 7.4). In order to support this aim, a platform titled the SEAHORSE Virtual Platform (SVP) was specified and developed. The SEAHORSE Virtual Platform hosts all SEAHORSE outputs and allows shipping companies to access the deliverables and utilize the developed tools to enhance the resilience of their organisations
The four main tools of the SEAHORSE Virtual Platform are:
1. Resilience Assessment Tool
2. Impact Assessment Tool
3. Procedure Improvement Tool
4. Crew Quality Audit Tool
The SEAHORSE Virtual Platform is designed to accommodate all SEAHORSE outputs, tools and deliverables. All outputs designed and developed under the SEAHORSE Project can be accessed on this platform by clicking here (http://seahorse.smt.strath.ac.uk/). The Credentials below can be utilized to access to the SVP:
Username: test1 Password:1234
This deliverable outlines the validation activities which were performed to finalise the development of the SEAHORSE Virtual Platform. The user experiences of 30 respondents were collected by distributing a widely-used “system usability scale questionnaire” developed by Brooke (1986). The results provided interesting insights into the user experiences of the SEAHORSE Virtual Platform users. It is evident form Error! Reference source not found. that resilience assessment tool, impact assessment tool and procedure improvement tool were found to be better than the industry wide threshold (68) according to SUS assessment. Only Crew Analysis Tool of the SVP got slightly lower score than the threshold amongst them. This has been already raised as an issue and efforts were made to enhance the user friendliness of the Crew Analysis Tool. The Procedure Improvement Tool is already in use by two shipping companies and negotiations are taking place with three more shipping companies to establish SVP for them. The Procedure Improvement Tool obtained the highest score within the SVP due to the prevalent usage of the tool in the maritime industry.
Administrator rights were distributed to 6 users (profile of system administrator) and their opinions were also collected via the same SUS questionnaire. Administrators dealt with different issues such as assigning new users, setting the appropriate weightings for the assessments while utilising the tools. The user experience score of the administrators of the SEAHORSE Virtual Platform is 78.33. The higher score was obtained due to the fact that administrator users utilised the system more frequently and they had a more comprehensive knowledge of the SVP.
Individual comments were also provided by the users on typos, functional integrity and IT problems faced. These were taken into account by the SEAHORSE Consortium and all the major changes have been accomplished.
Online platforms and websites are unfortunately prone to security issues. Any sort of weaknesses within the system can harm the developed assets quite severely. Therefore, significant importance is given the system security to keep the platform running. Two types of parameters were considered as vital:
1. Dos Attack
2. Brute Force Attack
Both these types of attacks were all generated and tested artificially within the system. The SEAHORSE Virtual Platform was resilient and robust enough to work sufficiently while the system was under attack. The SEAHORSE Virtual Platform is currently stored on the University of Strathclyde’s server, however, if a company would like to host the tool on their own server for some other security or policy reason this is feasible. All SEAHORSE Virtual Platform Tools have a copyright statement to secure the originality of the system and protect it from legal issues.
22. Report on the impact achieved in the task base and full scale validation studies
The aim of the SEAHORSE project is to improve safety and resilience in the maritime sector through transfer of knowledge between the aviation and maritime industries.
WP6 is the work package where the tools and solutions developed throughout the project for this purpose are validated and their impact assessed, confirming their applicability and implement ability for maritime companies.
The deliverables describe the different validation techniques covered in the work package including scenarios, use of simulators, interviews, assessments and by comparison within fleets of similar vessels. The validation methods involve consortium maritime partners as well as maritime advisory board members and selected external maritime stakeholders.
The tools and solutions developed and presented within WP6 present a compliment of implementable solutions for ship operators and shipping companies as validated by aviation and maritime experts. These range from software solutions including the SEAHORSE platform, a multi-tool application that includes the Procedure Improvement System (PIS), Maritime Resilience Model and Crew Quality Audit Tool to training and operational proposals including Human Factor Training programmes, Mandatory Safety Reporting and Checklist recommendations for the maritime industry, influenced by analysis of best practices in the aviation and maritime industries.
Furthermore, as evidenced by feedback from aviation consortium partners and the advisory board, many of the developed solutions could also be readily adapted and utilised by the aviation industry, emphasizing the cross transfer of best practices between the two transport sectors.
Deliverable 6.6 describes the impact that the developed solutions have had on the maritime companies within which they were validated. In summary these are;
Human Factors Training in Jumbo Shipping
A training plan designed specifically for Jumbo Shipping was implemented by Human Factors training experts(TNO and ESM) with adaptation to the maritime operator’s needs. Similar customisation would be required for participant maritime companies to ensure the applicability and effectiveness of the proposed Human Factors training provided. The impact of the training was then assessed by in company workshops and interviews. Finally, the participants were asked to complete the SEAHORSE impact assessment questionnaire. The feedback shows potential for significant impact of the solution within the company, especially when tailored to the company requirements. Useful information gathered was helped to inform and update the solution, with additional emphasis on “Train the trainer” sessions and appropriate adaptation to meet the seniority and skills levels of the trainees.
Standard Operating Procedure Development for Jumbo Shipping
The recommended process for Standard Operating Procedure Development includes assessment of the procedure for improvement, review of the proposed improvement and update, development of an SOP manual and communication to the company, implementation and feedback. This process was utilized in Jumbo Shipping with focus on SOPs relevant to their requirements.
The guidelines were found to be implementable and were followed with the approach broadly validated. Feedback confirmed that impact within the company requires adjustments and customisation of the procedure to the operator, its needs and structure of the company. Additional SOPs from Jumbo Shipping will be developed by the same process and their impact assessed by crew feedback questionnaires after project close.
Checklist Impact assessment by ITU
Checklist proposals for ‘Arrival in Port’ and ‘Preparation for Sea’ were conducted using simulators at the Full Mission Ship Handling Simulator at Istanbul Technical University’s Maritime Faculty. A series of questionnaires was then carried out to capture the qualitative feedback which then led to the checklists being re-structured.
The checklists were seen as having significant time saving impact on ship operators. An additional constructive proposal was that self-test functions should be included in electronic systems to save time during bridge checklists.
Checklists and Observation Study in CalMac Ferries
Checklist solutions were implemented across the CALMAC fleet allowing for fleet wide validation via similar vessels operating in similar environments. The checklists provide standardisation whereas different systems previously operated on different vessels.
Through questionnaires and feedback, the solution was deemed to be well perceived by crew and office staff with further refinement potential and additional reference to ‘information sheets’ to be included. Furthermore, the checklists were regarded as delivering benefits for CALMAC vessels in operation and during manoeuvring including for standardising bridge protocols. Safety improvements to be viewed over time are anticipated to include the elimination of single points of failure and the avoidance of high-potential safety incidents.
Mandatory Safety Reports and Safety Bulletins
Solutions developed regarding Taxonomy, Safety Report and Safety Bulletin templates were validated in a range of shipping companies and maritime partners.
Taxonomy validation and methodology was validated in a Chemical tanker company and by Sener Shipping. The taxonomy was viewed as implementable and useful and updated following feedback of missing criteria during validation. A standardised taxonomy was viewed not just as useful but essential to provide the most effective capture of incidents, analysis of reporting data and for accurate comparison of data.
Safety Report templates and Safety Bulletins were developed with consultation from maritime consortium partners, external maritime companies and by an external crewing company. Safety Bulletins were then further validated within AP&A and initial impact feedback gathered was positive, enhancing perceptions of management’s positive attitude to safety improvements and professionalising and standardising company safety communications to staff and crew. Further impact assessments are also to be carried out after 3 months via questionnaires in order to provide a sufficient period for regular bulletin circulation and awareness within the company.
Resilience Assessment Tool and Impact Assessment
The Resilience Assessment Tool includes a resilience questionnaire, a multi-level resilience matrix, a set of resilience solutions to be readily implemented by ship operators. Accompanying the Resilience Assessment Tool an impact assessment tool was developed to practically evaluate the impact of implemented resilience solutions.
Feedback on the tools was gathered via varied validation activities, involving different workshops with end-user stakeholders in the maritime industry. Most participants acknowledged that the tools provided insight into how to assess and improve resilience aspects in their organizations and could provide constructive guidance to help enhance safety through resilience in maritime companies. Specific feedback was also integrated into upgrades and revisions of the assessment tool, especially items regarding usability.
Just Culture
The solution for Just Culture was validated within a European shipping company. Stages of implementation include an initial Just Culture Survey, development of Safety Performance Indicators, company documentation review and review of application of just culture process in incidents or occurrences.
Initial impact of the solution was viewed positively with clear benefits for the shipping company, however due to the nature of Just Culture and the time required to embed the solution within the company, periodic annual reviews are recommended to fully assess impact on company culture.
Crew Quality Audit Tool in Sener Shipping
The Crew Quality Audit Tool, designed to improve employment quality through quantitative measurement of employee performance, was validated in Sener Shipping. Seafarers were assessed and evaluated against different criteria A-) Knowledge and skill base, B-) Adaptation to safety rules, C-) Professional behavior, D-) Leadership and social behavior, E-) Personal criteria, F-) Adaptation to Sea and ship life.
The Crew Quality Audit Tool was measured against an existing tool in the company and the benefits and potential future impact on safety was notable. The new tool allows the company to reliably measure ability at the job application stage which can enhance onboard safety via improved recruitment of seafarers and their compatability with the rest of the onboard team. Other improvements for the operator include the ability to compare seafarers on an equal basis and monitoring of seafarer’s development and training on deployment and across subsequent contracts.
Electronic Checklists
The electronic checklist validation by another shipping company includes implementation in over 30 vessels. The solution allows users to more effectively manage checklists onboard in a consistent and reliable way. It was also adapted to accommodate the limited internet connectivity on board with the software synchronizing data at intervals with the office.
Through questionnaires and impact assessment, the tool was viewed to be helpful in a number of ways including in time saving for seafarers (improved by auto-fill checklists that can be updated at intervals). The Electronic Checklist was also found to enhance reliability, data analysis and optimize communication between the vessel and office.
Safety Culture Assessment and Improvement
The aim of the safety culture assessment to identify the safety related problems of any shipping company, address the vulnerabilities and develop improvement strategies to enhance maritime safety. Two surveys, one for crew members and one for shore personnel, were developed in order to perform safety culture assessment in shipping companies. The questionnaires were distributed to different shipping companies in order to cover different operation types and create a benchmark for each safety climate dimension.
The results shed light on the strength and weaknesses of the each company. Crew members of the tanker shipping company achieved higher scores amongst the companies. However, the shore staff of the Bulk Carrier Company achieved the higher scores amongst the companies.
The results significantly vary between different vessels of the companies, even different results were observed between sister ships of the same company. These outcomes were thoroughly discussed to gain insight into underlying reasons of these discrepancies.
The safety climate assessment results generated a substantial short term impact within the each company. The companies gained insight into their current safety climate level on ten important dimensions amongst other companies. All of them have already started taking actions to improve the areas where they scored lower. The company managers and researchers from the University of Strathclyde had discussions on the results generated to address the gaps identified. Long term impact will be visible when researchers have enough data to compare the current situation of safety climates in each company and their maturity levels after the implementation of all given action plans.
The confidential assessment reports were provided to each company together with the recommendations and action plans. The majority of the recommendations have been already taken into account in all companies. All the results were discussed with the HSEQ departments in each shipping company and the Strathclyde will continue to support each company at the implementation of action plans.
There is a requirement to make these assessments annually in order to see the trends within the organizations. The safety climate framework will be implemented in a cruise liner and a ferry company beyond the project time to enhance the accuracy of the benchmark. The framework will be promoted to other types shipping companies in order to capture problems of different shipping operations. Ultimate impact will be achieved when numerous shipping companies start assessing their safety climate with the given methodology. In addition to the survey results, in each company, a thorough study will be conducted to investigate all the available KPIs and their positive impact of safety performance. After capturing the attitudes and perceptions of the employee’s and validating those subjective results with objective safety related key performance indicators, it is also of paramount importance to arrange on board observational studies to identify crucial parameters which affect seafarers’ adherence to safety management rules. Existing blame culture is the biggest challenge for the overall methodology. An effective just culture approach should be implemented in order to ensure the quality of the reporting and data collection methodologies.
23. Guidelines and standards
In this deliverable, the aim is to develop best practices guidelines for the shipping companies to enhance their safety maturity levels. Aviation industry’s successful solutions were chosen to be transferred after the feasibility analysis conducted in WP3 and these solutions were tailored for the shipping companies. Best practice documents were developed in order to guide organizations to implement those selected solutions. Each best practices document provides step by step instructions for the shipping companies to implement the solution with their organizations. There are five best practice guidelines given in this document, namely:
1. Standard Operating Procedure Development
2. Just Culture
3. Pre-departure and Arrival Checklists
4. Human Factors Training
5. Mandatory Safety Reporting
6. Safety Bulletins
Each best practices guide describes:
1. how to implement the given solution,
2. the prerequisites to implement the solution,
3. resources needed (time, personnel),
4. the duration of the implementation, and
5. How to assess impact.
All these best practice guidelines were implemented and validated by the end-users. The SOP Development Guide was utilized in a shipping company and they converted their Lifting procedures into a more structured, concise and clear format. The new SOP has been tested in some heavy lifting operation vessels and the other SOPs are now being converted by utilizing best practices guidelines.
The Just Culture solution was tested and validated via workshops and different experts were presented in these workshops. Regarding the Just Culture implementation, it takes a high amount of time to build the trust and increase all the reporting numbers. The results will be seen in a long term basis.
Arrival and departure checklists were developed by utilizing the knowledge gained from aviation. The best practice guideline was implemented to standardize the arrival and departure checklists for a fleet of 10 RoRo Passenger vessels. The developed checklist has the “check and verify” system, this removed the single point of failure within the system. The new checklists were validated in a shipping company and now it is in use in all RO-RO vessels.
Human Factors Training guidelines were also tested and validated in a shipping company. The guidelines were converted into training modules and delivered at the main office of the company. Relevant Human Factors issues were discussed that created a better awareness for the participating officers. The training was rated well by the participants.
Mandatory safety reporting guide aimed to standardize the reporting system and transfer the most relevant knowledge between the parties in a consistent manner. The designers within the consortium finalized the template for each mandatory safety reporting scheme in order to include all the compulsory information. These reporting templates were tested and validated in a tanker company.
24. Training modules
Safety is the common value for all maritime stakeholders and they are looking for means to enhance safety in the maritime industry. The majority of accidents (70-80%) are attributed to human element. A highly skilled and resilient crew is seen as a way forward to decrease human error related accidents and avoid reoccurrences of catastrophes such as the Herald of Free Enterprise, Deep Water Horizon and Costa Concordia. Shipping companies continuously invest in training to enhance their crew members’ knowledge, competency, skills and resilience. The Standard of Training, Certification and Watch-keeping Convention was proposed by IMO to define basic standards of training and create more resilient crew members for the maritime industry.
This deliverable aims to enhance crew members’ resilience levels by providing a set of training modules. These training modules were developed by utilizing the best practices knowledge available in the aviation industry. All of the modules were develop by maritime and aviation experts. The aviation experts checked the modules thoroughly to ensure all the required information is given and the implementation methodology is applied correctly. The maritime experts were responsible to tailor the training modules based on the needs of the maritime industry. Six training modules were generated for the shipping organizations. These training modules are:
1. SOP Development Training Module
2. Just Culture Training Module
3. Checklists Training Module
4. Human Factors Training Module
5. Mandatory Safety Reports Training Module
6. Safety Bulletins Training Module
The developed training modules were uploaded to the SEAHORE Virtual Platform and they are publicly available for the shipping companies. All the trainings were developed in PowerPoint format and are available at the website. This deliverable provides assistance for the readers on how to access each training module at SEAHORSE Virtual Platform. The deliverable was constructed as a reference document in order to direct readers to access the relevant information. All training modules were developed in a self-explanatory way, therefore no information regarding the details of the training modules is provided in this report. It is suggested to use training modules and the best practice document (Deliverable 7.1) together to enhance the outcome of the modules.
All training modules can be downloaded from the SEAHORSE Public Deliverables website. The website can be accessed under the SEAHORSE Virtual Platform by clicking here (http://seahorse.smt.strath.ac.uk/PublicDeliverables/).
Potential Impact:
1-Potential Impact
The main expected impact that SEAHORSE project would like to achieve is to clearly demonstrate that different transport modes can and should work together to share the best practices with practical impact on safety. The SEAHORSE project proposes to address human factors and safety in maritime transport by transferring the well proven practices and methodologies from air transport to maritime transport in an effective, collaborative and innovative manner.
Project clearly concluded that best practices in one transport mode can be utilised in another transport mode through establishing a transfer framework as solutions in one transport mode needs to be tailored to other transport mode. SEAHORSE was the first project under technology transfer and due to the proven success of the project, it is expected that more multi transport research project will be developed and more interactions between different transport modes will be encouraged and devised leading to safety enhancements in all transport modes in a cost and time effective manner.
EC DG research and Innovation –Transport Directorate is already working on establishing a Transport Safety Research Cluster covering all transport modes and encouraged by the SEAHORSE project’s results more multi-transport research calls can be expected in future work programmes.
The SEAHORSE project’s primary impact is an improved overall human factors and safety in maritime transport achieved through the introduction of innovative “SEAHORSE Procedure Improvement Methodology” and “SEAHORSE Multi-Level Resilience Model” supplemented by training and guidelines for the efficient implementation. Whole shipping industry is likely to benefit from the innovative outputs of the project as it will support the shipping companies and the maritime training institutions to understand the importance of Human and organisational factors to enhance the maritime safety and integrate them to their ship operations.
SEAHORSE created significant awareness within maritime community that, safety can be enhanced beyond compliance of rules and regulation through human and organisational factors through THE NEW SAFETY APPROACH.
While SEAHORSE developed novel, practical methodologies and tools to enhance maritime safety through transfer of best practices in human/organisational factors, in order to create industry wide take up and impact, it is necessary to have to turn these methodologies and tools to industry standards.
Through creating the training and guidance, the SEAHORSE project expects not only to impact the awareness and knowledge of current seafarers but also influence the future ones through integration in the education curriculum of the maritime schools. This will lead to skilled workforce creating more attractive environment for potential future cadets.
2- Exploitable Results
SEAHORSE consortium produced a number of outputs that can be utilised by the shipping companies, Maritime Education and training centres. As the aim of the project is to enhance the maritime safety by taking into account human and organisational factors in a structured manner, the tools are available to Shipping companies and educational establishments through free licencing. This is expected to influence the industry to take into account human element and organisational factors towards enhancing maritime safety. The consortium partners are available to assist the industry during the utilisation of SEAHORSE tools and methodologies. Ultimately, these tools and methodologies developed by the SEAHORSE consortium will be utilised to support IMO’s activities on Human element.
The SEAHORSE exploitable outputs are briefly introduced below. All the tools are available through WEB-based SEAHORSE VIRTUAL PLATFORM via free licencing.
Comparative State Of The Art & Gap Analysis
Comparative state of the art study has been conducted and the SEAHORSE Project identified 46 gaps between Aviation and Maritime under different headings including stakeholders; Functional Demands; Training; Regulations; Current Challenges; Human and Organisational Factors; Operational Demands; Socio-Economic Issues. Such a gap analysis has been reported first time between two transport modes.
Resilience Solutions and Transfer Methodology
Resilience solutions available in airlines are for the first time compiled into a database with an aim of transferring the solutions from air transport to maritime sector with the involvement of practising maritime experts as well as safety managers. The SEAHORSE transfer methodology was employed to shortlist and ranks the resilience resources based on a subjective evaluation made by maritime professionals and experts. The evaluation criteria were defined in order to assess clearly the potential applicability of the proposed solutions to the maritime domain
Procedure Improvement Methodology and Tool
SEAHORSE developed, first time in the maritime sector, a methodology and automated tool which provides a robust method for the collection and assessment of workarounds practised on-board vessels within the maritime industry. The collected workaround data are utilised as an opportunity to improve existing standard operating procedures for a given ship or a company.
Resilience and Impact Assessment Tool (Software):
Seahorse developed, first time in maritime sector, a multi-level resilience and impact assessment framework and tool to allow the safety managers to assess the company in terms of resilience compliance and identify the gaps/weaknesses/noncompliance that may lead to errors and accidents. Furthermore, the SEAHORSE Impact Assessment Tool is intended for workers who have been directly impacted by the implementation of one of the SEAHORSE solutions. It gathers information in a structured way about what workers think has changed as a result of the implementation of the SEAHORSE solutions. It also gathers behavioural data on changes to the way things are performed following the implementation of the SEAHORSE solution. Additionally, for safety and managerial staff, it provides a method for collating existing safety metrics and supplementing them with new measures to assess organisational safety impacts.
Crew Quality Audit Tool (Software): SEAHORSE developed a tool which allows companies, by utilising the databases, to audit the crew quality at individual as well as at team levels and identifies the strengths weaknesses of the crew and identifies the training needs or suitability for type of ships for operation.
Best Practise Guidelines
Best practice guidelines were developed for shipping companies to enhance their safety maturity levels based on the aviation industry’s successful solutions. Each best practices document provides step by step instructions for the shipping companies to implement the solution with their organizations. These solutions address safety culture, just culture, sop development, human factors training, arrival and departure checklist, mandatory safety reporting, safety bulletin.
Training modules of the solutions
These training modules were developed by utilizing the best practices knowledge available in the aviation industry. All of the modules were develop by maritime and aviation experts. These training modules are:
1. SOP Development Training Module
2. Just Culture Training Module
3. Checklists Training Module
4. Human Factors Training Module
5. Mandatory Safety Reports Training Module
6. Safety Bulletins Training Module
Safety Culture Assessment and Improvement Framework
A safety culture assessment and improvement framework developed to identify the existing problems in shipping companies and further enhance safety. It has been utilized in three different shipping companies to create an industry wide benchmark and currently more shipping companies are liaising with SEAHORSE project to utilise the framework.
SEAHORSE Virtual Platform
A WEB-BASED tool hosts all the tools developed within the project and allows companies to developed their databases and utilise the tools towards safety enhancement of a company. Free licencing can be provided to Shipping companies and Maritime Education/training institutions
( http://seahorse.smt.strath.ac.uk/bl.html)
3 Dissemination Activities
SEAHORSE consortium has adapted a coherent approach to make sure the resources among the consortium are applied efficiently to disseminate as widely as possible. Depending on the status of the project and the content of the results, the SEAHORSE performed a number of dissemination activities as listed below.
• Setting up project website: http://seahorseproject.eu/
• Setting up accounts in social media
o Linkedin: https://www.linkedin.com/company/seahorse-project
o Facebook: https://www.facebook.com/seahorseproject
o Twitter: https://twitter.com/SeahorseFP7
• Producing project brochure and banner
• Issuing newsletters
• Hosting public workshops and regional events
• Giving presentations in targeted scientific or commercial conferences
• Providing presentation to shipping companies or associations.
• Publications
• Press release and others
3.1 Workshops and conference organised by Seahorse Project
Seahorse project organised 7 different planned public events in 6 different locations and 262 people took part in these events.
3.1.1 Main Annual Workshops: Workshop 1, Workshop 2, and Workshop 3
SEAHORSE organised three flag ship SEAHORSE public workshops. These workshops are mainly set up to disseminate the project concept and progress. Meanwhile, these workshops also serve as a good platform to engage the local maritime stakeholders in supporting the validation of the solutions and tools developed throughout the project duration. The presentations of all the annual workshops are publicly available via the project website (http://seahorseproject.eu).
• SEAHORSE Workshop 1: 12 November 2014 (Month 13), Lloyd’s Register’s headquarters, London, United Kingdom. It was hosted by Lloyd’s Register and AP&A.
On the 12th November 2014, the first SEAHORSE workshop was held at Lloyd’s Register’s London office. London was chosen as the first venue due to its leadership in maritime business so that we can raise SEAHORSE’s profile in the industry at the beginning of the project.
It is a successful event, attracting 49 carefully selected participants. 77% of the participants came from the industry. We found that ship operators and owners (19%) are the industrial groups that tend to be most interested in this subject area. We have also secured a first keynote speaker from Cyprus Airways giving a presentation prepared with another advisory board member from Airbus. Other keynote speakers include visitors from Rolls-Royce and EasyJet. Through their in-depth knowledge in the aviation human factors, some health discussions have been raised throughout that day.
• SEAHORSE Workshop 2: 13 October 2015 (Month 24), Rotterdam, Netherlands. It was hosted by KAHN and TNO.
The Rotterdam workshop was successfully held in the Shipping and Transport College on the Tuesday 13th October 2015, in total more than 60 people participated. This workshop features a key note speaker – Markku Mylly, the Director of European Maritime Safety Agency (EMSA), and Arthur Dijkstra, the Captain of Boeing 777 of the KLM. The multi-level resilience assessment tool was further developed and validated in an interactive SEAHORSE workshop in Rotterdam with interested stakeholders. More than 40 maritime participants from industry provided suggestions of currently-used resilient solutions developed for particular challenges confronted in the maritime industry. Such challenges could be language problems or reduced crew sizes. This second SEAHORSE workshop successfully promoted brainstorming for new solutions, which will be implemented in a Virtual Platform in the course of the project. Such tools, together with the resilience questionnaire, will be made available to the maritime industry as well as any other interested parties. It also attracted the attention of fishing industry.
• SEAHORSE Workshop 3: 31 May 2016 (Month 31), Danaos’ headquarters, Piraeus/Athens, Greece. It was hosted by DANAOS and AP&A.
During the Piraeus workshop held on the 31th May 2016, it was highlighted that the work on safety improvement usually focuses on technical issues. However according the data collected from the Marine Accident Investigation Branch (MAIB), more than 80% of incidents are due to human factors. Together with the seafarers shortage, challenges lied ahead of the marine industry is clear.
Adopting the principles of resilience engineering in marine operation is one of the approach the project SEAHORSE is proposing. As highlighted during the workshop, learning, responding, monitoring and anticipating are the four key capabilities should monitor. The ownership of improving the weakness of ship operation should not be solely relied on the crew on board but also shared with marine professionals working onshore.
One of the SEAHORSE’s key advisory board members from aviation reminded the audience an important lesson learned from the aviation industry. It is common to see that near misses and accident rates going up at the beginning when similar safety solutions are implemented. It may appear as if the safety level is going down at the first glance, but this is actually a healthy exercise because an organisation can have a better visibility in where the problems are. Appropriate actions can then be introduced accordingly. Companies should be expected to observe such trends at first and should not be turned away from the continuation of the implementation process.
3.1.2 Local Workshops
In addition to 3 Annual Workshops which aim to disseminate the seahorse results internationally, three additional local workshops were organised in Glasgow, UK, Istanbul, Turkey and Madrid, Spain to disseminate the results to local maritime industries
Local Workshop 1: Glasgow, UK, 3 March 2015
The local workshop was attended by 40 people including people from different shipping companies including TEEKAY Shipping, Anglo-Eastern Ship Management, V-Ships, Calmac, Gaslog, Hellenic tankers; Aviation companies such as Airbus and Logan air, training companies and maritime colleges as well as representative of EU project CyCLades. A number of presentations made by Airbus, Loganair, CyCLades project as well as members of Seahorse consortium to inform wider audience about the objectives and achievements of the project. In the afternoon, all the attendees took part in a structured-interactive workshop to gather expert opinions of the safety managers about risk of workarounds to assess the methodology develop within SEAHORSE about managing the workarounds
Local Workshop 2: Istanbul, Turkey, 8 March 2016.
Istanbul local workshop, which was organised by Istanbul technical University, was attended by around 100 People from Shipping companies, maritime authorities, maritime schools, Cadets. Presentations were made to introduce seahorse aim, objectives and methodologies, Seahorse Tools and Virtual platform as well as industrial implementations of seahorse methodologies including implementation of pre-departure and arrival checklist approach to Calmac ferries. Workaround tools was also demonstrated in an interactive way involving industry representatives. Local shop was very successful as the Local maritime industry was made aware of the project outcomes and how they can utilise these tools.
Local Workshop 3: Madrid, Spain, 19 October 2016
Third local workshop was organised by ESM in Madrid and attended by 35 people from Aviation Authorities, Shipping Companies, Spanish Maritime authorities including search and rescue department as well as Road Transport Sector.
Antonio Castellano from the ESM opened the day. Prof. Osman Turan from the University of Strathclyde outlined the pressing issues regarding the human and organisational factors in the maritime operations. The Spanish Safety Investigation Authority (CIAIAC – Ministry of Transport of Spain) then delivered a thought-provoking presentation offering an example of difficulties in an investigation of a helicopter accident. Dr. Paul Liston from the Trinity College Dublin (TCD) elaborated the technology transfer methodology developed by the project SEAHORSE. 46 gaps have been identified. 3 scenarios and 73 solutions were shortlisted. F.A. (Dolf) van der Beek led the floor to the introduction of the resilience engineering paradigm. The procedure improvement tool developed by the project SEAHORSE was presented by Dr. Rafet Emek Kurt from the University of Strathclyde. The individual crew quality audit tool was jointly presented by Ozcan Arslan from the Istanbul Technical University and Volkan Arslan from the University of Strathclyde. The experience of testing some solutions in the industrial environment was presented by Ben Wood from CalMac, Fotis Oikonomou from Danaos Shipping. Other industrial issues were discussed during a panel session chaired by the University of Strathclyde. It is recognised that it will be important to continue seeking better solutions in improving ship safety. It is especially beneficial to have a new framework in addressing the current issues regarding the safety management system
3.1.3 International Seahorse Conference, 21-23 September 2016, Glasgow UK.
27 papers have been accepted and published in this International Conference held in Glasgow, between 21-23 September 2016. The conference, which was attended by 60 delegates, began with a welcome speech delivered by Prof Osman Turan, the co-ordinator of project SEAHORSE from the University of Strathclyde. There were 4 keynote speakers from the industry, including Harry Nelson from Airbus, Evangelos A. Tsoumpos from A.E.Nomikos Shipping, Martin Hernquist from the Swedish Club, and Captain Sanjay Patil from MISC Berhad. They shared their valuable experience from aviation to maritime industry and raised questions such as whether accidents can be prevented and human and organisational factors for the maritime safety. Practical issues are in heavy discussion among seafarers, safety managers and safety experts from other industries.
Prof Jan Maarten of TNO then introduced Resilience Assessment Tool (RAT. Prof Maarten also stated that RAT is the first tool available in time to check resilience of an organization. Furthermore, it was emphasised by different speakers that safety and just culture should be promoted into maritime along the conference programme. Blame culture affects the confidence of employees gravely and in return real feedback from employees cannot be retrieved. Fair approaches and fair treatments of behaviour are recommended.
During a small discussion session, the question ‘Can different transport modes work together in achieving common safety goals?’ was raised to the floor. Barriers to prevent action in achieving higher safety level have been discussed:
• Money
• Attitude
• Government
Impact assessment methodology was presented by Dr. Paul Liston of Trinity College Dublin and Vanessa Arrigoni of Deep Blue. They presented a transfer methodology which may be useful for others who is seeking to conduct similar transfer between different industries or practices.
The Seahorse Virtual Platform, resilience assessment software, was then introduced to the conference delegates and helped to ascertain the resilience of an organisation.
Conference proceeding are freely available to public through the website.
List of Websites:
http://www.seahorseproject.eu/
for further communication: info@seahorseproject.eu
contact details of the co-ordinator: o.turan@strath.ac.uk