Final Report Summary - CRISP (Evaluation and certification schemes for security products –Capability Project)
Executive Summary:
CRISP (Evaluation and Certification Schemes for Security Products) is a three year project that aimed to facilitate a harmonised playing field for the European security industry by developing a robust and innovative evaluation and certification methodology for security systems.
The overall objective of CRISP is to enhance existing security evaluation and certification schemes by offering an innovative evaluation methodology that integrates the security, trust, efficiency and freedom infringement assessment dimensions. The methodology is based on a new taxonomy, developed by the CRISP consortium, which encompasses a variety of security products and services across application areas. The project also took into account the varying roles of the diverse security stakeholder community including manufacturers, regulatory and certification bodies, data protection authorities and end users and engaged with each group to gather insights in order to assist with avoiding acceptance problems that are known to challenge current schemes. This took place through dissemination, promotion, and a variety of stakeholder-focused events such as workshops and a final conference.
In the CRISP project an innovative approach to evaluation and certification of security systems was developed. The most innovative part of the methodology is the assessment of systems from the perspective of four different, though interrelated dimensions. These dimensions are referred to as the STEFi dimensions (Security, Trust, Efficiency and Freedom infringement) and the methodology integrates these in its evaluation phase. This is an innovative approach as certification has, to date, primarily focused on the evaluation of technical requirements for security systems (the security dimension) or singled out other relevant dimensions (e.g. privacy or data protection in the freedom infringement dimension). The methodology, however, is not (over)simplifying the complexity of assessing security systems but acknowledges and addresses this complexity by identifying potential conflicts between the various assessment dimensions and related criteria and by providing an approach to resolve these conflicts in specific situations. The methodology does not single out technical, legal, social or economic aspects, but integrates these in a multidimensional and multi-stakeholder assessment.
The exploitation results consist of the following building blocks for the CRISP certification scheme: CRISP methodology, CEN Workshop Agreement, CRISP certification manual and roadmap, CRISP exploitation plan, and briefing papers on the proposed certification scheme and roadmap for the key stakeholders (regulators, manufacturers, certification bodies and end users).
As the certification scheme will need to be further developed and finalised and this is out of the scope of the CRISP project, documentation is included to guide the future CRISP organisation/scheme owner in taking the next steps.
Project Context and Objectives:
PROJECT CONTEXT
The market for security products and services is one of the most rapidly growing markets in the world. However, the European security market is highly fragmented, and no common framework applies to security products and systems. Further challenges specifically related to certification of security products and systems is the lack of common certification systems for security products as well as a lack of mechanism for mutual recognition across countries of products certified at a national level.
CRISP (Evaluation and Certification Schemes for Security Products) is a three year project aiming to facilitate a harmonised playing field for the European security industry by developing a robust and innovative evaluation and certification methodology for security systems.
CRISP addresses some specific challenges. One of the challenges is that there is no effective policy to regulate liability by setting quality standards and defining which services and products fall under certain criteria. A common approach to categorise both security products and services is needed. Next, an appropriate organisational structure for standardisation and certification of security products and services is missing resulting in a plethora of standards and certification schemes, with a lack of harmonisation of schemes on European (and international) level. This is not only a constraint for the success of the EU Internal Market but also constrains European companies from becoming security market leaders and effective global competitors. Furthermore, there is a lack of transparency in respect of security products and their implementation, and a lack of effective participation from stakeholders in setting security standards and certification. Finally, the early involvement of stakeholders is crucial, particularly for the acceptance of security products and services that have the potential to have various negative societal effects. Market success relies strongly on acceptance by the public and policy makers. For this, fostering and promoting stakeholder involvement at all levels of the standards and certification process is needed.
Accordingly, the CRISP project addresses the challenges by:
• Create a taxonomy that will allow the clustering of security products and services alongside adequate parameters that will lead into an effective and efficient classification of the two.
• Propose a methodology for a common certification scheme with a common European label, tht will not only benefit the EU Internal Market, but also help the EU consolidate and carry on its leadership in the global security market.
• Involve stakeholders and integrate their perspectives at all levels, acknowledging the societal impact of security products and systems.
• A strong stakeholder engagement strategy (which includes identifying stakeholders and engaging with them throughout the duration of the project through various means and platforms) to build confidence and enhance acceptance of the proposed approach.
OBJECTIVES
The overall objective of CRISP is to enhance existing security evaluation and certification schemes by offering an innovative evaluation methodology that integrates the security, trust, efficiency and freedom infringement assessment dimensions. The methodology is based on a new taxonomy, developed by the CRISP consortium, which encompasses a variety of security products and services across application areas. The project also took into account the varying roles of the diverse security stakeholder community including manufacturers, regulatory and certification bodies, data protection authorities and end users and engaged with each group to gather insights in order to assist with avoiding acceptance problems that are known to challenge current schemes. This took place through dissemination, promotion, and a variety of stakeholder-focused events such as workshops and a final conference.
The following mission statement and scope have been specified:
Mission statement of CRISP project and scheme
The CRISP project mission is to develop an innovative evaluation and certification methodology for the CRISP certification scheme for security systems. The CRISP scheme will:
● Contribute to measures that increase citizen trust and confidence in security technologies through the evaluation and certification of social and legal impacts of security systems;
● Facilitate a more harmonised playing field for the European security industry by providing pan-European certification for security systems. The scheme is intended to be accepted across Europe, which will enhance competitiveness by reducing commercialisation costs for the industry;
● Support the goal to provide protection in an efficient manner.
The methodology integrates the Security, Trust, Efficiency and Freedom infringement dimensions, the so-called STEFi dimensions, in its evaluation stage. This is a highly innovative approach, as certification has, to date, primarily focused on the assessment of technical requirements for security systems.
This novel approach to the inclusion of social and legal aspects in evaluation and certification methodology will first be piloted for video surveillance systems, to test and refine the approach. After the pilot phase, it is foreseen that the CRISP scheme can be extended to include other types of security systems.
The CRISP methodology will serve as the foundation of the CRISP certification scheme, which will, upon the completion of the CRISP project, be further developed by an interested organisation. The CRISP scheme will not redefine the technical requirements that are already in place (e.g. in European standards). Instead, the STEFi dimensions will offer the inclusion of social assessment criteria in the certification of security systems; the scheme will contribute to the protection of fundamental rights and promote compliance with relevant EU laws, with a particular focus on the General Data Protection Regulation (GDPR) 679/2016.
Certification by the CRISP scheme will assure that a security system has been assessed on the basis of the STEFi dimensions and has been found to comply with applicable assessment criteria for security, efficiency, trust and freedom infringement. It can be sought initially by both those who procure and those who run video surveillance security systems on their premises.
Scope of the scheme
The CRISP scheme will provide criteria and requirements for the evaluation and certification of installed video-surveillance security systems based on the CRISP STEFi dimensions (Security, Trust, Efficiency and Freedom infringement). For systems in development, evaluation according to the CRISP scheme can be applied. Certification of these systems will only be possible after installation.
Target group of the scheme
End users of the relevant systems, such as local authorities, emergency organisations, transport operators, law enforcement authorities, retail organisations, health organisations and educational organisations.
CRISP PROJECT WORK
The work of CRISP consisted of:
- Creating a taxonomy of security products and systems, concepts of operations, application areas, performance and set out criteria for comparing security products and systems;
- Providing a historical perspective on security standards and certification in Europe and analysing the state of the art in security standards, certification and accreditation at the Member State, regional and international level;
- Identifying and determining the role of different stakeholders, gauging their views on the challenges affecting security certification and determine requirements for a harmonised EU-wide approach;
- Examining the core assessment dimensions such as security, trust, efficiency, freedom infringement (STEFi) of security product certification and presenting the requirements for enhancing existing evaluation and certification schemes;
- Developing an innovative methodology, based on the STEFi dimensions;
- Developing a CEN Workshop Agreement (CWA) to record the evaluation approach, including evaluation questions and requirements;
- Presenting a certification manual, roadmap and implementation plan, and an exploitation plan to be taken up by the future scheme owner;
- Disseminate briefing papers to key stakeholders, enhancing confidence and acceptance of the CRISP approach to certification.
Project Results:
1 CATEGORISATION OF SECURITY EQUIPMENT, SYSTEMS AND SERVICES
1.1 Glossary of security products and systems
As a foundation for the CRISP project, the glossary of security products, systems and services (PSS) has the unique characteristic of focusing on the functionality of security PSS instead of the technique itself. Security PSS can be defined in many ways and defining a product, system or service in a technological way often involves a discretionary component, as each technological system is comprised of several interconnected elements and each of these elements can itself be analysed as a system of its own. A security PSS can encompass everything from a simple torchlight to a system connecting video surveillance, biometric scanners and infrared sensors. Instead with the functionality of a security PSS, we describe the intended result of the PSS when in operation.
In our CRISP glossary, we derived and defined the following security functions from the literature, policy papers and research reports: ‘locate, identify, verify, control, track, assess, authorise, communicate, create situational awareness, information collection, storage and management to produce intelligence, detain, prevent/protect’. Due to the different levels of operability of these security functions, they were classified regarding their interdependency and the interrelationships of each of the security functions. Primitive functions are the basic functions of security PSS and are essential for the further performance of other functions. Connective functions make use of the primitive functions, mainly through connecting available information with specific criteria which then again can be necessary for the performance of further security related activities and functions. Performative functions finally are carried out (security) actions, with a clearly defined and targeted result, making use of one or both previous functions.
Finally, the outcome of security functions can differ depending on where the PSS is deployed. Thus in order to reflect on the different security contexts, four areas of security were derived from the literature, in which similar classifications have been done already, however often with rather redundant categories. For the CRISP glossary, the four areas of security are:
(1) security of citizens which covers all possible threat aiming at European citizens in public and semi-public spaces as well as in private spaces;
(2) critical infrastructures as general infrastructures which are of high importance of the functioning of a vital society and thus a protection against threats aiming at the disruption or destruction of the like plays an important role in the European policy making and the security industry
(3) border security which includes the means for providing security of land, air and sea, but also of borders in embassies to prevent the illegitimate crossing of people and the detection of illegal products, goods and substances within custom services
(4) Crisis Management includes mainly the restoration of security in the aftermath of a crisis, which may result from a natural disaster, but also from deliberate attacks.
1.2 Taxonomy of security products, systems and services
Based on the glossary of the first deliverable, a categorisation of security PSS was established. In form of taxonomies for security products, security systems and security services, this categorisation examines the wide diversity in the domain of security PSS. The categorisation of the taxonomy is done on four levels and aims at depicting the wide array of deployment possibilities and the related function of identified security PSS.
On the first level are the security application areas. Like the glossary, the taxonomy categorises security PSS depending on where the intended application is based. The four categories are the ones presented already in the glossary, being the security of the citizens, critical infrastructures, border security/management and crisis management/emergency preparedness centres. The second level of categorisation of the taxonomy is the security demands. The demands are mainly based on reviews of commercial security company websites, European policy literature and related security research projects and further specify the areas where the security PSS can operate, while already refining the intended security function. The security demands represent the key requirements which are necessary in the respective area of security and which should be implemented by the security function. This includes access control, asset/freight/cargo security, cyber security, employer/visitor security, loss prevention/shrinkage, perimeter/building/area security, point of transaction, situation awareness. On the third level of the taxonomy are the security functions of the glossary and specifying the intended result of the security PSS.
Lastly the security product, system or service is listed, depending on their application within that classification. The security PSS is of course not exclusive classified within one categorisation, but can, like the security function, be part of different application areas and security demands. As such, this function-based taxonomy is a very flexible tool for the categorisation of security PSS.
2 REVIEW OF STANDARDS, CERTIFICATION AND ACCREDITATION FOR SECURITY PRODUCTS
2.1 Security standards and certification in Europe – A historical/evolutionary perspective
The consortium provided an analysis of the state of the art of security standards and certification in Europe and an analysis of the economic benefits of conformity assessments. Specific insight into the framework conditions of security certification in Europe were given and security-related documents which determine the current security certification landscape or can build the foundation for future improvements in the marketplace were explained. Furthermore, specific security-related standards and technical committees in Europe were analyzed and current and potential interrelations between standards and certification were shown. Possible needs for new security-related standards were also identified. Moreover, field research revealed that fostering more standards-based certification requires stimulation by appropriate European directives.
With regard to the interrelation of standardization and certification, information was gained from interviewees and emails from CEN/TCs and analyses of the work of numerous certification bodies. Many security-related TCs were quite new and new standards were under development. Therefore, it was recommended to seek collaborations with certification bodies in early stages. In addition, there were national certification bodies which participate in national mirror committees of CEN/CENELEC/TCs. They also provide a good example for the establishment of interrelations between both fields. Talks with experts from national certification bodies show that there are institutions which are aware of the advantages of participating in standardization, but that they are still too few.
Furthermore, security areas in which the use of open standards is limited were analyzed. In particular, the fields of digital signatures, airport screening equipment and air cargo are relevant. Usually several different governmental authorities and security authorities are responsible for these topics in a Member State, making the European landscape very complex in this regard. Although databases exist that show all national certification bodies which are accredited by a national member of the European co-operation for Accreditation (EA), databases of non-EA members are not available. Therefore, an extension of current databases or creating an additional database is recommended. An additional area in which standards are not used for certification is related to innovative solutions for which standards do not exist yet.
The most important result of the review activities presented in this section is a detailed overview of the state of harmonization and mutual recognition in Europe, which describes suggested concepts of “one stop testing”. This approach is highly recommended for new product classes, new requirements and related standards. With regard to existing certification services, alternative solutions may also offer advantages.
Fields in which appropriate harmonized certification solutions are missing are, for example areas of complex security systems. In addition, there are security issues which are shaped by different national preferences in Member States. EN 50131 which includes specific national amendments provides an example for that. It shows that there are areas which should not be covered by general harmonized solutions but by complementary certification. The number of these areas is to be kept as small as possible.
The analyses also offered interesting examples for European collaborations in security certification. They include, for example, the agreement of the Senior Officials Group Information System Security (SOG-IS) and the European Fire and Security Group (EFSG). Like the international CC Recognition Arrangement, SOG-IS provides mutual recognition for certificates on information systems security. Nevertheless, many needs for further action remain.
Quality is a key issue in the certification context. EFSG builds on European standards, and its members compare their test results regularly. The group is also active in standardization, although its attractive offer to obtain multiple quality marks with minimal duplication and cost is not usable in all European countries yet. In general, mutual recognition is often practiced by large, industrialized Member States with a large security market and industry. Efforts to date have been unsuccessful in removing barriers to greater harmonization. A major obstacle for the expansion of EFSG is, among others, the perceived quality of other national certificates in the relevant fields, which again highlights the quality issue.
Instruments used by the EFSG group include, for example, round robin exercises. To facilitate harmonization in other security fields, it is recommended to use this instrument alike.
As a starting point of various activities in different CRISP work packages, the concept of CertAlarm, with whom CRISP collaborated, was also analysed and led to helpful interim results even after these first review activities.
The analyses finished with specific suggestions and recommendations for the future.
2.2 Security standards, certification and accreditation – best practices and lessons learnt
The second series of review analyses led to the identification of various good practice approaches and recommendations for CRISP’s further work. It identified attractive technical solutions, interesting examples for collaborations among certification bodies as well as approaches to consider societal aspects in security-related certification.
In addition to the pioneers in harmonisation and mutual recognition in different security areas previously described in CRISP’s deliverable 2.1 more good certification practices on national and multinational levels could be identified.
There are a few security certification schemes in specific technical areas in the Member States, which avoid the narrow regional focus on national standards. ECB’s consequent orientation on European and international standards and its collaboration with various laboratories in different Member States is an interesting example in this regard.
A good practice example regarding collaboration is the ARGE DIN 14675, established by the German TÜV Rheinland. ARGE DIN 14675 is an association of currently ten accredited certification bodies and the owner of an accredited certification scheme for the planning, operating and maintenance of fire alarm systems and voice alarm systems based on the DIN standard on ‘Fire detection and fire alarm systems - Design and operation’. Good practice does not only characterise the common, standard based certification scheme but also the collaboration between certification bodies and their stakeholders and the procedures to update the relevant certification scheme.
An important example to learn from on an international level is provided by the U.S. SAFETY Act, which applies to a wide range of security solutions. Another good practice example from the U.S. is the National Strategy for CBRNE Standards to overcome the coordination problem caused by the high number of agencies responsible for CBRNE standards.
Societal aspects of security systems are another important aspect to consider, in particular regarding surveillance solutions, because "(u)nseen, uncontrolled or excessive surveillance activities (...) pose risks that go much further than just affecting privacy. They can foster a climate of suspicion and undermine trust”
Examples for partial response to this statement already exist in the certification field. Besides EuroPrise, a certificate for IT products, the British CCTV Code of Practice for CCTV systems example considers several specific aspects of freedom infringements. The VdS 2365 on CCTV systems also integrated a few issues of that kind in its certification process but does not consider the full variety of CCTV-related freedom infringement aspects.
Based on the further analyses of 14 country studies, multinational standards and certification schemes, nine recommendations were derived for the further work on the CRISP scheme, its implementation and usage. They consider the following issues:
- Fast certification processes: The security market is growing and demanding new secure, efficient and trustworthy PSS, which also take into consideration the freedom and rights of citizens. In this regard, faster certification processes and a shorter developmental period for standards are needed. There is also a need for certification organisations and institutions for better solutions in order to continuously improve and accelerate their proposed services on certification for security PSS.
- Embeddedness in the socio-technical context: To avoid redundancy, the design of the CRISP scheme should take into account the structure and content of well-established national and international standards, where additional requirements can be developed on top of existing ones. The standards currently under development (for example at the international level) should also be carefully examined to the extent that the development is of a public nature, in order to search for synergies. Furthermore, its needs to be taken into account that the extent to which the CRISP scheme will seek compliance with national legal requirements, will most likely affect the adoption of the scheme.
- Performance-based requirements: A trend towards performance-based requirements in certification was identified. CRISP should therefore consider the function of security PSS within its further research activities, as well as the need for performance standards. In this regard, an active exchange by concrete use cases between practical experts and researchers is highly recommended. This will work to assure that project findings will meet essential requirements and serve to keeping standards and the development of certification processes up-to-date.
- Interoperable solutions: The importance of interoperable solutions became apparent to enable security integrators to develop systems, whose components from different vendors can communicate with each other. Within CRISP’s further research activity, this had to be taken into account.
- Implementation of the scheme: The diverse landscape of existing standards related to security PSS, coupled with a network of Conformity Assessment Bodies (CABs) that may or may not be accredited, requires careful consideration of how the certification scheme developed by CRISP will fit in into these existing relationships between CABs, accreditation bodies and end-users. Which CABs may be appropriate to certify under the developed scheme and whether accreditation of CABs will be relied upon are only some of the questions that the CRISP project will continue to seek answers to.
- Accreditation practice: An important issue to reach transparency and fair competitiveness among security certification entities is the harmonisation of guidelines for their accrediting auditors. Cross trainings of accreditation auditors, mutual assistance between them and the reduction of differences among accreditation programs are needed in order to avoid substantial different test results in the same certification scheme.
- Regular follow-up controls of certified solutions: As CRISP’s analyses showed, regular follow-up controls of the certified security PSS are significant. The certificate should represent the actual compliance or quality of the product, system or service. With regular compliance control, the certified entities have an additional incentive to keep improving or at least maintain an adequate level of compliance. This also promotes trust in the certificate and the relevant security solutions.
- Solutions for SMEs: The costs of certification must not be a competitive barrier for small and medium enterprises. First ideas include the adjustment of costs to the size and complexity of the entity as well as measures to ensure equal treatment of customers independent of their geographic location and the residence of the auditors.
- Political support: The adoption of a new pan-European approach to certification for security PSS will necessarily require political support, as it follows from CRISP’s research that standards are heavily intertwined with national legislations. Further work on the development of strategies for the adoption of the CRISP scheme must therefore also focus on the political dimension of the European security certification landscape to receive appropriate support.
3 SECURITY CERTIFICATION STAKEHOLDER ANALYSIS
In order to conduct the stakeholder analysis, a review of the relevant policy, grey and academic literature, for the purpose of identifying key stakeholder groups in security certification was done. Second, three case studies of specific technology areas were examined to understand the general and field-specific issues of security certification and key stakeholder views on and needs from certification. The case studies (which included expert interviews) gave an insight into three technologies areas and the specific needs of stakeholders within them when it comes to the development of standards, certification schemes and how they may perceive the notion of a European certification scheme for security, products, systems and services. The three case study technologies were as follows:
• Video Surveillance Cameras (CCTV)
• Remotely Piloted Aerial Systems (RPAS)
• Alarm Systems (Fire and Intrusion Alarms)
Third, the findings on stakeholder needs and views were verified and discussed at a stakeholder workshop in December 2014, and then used to develop two web surveys to ascertain the prevalence of views expressed among a broader set of stakeholders in Europe.
The review of literature identified the following key groups of stakeholders:
• Security product manufacturers, suppliers and systems integrators (henceforth referred to as security industry)
• Conformity assessment and certification bodies
• Standardisation organisations
• Accreditation bodies
• Data protection authorities and other regulators
• End users
Secondary stakeholders (affected by and indirectly involved in security certification):
• Watchdogs and civil society organisations
• Individuals
• Academics
The bullet points below, give a brief summary of key stakeholder groups’ needs from security certification, as they were presented to us through the research:
• For the security industry, certification is important for accessing different national markets as well as providing requirements to which products and services are designed and developed. The key need of the security industry is for increased harmonisation of certification across borders for the purposes of avoiding re-certification which is costly and time-consuming and causes delays to market.
• Certification bodies express the need for a certification scheme, founded on robust standards and clear evaluation criteria. It is also of key importance that a certification scheme is well known, accepted and trusted, thus offering clear added value for industry. In the absence of the aforementioned qualities, certification bodies will have difficulty justifying and selling the scheme to security industry stakeholders.
• Accreditation and standardisation bodies, while sharing concerns regarding lack of transparency and complexity of the current certification landscape, view their processes for accreditation and developing standards as robust and fit for purpose for supporting a new certification scheme. Their needs regarding any new certification scheme is that its purpose, evaluation criteria are clear and transparent so that standards development or accreditation is consistent and straightforward.
• Out of the group of regulators relevant to certification, Data Protection Authorities (DPAs) were identified as very relevant in light of the social evaluation of the CRISP scheme. Currently, DPAs are usually not involved in standards development nor certification (with the exception of Germany) but due to their focus on privacy and data protection aspects of surveillance and other security technologies, they present as a stakeholder which will be necessary in further developing the certification scheme and ensuring acceptance and trust.
• End-users feel that the current certification system is complex and opaque and that there are too many schemes/seals and it is not clear to what the difference is between them. Increasing transparency regarding evaluation criteria for different schemes is thus imperative to better meet end-users needs regarding certification.
Regarding the CRISP certification scheme and its focus on evaluation of social dimensions, we found that overall stakeholders were positive towards this type of scheme. The research findings however revealed that stakeholders were uncertain about what such a scheme would look like and how it would be applied. There were some concerns that social dimensions would be too vague to serve as efficient evaluation criteria. This indicates that further to design and development of the scheme, the CRISP consortium must raise awareness about the prospective scheme amongst relevant stakeholder groups, to ensure acceptance, even implementation and take up across Europe. This, in addition to further stakeholder consultation and road mapping exercises, is the focus of the subsequent work packages of CRISP.
Working towards harmonisation of security certification in Europe is a complex task, given the myriad products, systems and services available, the pace of technological development and consequent social concerns and last, but not least, the differing national regulatory, legal, social and cultural aspects within the European countries. These draw our attention to how flexibility must be inherent in the CRISP scheme, to account for different technologies, means of operation, national cultures, as well as legal and regulatory frameworks in each member state. The case studies furthermore revealed that while some fields are well on their way towards a harmonised landscape (e.g. alarm systems), newer technologies, such as RPAS, are very much at the starting point of the work of setting up a system of certification across Europe.
National differences emerge as key barriers, which need to be fully understood and tackled in order to successfully work towards harmonisation, and consequently, the CRISP scheme should be flexible to the extent to include national requirements as supplementary to the overall minimum requirements stated in the scheme. The aim here would, however, be to keep national requirements to a minimum to avoid continuing fragmentation in certification.
Further to raising awareness and meeting the needs of key stakeholders, mechanisms for ensuring consistency and acceptance across Europe are key for the success of the CRISP scheme. As identified in the literature review and analysis of stakeholder roles, accreditation bodies and European cooperation for Accreditation (EA), emerge here as strong organisations to ensure a consistent accreditation and of the scheme across Europe. To ensure that the scheme complies with national regulation, collaboration between DPAs and accreditation bodies, and their respective Europe-wide bodies, could be a useful endeavour to ensure consistency and compliance, whilst avoiding fragmentation and uneven application.
This report ends with listing minimum requirements for a harmonised approach, which are derived from analysis of data derived from all research tasks. These recommendations will guide the remainder of the CRISP project as it embarks on designing and developing the scheme in accordance with stakeholder needs. The recommendations clearly outline what steps must be taken in order to develop, implement and encourage take up of the new scheme.
Following is the list of requirements for a harmonised approach:
(1) The certification system should be based on robust European or international standards. European and international standards are developed by a wide range of stakeholders and reflect the generally accepted perspective on the requirements to be set to a product, system or service. It is the role of the standardisation organisations to develop standards in such a way that they are robust and can be the basis for certification
(2) The certification schemes and the underlying requirements should be transparent and clear in what they evaluate and certify. The demand for transparency and clarity in certification was strong from all stakeholder groups. Lack of transparency for any stakeholder group will increase the likelihood that the scheme is not trusted, which will consequently lead to lack of take-up.
(3) The certification system must be accepted throughout Europe. Complete acceptance throughout Europe will contribute to the harmonization efforts of the EC and will benefit the security industry by avoiding re-certification for each national market. Working towards a pan-European acceptance will be a complex undertaking but there are some best-practice examples, such as the CertAlarm certification system for alarm systems, which is already widely accepted among European countries.
(4) Certification bodies should ensure that their evaluation and standards interpretation, is consistent throughout Europe. Consistency is of key importance as perceptions of a lack of or inconsistent test and evaluation methods in certification will negatively impact trust in the new scheme. This furthermore ensures that producers of products or services that have not met the requirements when examined by one certification body cannot get them certified by another.
(5) The certification system should be endorsed (or enforced) by regulators. The single strongest tool to ensure a European certification scheme is being accepted by stakeholders is for the European Commission to endorse it, as this will drive cross-border and stakeholder acceptance.
(6) The certification scheme should be operated under accreditation. To ensure quality of certification and trust in the certification system, it should be operated under accreditation. Whether or not a scheme is accredited is decided by the schemes owner.
(7) The certification system should provide one recognisable European seal. By using one recognisable seal, it is clear for all stakeholders that a product, system or service meets the requirements set in the standards. The certification scheme developed within CRISP will focus on requirements regarding four social dimensions: security, trust, efficiency and freedom infringement, but does not focus on technical requirements of security products. The CRISP seal should indicate that all four dimensions have been examined and adequately addressed in the evaluation.
(8) The CRISP scheme should take national specific requirements into consideration. National difference emerged as a key barrier to harmonization of certification throughout Europe. The CRISP scheme must take this into consideration to enhance uptake and acceptance in different countries in Europe. There is a possibility that these will be dealt with by writing a baseline scheme where national specific requirements only need re-certification.
(9) An appropriate implementation and awareness-raising process is necessary for the certification system to be successful. The quality of the way the certification scheme is implemented and presented is as important as the quality of the scheme itself. Stakeholders must have access to high quality and accurate information about the scheme, its benefits, criteria and evaluation process. In addition to contributing the design and development of the scheme itself, stakeholders also need to have a say in the way the system is implemented.
4 ANALYSIS OF STEFI DIMENSIONS: SECURITY, TRUST, EFFICIENCY, FREEDOM INFRINGEMENTS
4.1 Legal analysis of existing schemes
The analysis aimed to analyse the core issues of each of the STEFi dimensions and determine legal demands for security products, systems, and services. STEFi stands for Security, Trust, Efficiency and Freedom Infringement, and was an approach to evaluation of security systems developed in the FP7 project SIAM.
The study on the legal framework on evaluation and certification schemes identifies the legislation applicable and soft law setting the best practices for certification in the EU. The lack of European legislation on certification schemes is to a certain extent covered by the international and European standards and the guidance from standardisation bodies.
With regard to the legal study on security, the EU and the Member States share competence in the area of security. The security legislation in the EU is sector-specific and provides useful insights as to high-level issues for security products, systems, and services. Legal demands identified relate to physical controls and training of personnel, as well as the performance and the functioning of the security equipment. The multi-layered risks of physical and digital nature urge for accountability, security and risk assessments. Access to information systems and prevention from illegal interception and interference to the data and systems are also significant requirements incorporated to the EU legislation.
In terms of standardisation and certification, trust means that the security product, system, or service, among others, complies with the legislation and respects fundamental rights, is technically reliable, efficient and transparent, and responds in a predictable and acceptable manner. The role of evaluation and certification schemes in enhancing the needs of the citizens can be crucial when it involves stakeholders, is reviewed regularly, is operated by an independent body and includes requirements that address the main concerns of the scrutinised in the auditing or evaluation procedure.
Looking at efficiency, the case studies of this Work Package, reveal significant challenges as to how a security PSS should operate and perform in order to balance investments and achieve the objective of security. Energy efficiency and adaptability to new technologies with the minimum cost possible are elements relevant to efficiency for security PSS from a legal perspective.
Freedoms and fundamental rights of individuals are potentially at risk from the establishment and/or operation of security products, systems, and services, in different environments by entities such as public authorities, private legal persons, or individuals. The impacts of drones and CCTV systems in public spaces for crime prevention and detention to the right to privacy and data protection may range from extended surveillance, panoptic effect, profiling, to excessive collection of personal data, lack of notification to data subjects, lack of possibility to exercise data subject rights and others. Accordingly, biometric alarm systems pose risks to the rights to data protection and privacy, as well as others. Equal treatment and prohibition of discrimination, bodily integrity, presumption of innocence, due process and fair trial might be infringed by the security measures, either from the functionalities of the equipment itself or the use/abuse of the security product. The study identifies core requirements for security PSS to respect the rights and freedoms and be in line with the protective framework.
Further, the analysis examines to what extent the existing evaluation and certification schemes incorporate and address issues identified in the legal study of the Report. The analysis of existing schemes, showed that the type of entity operating the scheme and the type of the scheme play a crucial role into determining the quantitative and qualitative integration of the STEFi requirements. Public authorities tend to prioritise trust and freedom infringement requirements, while certification and standardisation bodies, as well as the industry focus on security and safety requirements.
The impact of the STEFi dimensions to the function and performance, user acceptability and legal compliance of the security PSS and the difficulty to identify a scheme with a comprehensive approach to the core aspects of all the four dimensions, highlighted the importance of the CRISP objective to develop an innovative evaluation methodology that integrates the security, trust, efficiency and freedom infringement assessment dimensions.
3.4.2 STEFi based SWOT analysis of existing schemes
The STEFi based SWOT (strengths-weaknesses-opportunities-threats) analysis of existing schemes aimed to: first to contribute to identification of core evaluation subjects for evaluation and certification schemes of security products, systems and services, based on the four dimensions of the STEFi model; second, to examine and analyse the strengths and weaknesses, opportunities and threats of existing schemes in order to further use, enhance and develop evaluation and certification schemes for the assessment and certification of products, systems and services used for physical security of people and infrastructures.
The SWOT model of analysis and the STEFi approach are employed for the analysis of the existing evaluation and certification schemes. In order to further identify high-level criteria based on STEFi, a showcase on a security problem is presented with selected potential security solutions such as CCTV, police patrols, community policing and others. The DESSI tool for supporting security investment decisions and its multi-dimensional criteria matrix embracing political, legal, ethical, social, economic and societal aspects is used for the security problem.
Further, a STEFi based analysis of CCTV, alarm systems and drones underlines and elaborates on the core issues of the three selected cases. Issues such as false alarm rates, performance and interoperability are discussed for security dimension; reliability, safety and transparency for the trust dimension; deployment and lifecycle costs for efficiency dimension and lack of awareness, big data, social sorting and discrimination for the freedom infringements dimension. The outcome of the case studies is suggestions for STEFi criteria.
Following the analysis of the three cases studies and the showcase from the DESSI project, the consortium identified ‘STEFi criteria for the evaluation of security products, systems and services’, which aimed to serve as a first basis for the analysis of existing schemes. In addition, the STEFi criteria, as high level evaluation subjects derived from the investigation phase of the project, aimed to facilitate the rest of the project. The list of STEFi criteria concentrates the main topics discussed in the deliverables into key topics (“evaluation subjects” according to the CRISPDoW) and provides an overview of issues raised in this analysis.
The analysis of selected existing schemes in the area of CCTV systems, drones, and alarm systems, identified shortcomings and best practices, along with opportunities for improvement for the purposes of the CRISP project. With regard to the shortcomings, most of the analysed schemes did not consider any freedoms and human rights aspects, nor the impact of the security measure on such rights. Efficiency is also an underrepresented aspect in the analysed schemes. In relation to the way the evaluation and certification process in general are organised, the research team identified issues of fragmented rules, that are spread in several documents, which in turn might lead to confusion, uncertainty, and mistrust to the scheme. In addition, mistrust might be generated by lack of transparency regarding the validity period of the certification and the limited availability of the scheme documentation. As the presentation of the SWOT analysis has shown, several best practice examples could be identified within each STEFi dimension. However, while singular criteria are highlighted by some schemes in one dimension, they are undervalued or are not considered at all in others. In terms of security, the criteria ‘Risk management’ was in more than 60 % of the analysed schemes fully met. The trust evaluation showed that the majority of the schemes might have reference to some STEFi trust criteria but they do not fully satisfy them. Examples are the criteria “transparency”, “observability” and “safety”. Efficiency aspects of security PSS are also not well covered. “Usability” and “energy efficiency” are therefore among the recommendations for improvement of the existing schemes. Last but not least, the freedom infringements evaluation had similar results with a few best practice exceptions. “Privacy” and “bodily integrity” are usually overlooked, whereas “personal data” attributes are only partly considered. For each criterion, and sometimes for each attribute, the analysis underlined schemes that can be considered as best practice according to STEFi, but as a general conclusion there is substantial gap in meeting the STEFi criteria.
The report concludes with recommendations based on the opportunities for improvement that were identified in the analysis of the existing schemes:
(1) Open and transparent rules, scope and processes
(2) Strong monitoring mechanisms to supervise the compliance of the PSS with the certification rules and its normative references.
(3) Accountability mechanisms
(4) Reliable normative references, such as international or European standards
(5) Governance which involves a standardisation body and/or a public authority
(6) Multinational participation in the development process of the scheme
(7) Differentiation of testing and evaluation levels for different security functions or required performance (granularity)
(8) Thorough rules on documentation to ensure accuracy and openness to the interested parties
(9) Publication of the revoked and expired certificates
5 DEVELOPMENT AND TESTING OF THE CRISP METHODOLOGY
5.1 Development of the CRISP methodology
The CRISP project followed a tailored strategy and involved stakeholder feedback at different stages in the project in order to foster greater acceptance and usability of the developed methodology. In the case of the validation and refinement of the CRISP methodology, efforts has been undertaken to ensure continuous stakeholder involvement in the development of the methodology, by means of holding interviews with experts from certification bodies, conducting a validation workshop to present and discuss the first draft of the methodology with a large group of stakeholders, and regularly consulting the advisory board of CRISP.
CRISP’s approach is composed of:
(1) Evaluation consisting of two main stages: configuration and STEFi assessment.
(2) Certification as third-party attestation related to products, processes, systems or persons and consisting of two stages: audit and attestation, which consider the results of the evaluation part.
In order to address significant gaps in the current certification landscape, the methodology claims to imply two important aspects which have been hardly or only scarcely considered in previous evaluation and certification activities: First, it includes social criteria for the evaluation and certification of security PSS. And second, it integrates the complexity of different assessment dimensions in one approach by enabling and encouraging an early participation of diverse stakeholders representing both the supply and demand side of security PSS.
The evaluation part of the CRISP methodology is participatory, systematic, and iterative by nature enabling the determination, selection and assessment of security PSS according to the four STEFi dimensions. It is participatory due to the encouragement of an (early) involvement of different stakeholders. It is systemic as a variety of differently dimensional criteria will be brought into a matrix structure. And it is iterative as the evaluation process is repeated until each potential conflict uncovered is addressed to relevant / involved stakeholders and, where appropriate solved.
Regardless of the issue that there might be security PSS that do not need to go through all proposed STEFi dimensions, the overall benefit of the participatory approach is about increasing the level of inter-subjectivity, releasing stakeholders “from their usual self-assured methods of simplification and [confronting] them with the complexity and the effects of their [previous] decisions outside their professional or common view”. This fact might seem to conflict with standardisations’ logic, which seeks to reduce complexity. Consequently, the challenge is to test how the theoretical complex appearing part of the CRISP approach fits in with the complex world of certification and standardisation.
Innovative aspects of the presented methodology are: the evaluation part based on STEFi which is designed to allow appropriate improvements/measures of a security PSS at different development stages which can act as incentives for certification as it allows a product, system or service optimisation in the first place. Consequently, a ‘project leader’ (e.g. manufacturer) should be motivated to start the STEFi assessment not only when it comes to attestation. Especially when it comes to a newly developed/designed security measure, an evaluation of a security PSS based on STEFi might help to reduce costs in the long run.
5.2 Testing and refinement of the CRISP methodology
Once the CRISP methodology was sufficiently developed, four scenario-based workshops (on border control system, on drones, on CCTV & alarm systems, and on housing areas) were organised to test and further refine the CRISP methodology. The workshops aimed at ensuring a strong stakeholder involvement and feedback towards the developed methodology from both the supply and demand side of security system, as well as other relevant stakeholders. Based on the outcome of the workshops, the CRISP methodology was refined. The following two important adjustments to the methodology resulted from the workshops:
- The conformity assessment function of ‘inspection’ (i.e. on-site assessment) was added to the function of “auditing”
During the feedback discussions, it became obvious that just to assess the provided documentation will not be enough to attest a security system and should be extended in the on-site assessment. This on-site assessment is necessary to verify that some statements from the Overall Evaluation Report conform to the requirements of the CRISP certification. Based on the ISO/IEC 17000:2014, this function is performed by inspection, defined as examination of a product design, product, process or installation and determination of its conformity with specific requirements or on the basis of professional judgment, with general requirements.
- The conformity assessment function of ‘surveillance’ was added
The decision to exclude surveillance as a standard function as it is not always performed within the certification process was made. However, the results of the scenario-based workshops highlighted the need for surveillance in order to control any changes in a security system and to maintain the validity of the CRISP certification. Therefore, the function “surveillance” was added to the certification part of the CRISP methodology.
The CRISP methodology was particularly refined on the certification part, resulting in three stages for the certification part.
During the evaluation part, an assessment of security PSS is to be carried out, according to the requirements based on the evaluation criteria as defined. Some of the evaluation criteria have been identified based on the current European normative documents, such as legislation and European standards. Other evaluation criteria have been chosen on the basis of relevant literature sources and research project outcomes. All identified evaluation criteria are used to formulate the requirements in the CEN Workshop Agreement (CWA).
Certification is based on requirements. These requirements are a combination of general requirements used in certification and requirements that specifically apply for the CRISP approach. The general requirements are drawn from the ISO/IEC conformity assessment standards, while the specific requirements are defined based the scenario-based workshops.
Finally, all of this work provided input for the certification manual and the development of the CWA (see next sections), where the refined CRISP methodology has been further worked out.
6 CERTIFICATION MANUAL
The certification manual provides basic information directed to the CRISP organisation, the future owner of the CRISP certification scheme, for being able to complement the CRISP certification scheme based on the STEFi approach. However, all parties involved can benefit from reading and using the certification manual by finding information on (basic) requirements for evaluation and certification in the context of a concrete system category (installed video surveillance systems) as well as clarifications on roles and responsibilities of all parties involved in the CRISP context.
A short overview of the information in the certification manual is provided next.
The evaluation process
An interested person needs to apply for CRISP certification at a certification body who is authorised by the future CRISP organisation. Once the client has applied for evaluation and certification, they will be delivered with a list of the stakeholders involved in the certification and evaluation processes. The client will be asked for cooperation with stakeholders involved in the certification and evaluation processes. The client shall provide the certification and evaluation body with all required information for the assessment of the system to be certified. In this context, the client must be capable of providing the system’s objectives as well as general and specific information. If this is not the case, the client should be able to consult experts (information provider), either from its own company or third parties, who are familiar with the system. Apart from the general requirements detailed above, the client shall adhere to the conditions that guarantee a fair, transparent and efficient procedure.
- The analysis conducted - configuration and assessment: General information on the security system shall be provided by the client in the first stage of configuration. The configuration results, functioning as input for the assessment stage and as an information source for the STEFi experts when they are invited to the assessment of the security system later on. The assessment stage follows the configuration stage. During this stage, a security system is evaluated by using an assessment questionnaire taking into account the four STEFi dimensions. ‘Evidence’ must be provided by the client and by the experts in order to make their decisions/answers evidence-based. An evaluation body shall ensure that all parts of the evaluation criteria questionnaire are addressed not only by the client or only one STEFi expert, but by both actor roles in order to identify any differences in opinions and (expert’s) views.
- The results derived from the analysis and conclusions to prepare for certification: Once the evaluation criteria are all answered, the assessment stage will be closed by the evaluation body. In a next step, the configuration and assessment information needs to be summarised and analysed in an evaluation report. CRISP evaluation does not only serve the purpose of gathering information on a specific use case of a security system which is more or less obvious, but also to identify and raise awareness for interrelations, revealing potential conflicts within and between STEFi criteria.
The STEFi approach integrate complex and multidimensional relationships into one approach; it does not mutually exclude the single perspectives of security, trust, efficiency, and freedoms, but rather unites them in a systematic and systemic way. Consequently, the merging of the criteria per dimension allows to detect potential conflicts between criteria and dimensions.
The evaluation body shall monitor the implementation and the integrity of STEFi assessment and coordinate the work with STEFi experts, which need to act independently and impartially. The evaluation body shall receive regular reports on the activities of the STEFi experts and establish supervision mechanisms, which allow STEFi experts to report any observations or irregularities they detect in the course of their work in an anonymous or confidential way. Once the evaluation phase is closed, the results derived from the analysis and the conclusions drawn will be forwarded to the certification body by the evaluation body. The evaluation reports shall serve as the basis for a third-party review, decision and attestation.
The certification process
The purpose of the certification phase is to verify whether the evaluation phase has been conducted according to the applicable requirements and whether the STEFi criteria have been met sufficiently. The certificate will only be granted after the confirmation of affirmative results of the evaluation.
The certification body shall check the evaluation reports with respect to completeness, reliability and conformance with applicable requirements (as specified in the future CRISP certification scheme). The extent to which the requirements of the STEFi criteria are fulfilled per dimension should be assessed by the auditor taking into account the context (situation/scenario) in which the security systems are used.
The review, decision and overall attestation process cannot be outsourced but needs to be conducted by the accredited certification body who is contracted by the client. However, it is necessary that the certification body outsources their evaluation activities, and only to organisations that meet the requirements as specified in the future CRISP certification scheme. The certification body shall have a legally binding contract with the body providing the outsourced services, including provisions for confidentiality and conflict of interest.
7 CEN WORKSHOP AGREEMENT
A CEN Workshop Agreement (CWA) is an agreement developed and approved in a CEN workshop, which is open to the direct participation of anyone with an interest in the development of the arrangement. The development of a CWA is fast and flexible, on average one year, and the document called a CWA is published by CEN.
CRISP developed a CWA titled “Guidelines for the evaluation process of installed security systems, based on the STEFi criteria”, which presents the innovative approach developed in the CRISP project: the methodology to assess systems from the perspective of four different, though interrelated dimensions. These dimensions are referred to as the STEFi dimensions (Security, Trust, Efficiency and Freedom infringement) and the methodology integrates these in its evaluation phase. The methodology acknowledges and addresses the complexity of assessing security systems by identifying potential conflicts between the various assessment dimensions and related criteria and by providing an approach to resolve these conflicts in specific situations. The methodology does not single out technical, legal, social or economic aspects, but integrates these in a multidimensional and multi-stakeholder assessment.
Efforts have been undertaken to present the evaluation methodology as clear as possible, including examples of evaluation questions and requirements for further understanding on the application of the evaluation. This allows the CWA to be used and tested in specific contexts in which security systems are implemented. The CWA is considered as a useful first concept of a checklist which can be further refined by the testing results. Moreover, the CWA stimulates awareness amongst end-users to consider the four STEFi dimensions in the field of security systems.
The CWA may be used as a standardization resource in the development of the certification scheme. Third-party certification should mostly appeal to large companies and installations. However, for smaller companies, such as SMEs, that may not be interested to go all the way to have a certified system, the CWA can still be useful when regarding as a tool for self-assessment, by self-evaluating the compliance of an installed security system according to the STEFi dimensions. Security systems in development cannot be certified, but the CWA may again be useful in this phase, allowing for evaluation of the system in development and consequently improving the design of a system.
The CEN Workshop Agreement (CWA) is published as CRISP deliverable, but will also be published as an official CEN-CENELEC standardisation document (expected publication date: May 2017). The CWA will then be available on the CEN-CENELEC website as well as via the members of CEN-CENELEC, i.e. the national standardisation bodies. Arrangement to make the CWA freely available has been taken care of, allowing any interested stakeholder to consult this document, spreading the knowledge beyond the project lifetime and via other sources.
The CEN-CENELEC procedures apply to the CWA, which consequently means that the document will be reviewed after three years. The review may result in: to keep the CWA as it is for another three years if the CWA is still up to date; to withdraw the document in case the CWA does is not valid anymore; or to revise/update the document if new insights have been collected and/or changes occurred in the context.
8 ROADMAP AND IMPLEMENTATION PLAN
Roadmaps are used to strategically plan and describe the steps to achieve outlined outcomes and goals. The CRISP roadmap is modelled on the basis of guidance from the IEA and thus includes the sections goals; milestones, priorities and a timeline; barriers and gaps; and implementation plan and action items. Furthermore, the roadmap includes an outlook chapter to visualise the future for the CRISP scheme.
At the beginning of its work on the roadmap, the CRISP consortium specified its mission, which was already presented at the beginning of this report. Based on this mission, the overall goal of the CRISP project was the production of an innovative assessment procedure, which leads to the certification of security technologies. It aimed to include social/societal aspects as key elements in the evaluation and certification methodology, which results in a future pan-European certification scheme and fosters harmonisation of the European security market. On this basis, the following sub-goals were defined:
(a) Develop a high-quality scheme and trustworthy CRISP certificates
(b) Develop a cost-effective CRISP scheme
(c) Achieve synergies with regulatory demands
(d) Become recognized and accepted by certification bodies
(e) Achieve backing-up by insurance companies
(f) Start gaining acceptance and penetration in appropriate regions
Based on a market analysis, the CRISP scheme shall be piloted within the area of video surveillance systems as a starting point. In total, CRISP’s roadmap includes 14 steps:
(1) Duration of the CRISP project
(2) Development of the CWA
(3) Building interest among stakeholders to establish a CRISP organisation
(4) Establishment CRISP organisation
(5) Completion CRISP’s pilot scheme
(6) Development Evaluation Tool
(7) Building interest among certification bodies to offer pilot and final scheme
(8) Pilot phase
(9) Building of close relationship with certification bodies, regulators, policy makers
(10) Marketing campaign
(11) Development of CRISP standards
(12) Development of the extended scheme and preparation of its accreditation
(13) Accreditation of the CRISP scheme and certification bodies
(14) Certification based on extended scheme
Furthermore, the CRISP consortium defined twelve milestones in three stages to represent interim performance targets for achieving the goals, assigned to specific time periods on the CRISP timeline that started in 2016 and is extended to beyond 2028:
Stage 1:
• M1: Completion of CRISP‘s CWA
• M2: Establishment of the CRISP organisation
• M3: Completion of a pilot scheme for video surveillance systems based on CRISP’s CWA
Stage 2:
• M4: Issuing the first pilot certificate
• M5: Early adoption of the CRISP pilot scheme for video surveillance by ten certification bodies from EU Member States
Stage 3:
• M6: Finalisation of the CRISP standards
• M7: Completion of the CRISP scheme extended to other security systems
• M8: Confirmation of the accredibility of the CRISP scheme
• M9: Accreditation of the CRISP scheme
• M10: First accreditation of certification bodies, which want to provide CRISP certification
• M11: Issuing of the first final CRISP certificate
• M12: Market penetration by 10% in Europe at least in the area of video-surveillance systems
The first stage includes three milestones. The CRISP project finished with the CWA in March 2017 (M1). The next milestone will be the specification of the members of the CRISP organisation (M2). The final milestone of the first stage (M3) refers to the CRISP organisation completing the pilot scheme based on the CWA and the innovative CRISP methodology. In parallel with this development , it is planned to develop an evaluation tool which shall be also completed at M3. The three milestones also represent the targets that show the process towards achieving the sub-goals a to c. (The preparation of M7, the completion of the final CRISP scheme will also build on these principles.)
The next “piloting” stage, which includes the issuing of the first certificate in the area of video surveillance systems (M4), relates to the goal of recognition and acceptance of the CRISP scheme by certification bodies (sub-goal d) and to the sub-goal of gaining early adopters and penetration (sub-goal f). M5 refers to the early adoption of the CRISP pilot scheme for video surveillance systems by ten certification bodies from EU Member States and serves as a target of achieving the above mentioned sub-goal c, as well as achieve backing up by insurance companies (sub-goal e). Success in the piloting phase will be shown by the market share of the CRISP scheme in the pilot area in the following years.
The last stage represents milestones related to the further development and extension of the CRISP scheme to be offered to other security systems rather than being limited to the pilot market of video surveillance systems. The milestones represent targets in relation to all of CRISP’s goals, as in the previous two stages, only now addressing the development and uptake of the extended CRISP scheme, available for other security systems and based on developed CRISP standards.
It is expected that CRISP standards will be developed by 2023 (M6), originating from the CWA developed as part of the project. Standardisation activities take time and therefore this process will run for five years in parallel with the pilot testing. M7 concerns the completion of the CRISP scheme extended to other security systems, whereas the following two milestones represent the accreditation of the CRISP scheme (M9) after receiving confirmation of the accredibility of the CRISP scheme (M8). The next two milestones refer to the accreditation of the first certification body wishing to offer CRISP certification and the issuing of the first CRISP certificate based on the final scheme by one of these organisations. The last milestone (M12) signals that the CRISP scheme reached the desired market share of 10%, at least in the area of video surveillance systems.
The roadmap extends beyond the duration of the CRISP project and outlines the role of a CRISP organisation that will take over the further development and implementation the CRISP scheme. The milestones are thus split between the CRISP consortium, the future CRISP organisation and other parties, which want to offer CRISP certification in the future. The milestone of the former was the completion of the CWA, which is already realized. The milestones assigned to the CRISP organisation consider, for example, the introduction of the CRISP scheme, the facilitation of pilot activities and the finalisation the CRISP standard(s) while the accreditation as well as the issuing of CRISP’s first pilot and final certificates build on combined efforts of the future CRISP organisation and the future providers of the relevant certificates.
For the CRISP scheme to succeed, external stakeholders are highly relevant, especially for consultation purposes, as well as promotion and support of the new scheme. The roadmap defines these groups as well as their roles and responsibilities for specific milestones and actions. It is clear that, for the purposes of building a strong and accepted scheme, the security industry, regulators, DPAs, certification, standardisation and accreditation bodies are identified as crucial in gaining broad acceptance and ensuring the take-up of the CRISP scheme by demonstrating support, offering and seeking certification and participating in standards development. The European Commission and Member States will also play an important role in facilitating the legal and regulatory framework, within which the CRISP scheme will operate.
The CRISP consortium realised the importance of acknowledging potential barriers, such as public acceptance, market structural barriers, regulatory limitations, technology limitations, gaps in knowledge or other barriers, that can potentially hinder the achievement of the goals and milestones. The barriers and gaps were extensively discussed with stakeholders and are addressed.
In addition to a roadmap, an implementation plan and a list of defined action items was also developed, so that the implementation of the CRISP scheme can be robustly planned. The key elements of the CRISP implementation plan are based upon the CRISP organisation that will take on the management of the CRISP scheme, to whom this roadmap is in fact directed to. The CRISP consortium has prepared selection criteria for such a CRISP organisation. Furthermore, the implementation plan identifies the resources needed by the chosen organisation to implement the CRISP scheme and action items, including securing financial and human resources to ensure adequate conduct, education, promotion, risk management and further development of the CRISP scheme. Action items for external stakeholders are also identified and arranged in order of priority and by stakeholder groups. Action items for the CRISP organisation include:
• Seek early engagement of insurance companies, consultants, installers to ensure their interest in the CRISP scheme
• Ensure early engagement of the public and other stakeholders, such as end users, manufacturers, data protection agencies, etc.
• Complete the CRISP scheme appropriately by ensuring that it provides significant added value for the end users
• Establish communication with stakeholders for appropriate standardisation activities at EU level
• Establish appropriate communication with experts on European and Member States’ regulation and legislation to identify best options for the characteristics of the scheme and to update the results of the CRISP project
• Ensure early engagement of regulators to ensure an appropriate development and refinement of the requirements of the scheme as well as greater promotion and acceptance of the CRISP scheme
• Negotiate the use of the CRISP scheme and mutual recognition agreements with European certification bodies
• Conduct activities to sign licensing agreements with certification bodies
• Conduct an appropriate marketing campaign to identify and highlight the added value of the CRISP scheme for end users and describe it properly to different markets of end users )
• Create an appropriate training programme for evaluation bodies, certification bodies etc.
Action items for external stakeholders to support the CRISP scheme are:
For the European Commission:
• Publically advocate for and support the CRISP scheme and its uptake in the EU Member States to facilitate positive framework conditions for the emergence of the CRISP organisation
• Develop a proposal for legislation regarding security certification with minimum requirements (a proposal for legislation)
For the industry/industry representatives, associations
• Partake in CRISP’s standards development
• Support the formation of the scheme to ensure that it is fit-for-purpose
For European standards bodies
• Develop (a) standard(s) based on CRISP’s CWA
• National bodies: agree on starting such a (standardisation) project, to work on the common document, seek agreement on its content later
• Members of relevant TCs and WGs: check the need for additional standards in the CRISP context and initiate relevant activities where necessary
For European certification bodies
• Promote the CRISP scheme to the customers
• Undertake training in assessment and evaluation methods for the CRISP scheme
• Participate in the further development of the CRISP scheme
• Partake in CRISP’s standards development
• Get CRISP accreditation after the completion of the relevant standard(s)
For European accreditation bodies
• Monitor that there are no national differences in CRISP-related certification
For Regulators
• Oversee the CRISP certification so that it complies with legislation and regulation to the highest possible degree
• Participate in the development of the CWA and the CRISP standards
For Member States
• Conduct market surveillance, ensure conformity, and inform the public
• Examine and identify whether the scheme posts any challenges to the national regulatory/legislative framework and care for appropriate solutions, if/where necessary
9 ENHANCING CONFIDENCE AND ACCEPTABILITY IN THE NEW CERTIFICATION MEASURES
Dedicated efforts have been made in enhancing confidence and acceptance of the proposed work amongst the key stakeholders.
First, efforts were undertaken to build confidence and enhance acceptance of the new certification measures, specifically in an EU-wide security standards and certification process. The CRISP consortium reached out to different stakeholder groups, presenting the CRISP project output and engaging them vigorously to provide feedback which was used to improve the results. Here the efforts made to engage and gather input from the regulators and certification bodies, together with the manufacturers and the experts involved in the relevant European standardisation working groups, are the most important since they are a crucial part of the EU wide standardization and certification processes. Additionally, , the project invested many efforts to engage stakeholders in the CWA development process as the CWA is one of the main outputs of the project, by itself leverages EU wide acceptance of CRISP methodology.
Secondly, measures were promoted to ensure the success of the EU-wide security standards and certification scheme. This has been addressed by extensively communicating to all the stakeholders the benefits of the CRISP innovative approach to holistic evaluation certification of security systems and the pan-European approach of the measures. Engagement of the stakeholders on as many occasions possible, by presentations, face-to-face communications and events, and inclusion of feedback in the development of the building blocks of the CRISP approach can also be seen as an excellent measure to ensure that their views and expectations are well built into the proposed solutions, thereby ensuring acceptance and success of the solutions that were designed in collaboration.
Furthermore, key stakeholders were engaged and involved in the process of EU-wide security standards and certification scheme making. The consortium actively approached the different stakeholders to get them acquainted with the CRISP project and to get them involved in the finalization of the outputs, in their local languages, by the local partners that have established reputation in their respective field. A direct link can be made here to the process of the CWA development, where the consortium used all the activities related to the promotion of the CRISP approach in specific stakeholder groups, to also draw them into the standardization process of the CWA development. The consortium was successful in this task, since it managed to bring around the table a diverse group of experts representing organizations from all stakeholder groups coming form 10 EU Member States, which give further credibility and value to the CWA output.
Finally, advocates among the key stakeholders were identified that could facilitate gaining confidence and enhance acceptance of the new evaluation and certification measures. All partners have developed excellent relationships with the respective experts, by informing them about the activities and the progress, meeting with them, involving them in the CWA process and gathering their input, as well as by inviting them to the events and activities of the project. Project partners have identified the advocates among stakeholders that represent some of the most important market players in the EU, the relevant EU and international working groups in standardization, consumer organizations, regulators and certification bodies, and they also kept a close tie with them for further promotion and dissemination of the CRISP project results.
Strong engagement with different stakeholders was achieved; partners have efficiently used resources to reach the stakeholders in a great number of EU Members States, resulting in close to perfect outreach to all EU Member States at the level of all activities, also making efforts to reach the smaller ones.
The CRISP partners prepared four briefing papers, which were aimed at presenting the CRISP project, approach and methodology to different stakeholder groups:
• For regulators
• For manufacturers
• For certification bodies
• For the public and consumers
The briefing papers aimed to enhance confidence in the CRISP evaluation methodology and raising awareness specifically within the above stakeholder groups. The approach included targeted communication to organizations and individuals (phone calls and emails) across Europe. The briefing papers were emailed to specific stakeholders (a mailing list consisted of 138 specifically selected stakeholders and organisations) and advertised on twitter and the project website.
This targeted communication included emails and telephone calls resulted in strong ties between the CRISP project and partners from 7 consumer organizations in different European countries, extensive communication with 62 individuals within the stakeholder group regulators and policy makers, and established a close working relationship with manufacturers from 8 EU member states.
More on dissemination activities of the project in general can be found in the next section.
Potential Impact:
EXPLOITATION RESULTS
The CRISP project mission was to develop an innovative evaluation and certification methodology for the CRISP certification scheme for security systems. The exploitation results consist of the following building blocks for the CRISP certification scheme:
• CRISP methodology (D5.1 D5.2)
• CEN Workshop Agreement, where the evaluation methodology is recorded, exemplified with evaluation criteria (D7.2)
• CRISP certification manual and roadmap (D6.1 and D6.2)
• CRISP exploitation plan (D7.4)
• Briefing papers on the proposed certification scheme and roadmap:
o For regulators
o For manufacturers
o For certification bodies
o For the public and consumers
These building blocks will allow the future scheme owner to take up and further develop this, finalising the development of the CRISP certification scheme and operationalize it.
POTENTIAL IMPACT
The major potential impacts of the CRISP project are:
• Enhancing the knowledge base of evaluation and certification of security systems
• Stimulate awareness amongst end-users in social, economic and legal aspects in evaluation and certification of security systems
• Providing a level playing field for the security industry in Europe
• Enhancing the competiveness of the European security industry
• Enhancing trust of stakeholders and especially end-users and citizens in certification of security systems
• Fostering dialogue on security certification at all levels
• Facilitating implementation of relevant EU legislation related to security systems, in particular the data protection regulation
These potential impacts are briefly addressed in the following paragraphs.
• Enhancing the knowledge base of evaluation and certification of security systems
The project is highly interdisciplinary and draws upon expertise in the diverse disciplines and security domains. As the integration of social and legal aspects in the methodology is very innovative, the urgent need to involve and engage with stakeholders from different categories was recognised, and this has been incorporated throughout the project.
The early involvement of and frequent interaction with stakeholders (including an Advisory Board) also had a positive influence on the quality of the research work as well as the support for the work, as feedback and review from the stakeholders was included throughout the project.
In the CRISP project an innovative approach to evaluation and certification of security systems was developed. The most innovative part of the methodology is the assessment of systems from the perspective of four different, though interrelated dimensions. These dimensions are referred to as the STEFi dimensions (Security, Trust, Efficiency and Freedom infringement) and the methodology integrates these in its evaluation phase. This is an innovative approach as certification has, to date, primarily focused on the evaluation of technical requirements for security systems (the security dimension) or singled out other relevant dimensions (e.g. privacy or data protection in the freedom infringement dimension). The methodology described in this CWA, however, is not (over)simplifying the complexity of assessing security systems but acknowledges and addresses this complexity by identifying potential conflicts between the various assessment dimensions and related criteria and by providing an approach to resolve these conflicts in specific situations. The methodology does not single out technical, legal, social or economic aspects, but integrates these in a multidimensional and multi-stakeholder assessment.
The project developed collective capability, by consolidating the knowledge and expertise on security standardisation and certification. The research results are published, widely disseminated, and publicly available.
• Stimulate awareness amongst end-users in social, economic and legal aspects in evaluation and certification of security systems
Advancing the state of the art in security standardisation and certification, the project offers a methodology that not only takes into consideration the technical aspects, but encourages the inclusion of social, economic and legal aspects in a multi-dimensional and multi-stakeholder environment. This innovative approach on evaluating security systems should provide guidance to stakeholders that are interested in the inclusion of all of these dimensions in design and operation of security systems.
Next to using the CWA for certification, the CWA can also be regarded as a tool for self-assessment, by self-evaluating the compliance of an installed security system according to the STEFi dimensions. For security systems in development, the CWA may be useful in evaluating the system in development and consequently improving the design of the system. The CWA may stimulate awareness amongst end-users to consider the four STEFi dimensions in the field of security systems.
• Providing a level playing field for the security industry in Europe
The current certification market for security systems is highly fragmented as different requirements are set at the national levels and a variety of different certification schemes is applied. The CRISP approach is complementary to the already existing standards and certification schemes, not competing with them. The results of the project will facilitate a more harmonised playing field for the European security industry by providing the basis for a pan-European certification scheme for security systems. The future scheme is intended to be accepted across Europe, which will also enhance competitiveness by reducing commercialisation costs for the industry.
• Enhancing the competitiveness of the European security industry
The European security industry is an important industrial sector, however, due to a fragmented approach by differing national legal requirements and governmental policies and still a lack of European standards for a range of components and systems, the sector is not as competitive world-wide as could be. The results of the CRISP project will contribute to more cost-effective operations for the security industry by opening the European market.
• Enhancing trust of stakeholders and especially end-users and citizens in certification of security systems
The results of the CRISP will enhance trust of stakeholders in certification of security systems because in addition to technical requirements that ensure that a sufficient level of security is achieved also requirements related to societal concerns such as safety, cost efficiency and freedom infringement are taken into account in the evaluation and certification of security systems.
• Fostering dialogue on security certification at all levels
Since security evaluation and certification is an iterative process, CRISP fostered dialogue on security certification at all levels, particularly amongst:
- The stakeholders that can opt for CRISP evaluation/certification: security industry, installers and end-users of security systems, also via industry associations
- The certification bodies, i.e. providers of certification
- The regulators, who gain from CRISP evaluation/certification efforts of contributing to compliance with regulatory demands
- The public and consumers, as the ones being scrutinized by security systems and having interest in safeguarding their rights during surveillance
Not only workshops addressing all stakeholders, but also some dedicated workshops for specific stakeholder groups have been organised. Furthermore, briefing papers for each of the above-mentioned stakeholder group were developed. In the briefing papers special attention was given to build confidence and enhance acceptance of the proposed scheme, in order to stimulate interest in the future steps after the project.
As the certification scheme will need to be further developed and finalised and this is out of the scope of the CRISP project, documentation is included to guide the future CRISP organisation/scheme owner in taking the next steps. Next to the roadmap and implementation plan, the certification manual and exploitation plan are specifically addressed to the future scheme owner.
The value of the future certification scheme for the four main stakeholder categories is voiced and validated with the data protection/regulatory demands as main driver for security industry/ installers of systems and certification bodies.
• Facilitating implementation of relevant EU legislation related to security systems, in particular the data protection regulation
The CRISP methodology supports compliance with European legislation: the CRISP scheme will contribute to and promote compliance with relevant legislation, in particular with the General Data Protection Regulation provisions, and consumer protection legislation, as well as with relevant technical standards. By employing a multi-dimensional approach it will offer insight to the regulators on dimensions that are not in their primary focus, but nevertheless significantly impact the security solution in question. A trustworthy certification, issued by appropriately accredited certification bodies, also brings direct benefits to supervisory bodies in their supervisory procedures, as it is possible for them to inspect the documentation behind the issued certification which aids effectiveness of their procedures.
MAIN DISSEMINATION ACTIVITIES
Specific activities focused on enhancing confidence in the new certification measures were undertaken, and within this remit different stakeholder groups were approached with targeted information to promote and boost interest in the CRISP methodology. This has been described as part of the S&T results. This section focuses on the dissemination and communication of the project and project results. Overall, the project included stakeholder contact in some form, through research and validation activities, e.g. interviews, surveys, and workshops, through which the profile of the CRISP project was raised and its findings promoted.
The CRISP consortium successfully promoted the project, findings and events throughout the three years and awareness amongst key stakeholder groups was high. Due to the innovative approach to certification, stakeholders expressed a keen interest in the project findings, and in the future of the CRISP methodology. This was especially apparent during the final conference where questions about the future CRISP scheme were frequent.
The project dissemination strategy included a variety of approaches and tools that partners used in conjunction to reach as many relevant audiences as possible. This included publications and attendance at third party events, website, social media, electronic newsletters and CRISP led events, such as workshops, roundtable event and the CRISP final conference.
Website and social media
The CRISP website at http://crispproject.eu was set up at the start of the project. The website holds all information on the project, the project consortium, related projects and all public project deliverables. The project partners also used the website to communicate information about project events and new deliverables. The project website will be maintained for a year after the project ends, to ensure that all materials continue to be available to those who wish to use the project deliverables for further research or innovation. All project deliverables will be uploaded to Zenodo to ensure that they are available for future use, after the website is taken down.
Traffic to the website was tracked by using Google Analytics. Throughout the project there have been over 11 thousand visits to the website from just under 9 thousand individual users. In total 1,493 downloads have were made from the website during the project. The deliverables on the overview of standards and certification in Europe and internationally have consistently been at the top of downloads throughout the project, demonstrating that this knowledge is very relevant and useful to stakeholders working within the field of security certification. In April 2015 the CRISP twitter account (@CRISP_project) was set up to communicate project news, events and deliverables could be further communicated. At the end of the project the twitter account has a very good following of 155 followers and has tweeted 313 times. A Slideshare account for the CRISP project was also set up through which to disseminate slides from external and internal events. The Slideshare enables greater sharing and promotion of CRISP findings in short and easily digestible format. The CRISP Slideshare account now hosts 15 slide presentations from project events, third party conferences and from the CRISP final conference. For the final year of the project (March 2016 - April 2017) slides from the account have been viewed 721 times in total and downloaded 61 times.
The project has a very strong contact list of 380 contacts that the partners have added to throughout the project. These contacts have received 9 electronic newsletters to inform them about the progress of the project, key deliverables and upcoming events.
CRISP Events
Twelve stakeholder events were held by the CRISP partners during the project, these included workshops (stakeholder, validation and scenario), roundtable with Data Protection Authorities, and a Final Conference. All events were attended by experts in security, data protection, certification, standardisation and accreditation. In addition, a number of events were also attended by security industry, and industry association representatives. The events served to raise the profile of the CRISP project, and allowed the partners to disseminate findings and validate them with the help of the attending experts.
The CRISP Final Conference was held on 16th March at the BAO Congress Centre in Brussels, Belgium. The conference was co-located with the 6th Plenary of the Community of Users on Secure, Safe and Resilient Societies, and related Thematic Workshops, held by the European Commission DG HOME. The conference consisted of two parts, a plenary session in the morning and two topic specific workshops in the afternoon. The plenary session consisted of presentations on the CRISP results from the project coordinator, as well as externa presentations on the broader security industry, policy and data protection context and how CRISP results can prove useful to address specific issues. The CRISP partners and experts then took questions from the audience and facilitated discussion on the future of the CRISP scheme. The two afternoon workshops consisted of 1) an in-depth presentation of the STEFi methodology and the CRISP CWA, and 2) standardisation and certification basics, and a discussion on how standards are used within the security field.
Attendees could sign up for the whole event or choose to attend either plenary session or workshop only. The final conference was attended by 52 participants from various European countries and stakeholder groups, e.g. certification bodies, industry, standardisation bodies, security academics, data protection authorities, and accreditation bodies. All presentation slides from the conference are available from the project SlideShare account.
Publications and presentations at third party events
The CRISP partners attended 49 external events in Europe, and internationally through which they presented CRISP results through presentations and posters. Through these events the CRISP project has been promoted to audiences in certification, standardisation, security, disaster resilience, privacy, data protection, cyber security and surveillance studies. A number of papers have been published and at least three papers are forthcoming, along with two book chapters. CRISP partners have also published in magazines and via conference proceedings.
SUCCESS FACTORS FOR FURTHER EXPLOITATION
The CRISP project holds the opinion that the CRISP certification scheme shall be set up and maintained by the future scheme owner in cooperation with interested stakeholders. For the acceptance of the scheme by all relevant stakeholders and to realise its full potential, the following preconditions and key success factors for the future CRISP certification scheme is summarised; the exploitation plan provides further detailed information.
The following preconditions have been put forward:
• Commitment and active support to the certification scheme by all relevant stakeholders;
• Consider the best practices of other relevant certification schemes and existing European and international standards as input for the development of the certification scheme;
• Organize the commitment of the regulatory and political authorities and other relevant stakeholders by involving these as members of a future Advisory or Steering Board which acts as an external monitoring and oversight group to the CRISP organisation;
• Endorsement of the CRISP certification scheme by the regulators, EC and the Member States would result in faster implementation and uptake of the scheme, since compliance with regulatory demands, especially data protection, is an important driver for the market CRISP certification scheme is targeting at;
• Starting with pilot studies during the development of the certification scheme to use the outputs of the pilots for improving the eventual certification scheme.
Crucial factors which need to be taken into account by the future CRISP organisation in the process of finalising the CRISP certification scheme are as followed, as they present key considerations in the formulation of a successful scheme:
• Quality of the scheme - simplicity, openness, transparency and accreditation
The legal standing of the proposed CRISP scheme must be clear and should be open to any interested organisation and its details should be accessible by consumers. The certification bodies offering certification according to the future CRISP certification scheme will need to be accredited in order to achieve the necessary level of trustworthiness, quality and acceptance. Interactions with national accreditation bodies are necessary to facilitate their future task of accrediting certification bodies based on the relevant documents of the CRISP scheme. Finally, the certification procedures shall be as simple as possible.
• Support, acceptance and promotion from key stakeholder groups
The support, acceptance and/or promotion from the different stakeholders, such as the European Commission, national authorities, the industry and security industry bodies (e.g. Euralarm), regulators and certification bodies is the main precondition for the development of the certification scheme.
• Appropriate Management
The proposed CRISP certification scheme must be kept up to date to ensure its relevance, resulting in a scheme that is kept flexible to adapt to technological innovations and any economic and societal changes. In addition, oversight mechanisms must be in place to assure consistency of the scheme, relevance of criteria and validity of certifications. The scheme management must oversee enforcement and assessment processes to ensure that all requirements of the scheme are complied with.
• Impartiality
Impartiality is one of the main aspects associated with CRISP certification and ultimately crucial for the success of the CRISP certification scheme and the certificates issued. The independence of personnel can be safeguarded with impartiality agreements between the scheme owners and the personnel.
List of Websites:
CRISP project website:
www.crispproject.eu
Contact details CRISP coordinator:
Ronald Boon, Netherlands Standardisation Institute (NEN): R.Boon@nen.nl
CRISP (Evaluation and Certification Schemes for Security Products) is a three year project that aimed to facilitate a harmonised playing field for the European security industry by developing a robust and innovative evaluation and certification methodology for security systems.
The overall objective of CRISP is to enhance existing security evaluation and certification schemes by offering an innovative evaluation methodology that integrates the security, trust, efficiency and freedom infringement assessment dimensions. The methodology is based on a new taxonomy, developed by the CRISP consortium, which encompasses a variety of security products and services across application areas. The project also took into account the varying roles of the diverse security stakeholder community including manufacturers, regulatory and certification bodies, data protection authorities and end users and engaged with each group to gather insights in order to assist with avoiding acceptance problems that are known to challenge current schemes. This took place through dissemination, promotion, and a variety of stakeholder-focused events such as workshops and a final conference.
In the CRISP project an innovative approach to evaluation and certification of security systems was developed. The most innovative part of the methodology is the assessment of systems from the perspective of four different, though interrelated dimensions. These dimensions are referred to as the STEFi dimensions (Security, Trust, Efficiency and Freedom infringement) and the methodology integrates these in its evaluation phase. This is an innovative approach as certification has, to date, primarily focused on the evaluation of technical requirements for security systems (the security dimension) or singled out other relevant dimensions (e.g. privacy or data protection in the freedom infringement dimension). The methodology, however, is not (over)simplifying the complexity of assessing security systems but acknowledges and addresses this complexity by identifying potential conflicts between the various assessment dimensions and related criteria and by providing an approach to resolve these conflicts in specific situations. The methodology does not single out technical, legal, social or economic aspects, but integrates these in a multidimensional and multi-stakeholder assessment.
The exploitation results consist of the following building blocks for the CRISP certification scheme: CRISP methodology, CEN Workshop Agreement, CRISP certification manual and roadmap, CRISP exploitation plan, and briefing papers on the proposed certification scheme and roadmap for the key stakeholders (regulators, manufacturers, certification bodies and end users).
As the certification scheme will need to be further developed and finalised and this is out of the scope of the CRISP project, documentation is included to guide the future CRISP organisation/scheme owner in taking the next steps.
Project Context and Objectives:
PROJECT CONTEXT
The market for security products and services is one of the most rapidly growing markets in the world. However, the European security market is highly fragmented, and no common framework applies to security products and systems. Further challenges specifically related to certification of security products and systems is the lack of common certification systems for security products as well as a lack of mechanism for mutual recognition across countries of products certified at a national level.
CRISP (Evaluation and Certification Schemes for Security Products) is a three year project aiming to facilitate a harmonised playing field for the European security industry by developing a robust and innovative evaluation and certification methodology for security systems.
CRISP addresses some specific challenges. One of the challenges is that there is no effective policy to regulate liability by setting quality standards and defining which services and products fall under certain criteria. A common approach to categorise both security products and services is needed. Next, an appropriate organisational structure for standardisation and certification of security products and services is missing resulting in a plethora of standards and certification schemes, with a lack of harmonisation of schemes on European (and international) level. This is not only a constraint for the success of the EU Internal Market but also constrains European companies from becoming security market leaders and effective global competitors. Furthermore, there is a lack of transparency in respect of security products and their implementation, and a lack of effective participation from stakeholders in setting security standards and certification. Finally, the early involvement of stakeholders is crucial, particularly for the acceptance of security products and services that have the potential to have various negative societal effects. Market success relies strongly on acceptance by the public and policy makers. For this, fostering and promoting stakeholder involvement at all levels of the standards and certification process is needed.
Accordingly, the CRISP project addresses the challenges by:
• Create a taxonomy that will allow the clustering of security products and services alongside adequate parameters that will lead into an effective and efficient classification of the two.
• Propose a methodology for a common certification scheme with a common European label, tht will not only benefit the EU Internal Market, but also help the EU consolidate and carry on its leadership in the global security market.
• Involve stakeholders and integrate their perspectives at all levels, acknowledging the societal impact of security products and systems.
• A strong stakeholder engagement strategy (which includes identifying stakeholders and engaging with them throughout the duration of the project through various means and platforms) to build confidence and enhance acceptance of the proposed approach.
OBJECTIVES
The overall objective of CRISP is to enhance existing security evaluation and certification schemes by offering an innovative evaluation methodology that integrates the security, trust, efficiency and freedom infringement assessment dimensions. The methodology is based on a new taxonomy, developed by the CRISP consortium, which encompasses a variety of security products and services across application areas. The project also took into account the varying roles of the diverse security stakeholder community including manufacturers, regulatory and certification bodies, data protection authorities and end users and engaged with each group to gather insights in order to assist with avoiding acceptance problems that are known to challenge current schemes. This took place through dissemination, promotion, and a variety of stakeholder-focused events such as workshops and a final conference.
The following mission statement and scope have been specified:
Mission statement of CRISP project and scheme
The CRISP project mission is to develop an innovative evaluation and certification methodology for the CRISP certification scheme for security systems. The CRISP scheme will:
● Contribute to measures that increase citizen trust and confidence in security technologies through the evaluation and certification of social and legal impacts of security systems;
● Facilitate a more harmonised playing field for the European security industry by providing pan-European certification for security systems. The scheme is intended to be accepted across Europe, which will enhance competitiveness by reducing commercialisation costs for the industry;
● Support the goal to provide protection in an efficient manner.
The methodology integrates the Security, Trust, Efficiency and Freedom infringement dimensions, the so-called STEFi dimensions, in its evaluation stage. This is a highly innovative approach, as certification has, to date, primarily focused on the assessment of technical requirements for security systems.
This novel approach to the inclusion of social and legal aspects in evaluation and certification methodology will first be piloted for video surveillance systems, to test and refine the approach. After the pilot phase, it is foreseen that the CRISP scheme can be extended to include other types of security systems.
The CRISP methodology will serve as the foundation of the CRISP certification scheme, which will, upon the completion of the CRISP project, be further developed by an interested organisation. The CRISP scheme will not redefine the technical requirements that are already in place (e.g. in European standards). Instead, the STEFi dimensions will offer the inclusion of social assessment criteria in the certification of security systems; the scheme will contribute to the protection of fundamental rights and promote compliance with relevant EU laws, with a particular focus on the General Data Protection Regulation (GDPR) 679/2016.
Certification by the CRISP scheme will assure that a security system has been assessed on the basis of the STEFi dimensions and has been found to comply with applicable assessment criteria for security, efficiency, trust and freedom infringement. It can be sought initially by both those who procure and those who run video surveillance security systems on their premises.
Scope of the scheme
The CRISP scheme will provide criteria and requirements for the evaluation and certification of installed video-surveillance security systems based on the CRISP STEFi dimensions (Security, Trust, Efficiency and Freedom infringement). For systems in development, evaluation according to the CRISP scheme can be applied. Certification of these systems will only be possible after installation.
Target group of the scheme
End users of the relevant systems, such as local authorities, emergency organisations, transport operators, law enforcement authorities, retail organisations, health organisations and educational organisations.
CRISP PROJECT WORK
The work of CRISP consisted of:
- Creating a taxonomy of security products and systems, concepts of operations, application areas, performance and set out criteria for comparing security products and systems;
- Providing a historical perspective on security standards and certification in Europe and analysing the state of the art in security standards, certification and accreditation at the Member State, regional and international level;
- Identifying and determining the role of different stakeholders, gauging their views on the challenges affecting security certification and determine requirements for a harmonised EU-wide approach;
- Examining the core assessment dimensions such as security, trust, efficiency, freedom infringement (STEFi) of security product certification and presenting the requirements for enhancing existing evaluation and certification schemes;
- Developing an innovative methodology, based on the STEFi dimensions;
- Developing a CEN Workshop Agreement (CWA) to record the evaluation approach, including evaluation questions and requirements;
- Presenting a certification manual, roadmap and implementation plan, and an exploitation plan to be taken up by the future scheme owner;
- Disseminate briefing papers to key stakeholders, enhancing confidence and acceptance of the CRISP approach to certification.
Project Results:
1 CATEGORISATION OF SECURITY EQUIPMENT, SYSTEMS AND SERVICES
1.1 Glossary of security products and systems
As a foundation for the CRISP project, the glossary of security products, systems and services (PSS) has the unique characteristic of focusing on the functionality of security PSS instead of the technique itself. Security PSS can be defined in many ways and defining a product, system or service in a technological way often involves a discretionary component, as each technological system is comprised of several interconnected elements and each of these elements can itself be analysed as a system of its own. A security PSS can encompass everything from a simple torchlight to a system connecting video surveillance, biometric scanners and infrared sensors. Instead with the functionality of a security PSS, we describe the intended result of the PSS when in operation.
In our CRISP glossary, we derived and defined the following security functions from the literature, policy papers and research reports: ‘locate, identify, verify, control, track, assess, authorise, communicate, create situational awareness, information collection, storage and management to produce intelligence, detain, prevent/protect’. Due to the different levels of operability of these security functions, they were classified regarding their interdependency and the interrelationships of each of the security functions. Primitive functions are the basic functions of security PSS and are essential for the further performance of other functions. Connective functions make use of the primitive functions, mainly through connecting available information with specific criteria which then again can be necessary for the performance of further security related activities and functions. Performative functions finally are carried out (security) actions, with a clearly defined and targeted result, making use of one or both previous functions.
Finally, the outcome of security functions can differ depending on where the PSS is deployed. Thus in order to reflect on the different security contexts, four areas of security were derived from the literature, in which similar classifications have been done already, however often with rather redundant categories. For the CRISP glossary, the four areas of security are:
(1) security of citizens which covers all possible threat aiming at European citizens in public and semi-public spaces as well as in private spaces;
(2) critical infrastructures as general infrastructures which are of high importance of the functioning of a vital society and thus a protection against threats aiming at the disruption or destruction of the like plays an important role in the European policy making and the security industry
(3) border security which includes the means for providing security of land, air and sea, but also of borders in embassies to prevent the illegitimate crossing of people and the detection of illegal products, goods and substances within custom services
(4) Crisis Management includes mainly the restoration of security in the aftermath of a crisis, which may result from a natural disaster, but also from deliberate attacks.
1.2 Taxonomy of security products, systems and services
Based on the glossary of the first deliverable, a categorisation of security PSS was established. In form of taxonomies for security products, security systems and security services, this categorisation examines the wide diversity in the domain of security PSS. The categorisation of the taxonomy is done on four levels and aims at depicting the wide array of deployment possibilities and the related function of identified security PSS.
On the first level are the security application areas. Like the glossary, the taxonomy categorises security PSS depending on where the intended application is based. The four categories are the ones presented already in the glossary, being the security of the citizens, critical infrastructures, border security/management and crisis management/emergency preparedness centres. The second level of categorisation of the taxonomy is the security demands. The demands are mainly based on reviews of commercial security company websites, European policy literature and related security research projects and further specify the areas where the security PSS can operate, while already refining the intended security function. The security demands represent the key requirements which are necessary in the respective area of security and which should be implemented by the security function. This includes access control, asset/freight/cargo security, cyber security, employer/visitor security, loss prevention/shrinkage, perimeter/building/area security, point of transaction, situation awareness. On the third level of the taxonomy are the security functions of the glossary and specifying the intended result of the security PSS.
Lastly the security product, system or service is listed, depending on their application within that classification. The security PSS is of course not exclusive classified within one categorisation, but can, like the security function, be part of different application areas and security demands. As such, this function-based taxonomy is a very flexible tool for the categorisation of security PSS.
2 REVIEW OF STANDARDS, CERTIFICATION AND ACCREDITATION FOR SECURITY PRODUCTS
2.1 Security standards and certification in Europe – A historical/evolutionary perspective
The consortium provided an analysis of the state of the art of security standards and certification in Europe and an analysis of the economic benefits of conformity assessments. Specific insight into the framework conditions of security certification in Europe were given and security-related documents which determine the current security certification landscape or can build the foundation for future improvements in the marketplace were explained. Furthermore, specific security-related standards and technical committees in Europe were analyzed and current and potential interrelations between standards and certification were shown. Possible needs for new security-related standards were also identified. Moreover, field research revealed that fostering more standards-based certification requires stimulation by appropriate European directives.
With regard to the interrelation of standardization and certification, information was gained from interviewees and emails from CEN/TCs and analyses of the work of numerous certification bodies. Many security-related TCs were quite new and new standards were under development. Therefore, it was recommended to seek collaborations with certification bodies in early stages. In addition, there were national certification bodies which participate in national mirror committees of CEN/CENELEC/TCs. They also provide a good example for the establishment of interrelations between both fields. Talks with experts from national certification bodies show that there are institutions which are aware of the advantages of participating in standardization, but that they are still too few.
Furthermore, security areas in which the use of open standards is limited were analyzed. In particular, the fields of digital signatures, airport screening equipment and air cargo are relevant. Usually several different governmental authorities and security authorities are responsible for these topics in a Member State, making the European landscape very complex in this regard. Although databases exist that show all national certification bodies which are accredited by a national member of the European co-operation for Accreditation (EA), databases of non-EA members are not available. Therefore, an extension of current databases or creating an additional database is recommended. An additional area in which standards are not used for certification is related to innovative solutions for which standards do not exist yet.
The most important result of the review activities presented in this section is a detailed overview of the state of harmonization and mutual recognition in Europe, which describes suggested concepts of “one stop testing”. This approach is highly recommended for new product classes, new requirements and related standards. With regard to existing certification services, alternative solutions may also offer advantages.
Fields in which appropriate harmonized certification solutions are missing are, for example areas of complex security systems. In addition, there are security issues which are shaped by different national preferences in Member States. EN 50131 which includes specific national amendments provides an example for that. It shows that there are areas which should not be covered by general harmonized solutions but by complementary certification. The number of these areas is to be kept as small as possible.
The analyses also offered interesting examples for European collaborations in security certification. They include, for example, the agreement of the Senior Officials Group Information System Security (SOG-IS) and the European Fire and Security Group (EFSG). Like the international CC Recognition Arrangement, SOG-IS provides mutual recognition for certificates on information systems security. Nevertheless, many needs for further action remain.
Quality is a key issue in the certification context. EFSG builds on European standards, and its members compare their test results regularly. The group is also active in standardization, although its attractive offer to obtain multiple quality marks with minimal duplication and cost is not usable in all European countries yet. In general, mutual recognition is often practiced by large, industrialized Member States with a large security market and industry. Efforts to date have been unsuccessful in removing barriers to greater harmonization. A major obstacle for the expansion of EFSG is, among others, the perceived quality of other national certificates in the relevant fields, which again highlights the quality issue.
Instruments used by the EFSG group include, for example, round robin exercises. To facilitate harmonization in other security fields, it is recommended to use this instrument alike.
As a starting point of various activities in different CRISP work packages, the concept of CertAlarm, with whom CRISP collaborated, was also analysed and led to helpful interim results even after these first review activities.
The analyses finished with specific suggestions and recommendations for the future.
2.2 Security standards, certification and accreditation – best practices and lessons learnt
The second series of review analyses led to the identification of various good practice approaches and recommendations for CRISP’s further work. It identified attractive technical solutions, interesting examples for collaborations among certification bodies as well as approaches to consider societal aspects in security-related certification.
In addition to the pioneers in harmonisation and mutual recognition in different security areas previously described in CRISP’s deliverable 2.1 more good certification practices on national and multinational levels could be identified.
There are a few security certification schemes in specific technical areas in the Member States, which avoid the narrow regional focus on national standards. ECB’s consequent orientation on European and international standards and its collaboration with various laboratories in different Member States is an interesting example in this regard.
A good practice example regarding collaboration is the ARGE DIN 14675, established by the German TÜV Rheinland. ARGE DIN 14675 is an association of currently ten accredited certification bodies and the owner of an accredited certification scheme for the planning, operating and maintenance of fire alarm systems and voice alarm systems based on the DIN standard on ‘Fire detection and fire alarm systems - Design and operation’. Good practice does not only characterise the common, standard based certification scheme but also the collaboration between certification bodies and their stakeholders and the procedures to update the relevant certification scheme.
An important example to learn from on an international level is provided by the U.S. SAFETY Act, which applies to a wide range of security solutions. Another good practice example from the U.S. is the National Strategy for CBRNE Standards to overcome the coordination problem caused by the high number of agencies responsible for CBRNE standards.
Societal aspects of security systems are another important aspect to consider, in particular regarding surveillance solutions, because "(u)nseen, uncontrolled or excessive surveillance activities (...) pose risks that go much further than just affecting privacy. They can foster a climate of suspicion and undermine trust”
Examples for partial response to this statement already exist in the certification field. Besides EuroPrise, a certificate for IT products, the British CCTV Code of Practice for CCTV systems example considers several specific aspects of freedom infringements. The VdS 2365 on CCTV systems also integrated a few issues of that kind in its certification process but does not consider the full variety of CCTV-related freedom infringement aspects.
Based on the further analyses of 14 country studies, multinational standards and certification schemes, nine recommendations were derived for the further work on the CRISP scheme, its implementation and usage. They consider the following issues:
- Fast certification processes: The security market is growing and demanding new secure, efficient and trustworthy PSS, which also take into consideration the freedom and rights of citizens. In this regard, faster certification processes and a shorter developmental period for standards are needed. There is also a need for certification organisations and institutions for better solutions in order to continuously improve and accelerate their proposed services on certification for security PSS.
- Embeddedness in the socio-technical context: To avoid redundancy, the design of the CRISP scheme should take into account the structure and content of well-established national and international standards, where additional requirements can be developed on top of existing ones. The standards currently under development (for example at the international level) should also be carefully examined to the extent that the development is of a public nature, in order to search for synergies. Furthermore, its needs to be taken into account that the extent to which the CRISP scheme will seek compliance with national legal requirements, will most likely affect the adoption of the scheme.
- Performance-based requirements: A trend towards performance-based requirements in certification was identified. CRISP should therefore consider the function of security PSS within its further research activities, as well as the need for performance standards. In this regard, an active exchange by concrete use cases between practical experts and researchers is highly recommended. This will work to assure that project findings will meet essential requirements and serve to keeping standards and the development of certification processes up-to-date.
- Interoperable solutions: The importance of interoperable solutions became apparent to enable security integrators to develop systems, whose components from different vendors can communicate with each other. Within CRISP’s further research activity, this had to be taken into account.
- Implementation of the scheme: The diverse landscape of existing standards related to security PSS, coupled with a network of Conformity Assessment Bodies (CABs) that may or may not be accredited, requires careful consideration of how the certification scheme developed by CRISP will fit in into these existing relationships between CABs, accreditation bodies and end-users. Which CABs may be appropriate to certify under the developed scheme and whether accreditation of CABs will be relied upon are only some of the questions that the CRISP project will continue to seek answers to.
- Accreditation practice: An important issue to reach transparency and fair competitiveness among security certification entities is the harmonisation of guidelines for their accrediting auditors. Cross trainings of accreditation auditors, mutual assistance between them and the reduction of differences among accreditation programs are needed in order to avoid substantial different test results in the same certification scheme.
- Regular follow-up controls of certified solutions: As CRISP’s analyses showed, regular follow-up controls of the certified security PSS are significant. The certificate should represent the actual compliance or quality of the product, system or service. With regular compliance control, the certified entities have an additional incentive to keep improving or at least maintain an adequate level of compliance. This also promotes trust in the certificate and the relevant security solutions.
- Solutions for SMEs: The costs of certification must not be a competitive barrier for small and medium enterprises. First ideas include the adjustment of costs to the size and complexity of the entity as well as measures to ensure equal treatment of customers independent of their geographic location and the residence of the auditors.
- Political support: The adoption of a new pan-European approach to certification for security PSS will necessarily require political support, as it follows from CRISP’s research that standards are heavily intertwined with national legislations. Further work on the development of strategies for the adoption of the CRISP scheme must therefore also focus on the political dimension of the European security certification landscape to receive appropriate support.
3 SECURITY CERTIFICATION STAKEHOLDER ANALYSIS
In order to conduct the stakeholder analysis, a review of the relevant policy, grey and academic literature, for the purpose of identifying key stakeholder groups in security certification was done. Second, three case studies of specific technology areas were examined to understand the general and field-specific issues of security certification and key stakeholder views on and needs from certification. The case studies (which included expert interviews) gave an insight into three technologies areas and the specific needs of stakeholders within them when it comes to the development of standards, certification schemes and how they may perceive the notion of a European certification scheme for security, products, systems and services. The three case study technologies were as follows:
• Video Surveillance Cameras (CCTV)
• Remotely Piloted Aerial Systems (RPAS)
• Alarm Systems (Fire and Intrusion Alarms)
Third, the findings on stakeholder needs and views were verified and discussed at a stakeholder workshop in December 2014, and then used to develop two web surveys to ascertain the prevalence of views expressed among a broader set of stakeholders in Europe.
The review of literature identified the following key groups of stakeholders:
• Security product manufacturers, suppliers and systems integrators (henceforth referred to as security industry)
• Conformity assessment and certification bodies
• Standardisation organisations
• Accreditation bodies
• Data protection authorities and other regulators
• End users
Secondary stakeholders (affected by and indirectly involved in security certification):
• Watchdogs and civil society organisations
• Individuals
• Academics
The bullet points below, give a brief summary of key stakeholder groups’ needs from security certification, as they were presented to us through the research:
• For the security industry, certification is important for accessing different national markets as well as providing requirements to which products and services are designed and developed. The key need of the security industry is for increased harmonisation of certification across borders for the purposes of avoiding re-certification which is costly and time-consuming and causes delays to market.
• Certification bodies express the need for a certification scheme, founded on robust standards and clear evaluation criteria. It is also of key importance that a certification scheme is well known, accepted and trusted, thus offering clear added value for industry. In the absence of the aforementioned qualities, certification bodies will have difficulty justifying and selling the scheme to security industry stakeholders.
• Accreditation and standardisation bodies, while sharing concerns regarding lack of transparency and complexity of the current certification landscape, view their processes for accreditation and developing standards as robust and fit for purpose for supporting a new certification scheme. Their needs regarding any new certification scheme is that its purpose, evaluation criteria are clear and transparent so that standards development or accreditation is consistent and straightforward.
• Out of the group of regulators relevant to certification, Data Protection Authorities (DPAs) were identified as very relevant in light of the social evaluation of the CRISP scheme. Currently, DPAs are usually not involved in standards development nor certification (with the exception of Germany) but due to their focus on privacy and data protection aspects of surveillance and other security technologies, they present as a stakeholder which will be necessary in further developing the certification scheme and ensuring acceptance and trust.
• End-users feel that the current certification system is complex and opaque and that there are too many schemes/seals and it is not clear to what the difference is between them. Increasing transparency regarding evaluation criteria for different schemes is thus imperative to better meet end-users needs regarding certification.
Regarding the CRISP certification scheme and its focus on evaluation of social dimensions, we found that overall stakeholders were positive towards this type of scheme. The research findings however revealed that stakeholders were uncertain about what such a scheme would look like and how it would be applied. There were some concerns that social dimensions would be too vague to serve as efficient evaluation criteria. This indicates that further to design and development of the scheme, the CRISP consortium must raise awareness about the prospective scheme amongst relevant stakeholder groups, to ensure acceptance, even implementation and take up across Europe. This, in addition to further stakeholder consultation and road mapping exercises, is the focus of the subsequent work packages of CRISP.
Working towards harmonisation of security certification in Europe is a complex task, given the myriad products, systems and services available, the pace of technological development and consequent social concerns and last, but not least, the differing national regulatory, legal, social and cultural aspects within the European countries. These draw our attention to how flexibility must be inherent in the CRISP scheme, to account for different technologies, means of operation, national cultures, as well as legal and regulatory frameworks in each member state. The case studies furthermore revealed that while some fields are well on their way towards a harmonised landscape (e.g. alarm systems), newer technologies, such as RPAS, are very much at the starting point of the work of setting up a system of certification across Europe.
National differences emerge as key barriers, which need to be fully understood and tackled in order to successfully work towards harmonisation, and consequently, the CRISP scheme should be flexible to the extent to include national requirements as supplementary to the overall minimum requirements stated in the scheme. The aim here would, however, be to keep national requirements to a minimum to avoid continuing fragmentation in certification.
Further to raising awareness and meeting the needs of key stakeholders, mechanisms for ensuring consistency and acceptance across Europe are key for the success of the CRISP scheme. As identified in the literature review and analysis of stakeholder roles, accreditation bodies and European cooperation for Accreditation (EA), emerge here as strong organisations to ensure a consistent accreditation and of the scheme across Europe. To ensure that the scheme complies with national regulation, collaboration between DPAs and accreditation bodies, and their respective Europe-wide bodies, could be a useful endeavour to ensure consistency and compliance, whilst avoiding fragmentation and uneven application.
This report ends with listing minimum requirements for a harmonised approach, which are derived from analysis of data derived from all research tasks. These recommendations will guide the remainder of the CRISP project as it embarks on designing and developing the scheme in accordance with stakeholder needs. The recommendations clearly outline what steps must be taken in order to develop, implement and encourage take up of the new scheme.
Following is the list of requirements for a harmonised approach:
(1) The certification system should be based on robust European or international standards. European and international standards are developed by a wide range of stakeholders and reflect the generally accepted perspective on the requirements to be set to a product, system or service. It is the role of the standardisation organisations to develop standards in such a way that they are robust and can be the basis for certification
(2) The certification schemes and the underlying requirements should be transparent and clear in what they evaluate and certify. The demand for transparency and clarity in certification was strong from all stakeholder groups. Lack of transparency for any stakeholder group will increase the likelihood that the scheme is not trusted, which will consequently lead to lack of take-up.
(3) The certification system must be accepted throughout Europe. Complete acceptance throughout Europe will contribute to the harmonization efforts of the EC and will benefit the security industry by avoiding re-certification for each national market. Working towards a pan-European acceptance will be a complex undertaking but there are some best-practice examples, such as the CertAlarm certification system for alarm systems, which is already widely accepted among European countries.
(4) Certification bodies should ensure that their evaluation and standards interpretation, is consistent throughout Europe. Consistency is of key importance as perceptions of a lack of or inconsistent test and evaluation methods in certification will negatively impact trust in the new scheme. This furthermore ensures that producers of products or services that have not met the requirements when examined by one certification body cannot get them certified by another.
(5) The certification system should be endorsed (or enforced) by regulators. The single strongest tool to ensure a European certification scheme is being accepted by stakeholders is for the European Commission to endorse it, as this will drive cross-border and stakeholder acceptance.
(6) The certification scheme should be operated under accreditation. To ensure quality of certification and trust in the certification system, it should be operated under accreditation. Whether or not a scheme is accredited is decided by the schemes owner.
(7) The certification system should provide one recognisable European seal. By using one recognisable seal, it is clear for all stakeholders that a product, system or service meets the requirements set in the standards. The certification scheme developed within CRISP will focus on requirements regarding four social dimensions: security, trust, efficiency and freedom infringement, but does not focus on technical requirements of security products. The CRISP seal should indicate that all four dimensions have been examined and adequately addressed in the evaluation.
(8) The CRISP scheme should take national specific requirements into consideration. National difference emerged as a key barrier to harmonization of certification throughout Europe. The CRISP scheme must take this into consideration to enhance uptake and acceptance in different countries in Europe. There is a possibility that these will be dealt with by writing a baseline scheme where national specific requirements only need re-certification.
(9) An appropriate implementation and awareness-raising process is necessary for the certification system to be successful. The quality of the way the certification scheme is implemented and presented is as important as the quality of the scheme itself. Stakeholders must have access to high quality and accurate information about the scheme, its benefits, criteria and evaluation process. In addition to contributing the design and development of the scheme itself, stakeholders also need to have a say in the way the system is implemented.
4 ANALYSIS OF STEFI DIMENSIONS: SECURITY, TRUST, EFFICIENCY, FREEDOM INFRINGEMENTS
4.1 Legal analysis of existing schemes
The analysis aimed to analyse the core issues of each of the STEFi dimensions and determine legal demands for security products, systems, and services. STEFi stands for Security, Trust, Efficiency and Freedom Infringement, and was an approach to evaluation of security systems developed in the FP7 project SIAM.
The study on the legal framework on evaluation and certification schemes identifies the legislation applicable and soft law setting the best practices for certification in the EU. The lack of European legislation on certification schemes is to a certain extent covered by the international and European standards and the guidance from standardisation bodies.
With regard to the legal study on security, the EU and the Member States share competence in the area of security. The security legislation in the EU is sector-specific and provides useful insights as to high-level issues for security products, systems, and services. Legal demands identified relate to physical controls and training of personnel, as well as the performance and the functioning of the security equipment. The multi-layered risks of physical and digital nature urge for accountability, security and risk assessments. Access to information systems and prevention from illegal interception and interference to the data and systems are also significant requirements incorporated to the EU legislation.
In terms of standardisation and certification, trust means that the security product, system, or service, among others, complies with the legislation and respects fundamental rights, is technically reliable, efficient and transparent, and responds in a predictable and acceptable manner. The role of evaluation and certification schemes in enhancing the needs of the citizens can be crucial when it involves stakeholders, is reviewed regularly, is operated by an independent body and includes requirements that address the main concerns of the scrutinised in the auditing or evaluation procedure.
Looking at efficiency, the case studies of this Work Package, reveal significant challenges as to how a security PSS should operate and perform in order to balance investments and achieve the objective of security. Energy efficiency and adaptability to new technologies with the minimum cost possible are elements relevant to efficiency for security PSS from a legal perspective.
Freedoms and fundamental rights of individuals are potentially at risk from the establishment and/or operation of security products, systems, and services, in different environments by entities such as public authorities, private legal persons, or individuals. The impacts of drones and CCTV systems in public spaces for crime prevention and detention to the right to privacy and data protection may range from extended surveillance, panoptic effect, profiling, to excessive collection of personal data, lack of notification to data subjects, lack of possibility to exercise data subject rights and others. Accordingly, biometric alarm systems pose risks to the rights to data protection and privacy, as well as others. Equal treatment and prohibition of discrimination, bodily integrity, presumption of innocence, due process and fair trial might be infringed by the security measures, either from the functionalities of the equipment itself or the use/abuse of the security product. The study identifies core requirements for security PSS to respect the rights and freedoms and be in line with the protective framework.
Further, the analysis examines to what extent the existing evaluation and certification schemes incorporate and address issues identified in the legal study of the Report. The analysis of existing schemes, showed that the type of entity operating the scheme and the type of the scheme play a crucial role into determining the quantitative and qualitative integration of the STEFi requirements. Public authorities tend to prioritise trust and freedom infringement requirements, while certification and standardisation bodies, as well as the industry focus on security and safety requirements.
The impact of the STEFi dimensions to the function and performance, user acceptability and legal compliance of the security PSS and the difficulty to identify a scheme with a comprehensive approach to the core aspects of all the four dimensions, highlighted the importance of the CRISP objective to develop an innovative evaluation methodology that integrates the security, trust, efficiency and freedom infringement assessment dimensions.
3.4.2 STEFi based SWOT analysis of existing schemes
The STEFi based SWOT (strengths-weaknesses-opportunities-threats) analysis of existing schemes aimed to: first to contribute to identification of core evaluation subjects for evaluation and certification schemes of security products, systems and services, based on the four dimensions of the STEFi model; second, to examine and analyse the strengths and weaknesses, opportunities and threats of existing schemes in order to further use, enhance and develop evaluation and certification schemes for the assessment and certification of products, systems and services used for physical security of people and infrastructures.
The SWOT model of analysis and the STEFi approach are employed for the analysis of the existing evaluation and certification schemes. In order to further identify high-level criteria based on STEFi, a showcase on a security problem is presented with selected potential security solutions such as CCTV, police patrols, community policing and others. The DESSI tool for supporting security investment decisions and its multi-dimensional criteria matrix embracing political, legal, ethical, social, economic and societal aspects is used for the security problem.
Further, a STEFi based analysis of CCTV, alarm systems and drones underlines and elaborates on the core issues of the three selected cases. Issues such as false alarm rates, performance and interoperability are discussed for security dimension; reliability, safety and transparency for the trust dimension; deployment and lifecycle costs for efficiency dimension and lack of awareness, big data, social sorting and discrimination for the freedom infringements dimension. The outcome of the case studies is suggestions for STEFi criteria.
Following the analysis of the three cases studies and the showcase from the DESSI project, the consortium identified ‘STEFi criteria for the evaluation of security products, systems and services’, which aimed to serve as a first basis for the analysis of existing schemes. In addition, the STEFi criteria, as high level evaluation subjects derived from the investigation phase of the project, aimed to facilitate the rest of the project. The list of STEFi criteria concentrates the main topics discussed in the deliverables into key topics (“evaluation subjects” according to the CRISPDoW) and provides an overview of issues raised in this analysis.
The analysis of selected existing schemes in the area of CCTV systems, drones, and alarm systems, identified shortcomings and best practices, along with opportunities for improvement for the purposes of the CRISP project. With regard to the shortcomings, most of the analysed schemes did not consider any freedoms and human rights aspects, nor the impact of the security measure on such rights. Efficiency is also an underrepresented aspect in the analysed schemes. In relation to the way the evaluation and certification process in general are organised, the research team identified issues of fragmented rules, that are spread in several documents, which in turn might lead to confusion, uncertainty, and mistrust to the scheme. In addition, mistrust might be generated by lack of transparency regarding the validity period of the certification and the limited availability of the scheme documentation. As the presentation of the SWOT analysis has shown, several best practice examples could be identified within each STEFi dimension. However, while singular criteria are highlighted by some schemes in one dimension, they are undervalued or are not considered at all in others. In terms of security, the criteria ‘Risk management’ was in more than 60 % of the analysed schemes fully met. The trust evaluation showed that the majority of the schemes might have reference to some STEFi trust criteria but they do not fully satisfy them. Examples are the criteria “transparency”, “observability” and “safety”. Efficiency aspects of security PSS are also not well covered. “Usability” and “energy efficiency” are therefore among the recommendations for improvement of the existing schemes. Last but not least, the freedom infringements evaluation had similar results with a few best practice exceptions. “Privacy” and “bodily integrity” are usually overlooked, whereas “personal data” attributes are only partly considered. For each criterion, and sometimes for each attribute, the analysis underlined schemes that can be considered as best practice according to STEFi, but as a general conclusion there is substantial gap in meeting the STEFi criteria.
The report concludes with recommendations based on the opportunities for improvement that were identified in the analysis of the existing schemes:
(1) Open and transparent rules, scope and processes
(2) Strong monitoring mechanisms to supervise the compliance of the PSS with the certification rules and its normative references.
(3) Accountability mechanisms
(4) Reliable normative references, such as international or European standards
(5) Governance which involves a standardisation body and/or a public authority
(6) Multinational participation in the development process of the scheme
(7) Differentiation of testing and evaluation levels for different security functions or required performance (granularity)
(8) Thorough rules on documentation to ensure accuracy and openness to the interested parties
(9) Publication of the revoked and expired certificates
5 DEVELOPMENT AND TESTING OF THE CRISP METHODOLOGY
5.1 Development of the CRISP methodology
The CRISP project followed a tailored strategy and involved stakeholder feedback at different stages in the project in order to foster greater acceptance and usability of the developed methodology. In the case of the validation and refinement of the CRISP methodology, efforts has been undertaken to ensure continuous stakeholder involvement in the development of the methodology, by means of holding interviews with experts from certification bodies, conducting a validation workshop to present and discuss the first draft of the methodology with a large group of stakeholders, and regularly consulting the advisory board of CRISP.
CRISP’s approach is composed of:
(1) Evaluation consisting of two main stages: configuration and STEFi assessment.
(2) Certification as third-party attestation related to products, processes, systems or persons and consisting of two stages: audit and attestation, which consider the results of the evaluation part.
In order to address significant gaps in the current certification landscape, the methodology claims to imply two important aspects which have been hardly or only scarcely considered in previous evaluation and certification activities: First, it includes social criteria for the evaluation and certification of security PSS. And second, it integrates the complexity of different assessment dimensions in one approach by enabling and encouraging an early participation of diverse stakeholders representing both the supply and demand side of security PSS.
The evaluation part of the CRISP methodology is participatory, systematic, and iterative by nature enabling the determination, selection and assessment of security PSS according to the four STEFi dimensions. It is participatory due to the encouragement of an (early) involvement of different stakeholders. It is systemic as a variety of differently dimensional criteria will be brought into a matrix structure. And it is iterative as the evaluation process is repeated until each potential conflict uncovered is addressed to relevant / involved stakeholders and, where appropriate solved.
Regardless of the issue that there might be security PSS that do not need to go through all proposed STEFi dimensions, the overall benefit of the participatory approach is about increasing the level of inter-subjectivity, releasing stakeholders “from their usual self-assured methods of simplification and [confronting] them with the complexity and the effects of their [previous] decisions outside their professional or common view”. This fact might seem to conflict with standardisations’ logic, which seeks to reduce complexity. Consequently, the challenge is to test how the theoretical complex appearing part of the CRISP approach fits in with the complex world of certification and standardisation.
Innovative aspects of the presented methodology are: the evaluation part based on STEFi which is designed to allow appropriate improvements/measures of a security PSS at different development stages which can act as incentives for certification as it allows a product, system or service optimisation in the first place. Consequently, a ‘project leader’ (e.g. manufacturer) should be motivated to start the STEFi assessment not only when it comes to attestation. Especially when it comes to a newly developed/designed security measure, an evaluation of a security PSS based on STEFi might help to reduce costs in the long run.
5.2 Testing and refinement of the CRISP methodology
Once the CRISP methodology was sufficiently developed, four scenario-based workshops (on border control system, on drones, on CCTV & alarm systems, and on housing areas) were organised to test and further refine the CRISP methodology. The workshops aimed at ensuring a strong stakeholder involvement and feedback towards the developed methodology from both the supply and demand side of security system, as well as other relevant stakeholders. Based on the outcome of the workshops, the CRISP methodology was refined. The following two important adjustments to the methodology resulted from the workshops:
- The conformity assessment function of ‘inspection’ (i.e. on-site assessment) was added to the function of “auditing”
During the feedback discussions, it became obvious that just to assess the provided documentation will not be enough to attest a security system and should be extended in the on-site assessment. This on-site assessment is necessary to verify that some statements from the Overall Evaluation Report conform to the requirements of the CRISP certification. Based on the ISO/IEC 17000:2014, this function is performed by inspection, defined as examination of a product design, product, process or installation and determination of its conformity with specific requirements or on the basis of professional judgment, with general requirements.
- The conformity assessment function of ‘surveillance’ was added
The decision to exclude surveillance as a standard function as it is not always performed within the certification process was made. However, the results of the scenario-based workshops highlighted the need for surveillance in order to control any changes in a security system and to maintain the validity of the CRISP certification. Therefore, the function “surveillance” was added to the certification part of the CRISP methodology.
The CRISP methodology was particularly refined on the certification part, resulting in three stages for the certification part.
During the evaluation part, an assessment of security PSS is to be carried out, according to the requirements based on the evaluation criteria as defined. Some of the evaluation criteria have been identified based on the current European normative documents, such as legislation and European standards. Other evaluation criteria have been chosen on the basis of relevant literature sources and research project outcomes. All identified evaluation criteria are used to formulate the requirements in the CEN Workshop Agreement (CWA).
Certification is based on requirements. These requirements are a combination of general requirements used in certification and requirements that specifically apply for the CRISP approach. The general requirements are drawn from the ISO/IEC conformity assessment standards, while the specific requirements are defined based the scenario-based workshops.
Finally, all of this work provided input for the certification manual and the development of the CWA (see next sections), where the refined CRISP methodology has been further worked out.
6 CERTIFICATION MANUAL
The certification manual provides basic information directed to the CRISP organisation, the future owner of the CRISP certification scheme, for being able to complement the CRISP certification scheme based on the STEFi approach. However, all parties involved can benefit from reading and using the certification manual by finding information on (basic) requirements for evaluation and certification in the context of a concrete system category (installed video surveillance systems) as well as clarifications on roles and responsibilities of all parties involved in the CRISP context.
A short overview of the information in the certification manual is provided next.
The evaluation process
An interested person needs to apply for CRISP certification at a certification body who is authorised by the future CRISP organisation. Once the client has applied for evaluation and certification, they will be delivered with a list of the stakeholders involved in the certification and evaluation processes. The client will be asked for cooperation with stakeholders involved in the certification and evaluation processes. The client shall provide the certification and evaluation body with all required information for the assessment of the system to be certified. In this context, the client must be capable of providing the system’s objectives as well as general and specific information. If this is not the case, the client should be able to consult experts (information provider), either from its own company or third parties, who are familiar with the system. Apart from the general requirements detailed above, the client shall adhere to the conditions that guarantee a fair, transparent and efficient procedure.
- The analysis conducted - configuration and assessment: General information on the security system shall be provided by the client in the first stage of configuration. The configuration results, functioning as input for the assessment stage and as an information source for the STEFi experts when they are invited to the assessment of the security system later on. The assessment stage follows the configuration stage. During this stage, a security system is evaluated by using an assessment questionnaire taking into account the four STEFi dimensions. ‘Evidence’ must be provided by the client and by the experts in order to make their decisions/answers evidence-based. An evaluation body shall ensure that all parts of the evaluation criteria questionnaire are addressed not only by the client or only one STEFi expert, but by both actor roles in order to identify any differences in opinions and (expert’s) views.
- The results derived from the analysis and conclusions to prepare for certification: Once the evaluation criteria are all answered, the assessment stage will be closed by the evaluation body. In a next step, the configuration and assessment information needs to be summarised and analysed in an evaluation report. CRISP evaluation does not only serve the purpose of gathering information on a specific use case of a security system which is more or less obvious, but also to identify and raise awareness for interrelations, revealing potential conflicts within and between STEFi criteria.
The STEFi approach integrate complex and multidimensional relationships into one approach; it does not mutually exclude the single perspectives of security, trust, efficiency, and freedoms, but rather unites them in a systematic and systemic way. Consequently, the merging of the criteria per dimension allows to detect potential conflicts between criteria and dimensions.
The evaluation body shall monitor the implementation and the integrity of STEFi assessment and coordinate the work with STEFi experts, which need to act independently and impartially. The evaluation body shall receive regular reports on the activities of the STEFi experts and establish supervision mechanisms, which allow STEFi experts to report any observations or irregularities they detect in the course of their work in an anonymous or confidential way. Once the evaluation phase is closed, the results derived from the analysis and the conclusions drawn will be forwarded to the certification body by the evaluation body. The evaluation reports shall serve as the basis for a third-party review, decision and attestation.
The certification process
The purpose of the certification phase is to verify whether the evaluation phase has been conducted according to the applicable requirements and whether the STEFi criteria have been met sufficiently. The certificate will only be granted after the confirmation of affirmative results of the evaluation.
The certification body shall check the evaluation reports with respect to completeness, reliability and conformance with applicable requirements (as specified in the future CRISP certification scheme). The extent to which the requirements of the STEFi criteria are fulfilled per dimension should be assessed by the auditor taking into account the context (situation/scenario) in which the security systems are used.
The review, decision and overall attestation process cannot be outsourced but needs to be conducted by the accredited certification body who is contracted by the client. However, it is necessary that the certification body outsources their evaluation activities, and only to organisations that meet the requirements as specified in the future CRISP certification scheme. The certification body shall have a legally binding contract with the body providing the outsourced services, including provisions for confidentiality and conflict of interest.
7 CEN WORKSHOP AGREEMENT
A CEN Workshop Agreement (CWA) is an agreement developed and approved in a CEN workshop, which is open to the direct participation of anyone with an interest in the development of the arrangement. The development of a CWA is fast and flexible, on average one year, and the document called a CWA is published by CEN.
CRISP developed a CWA titled “Guidelines for the evaluation process of installed security systems, based on the STEFi criteria”, which presents the innovative approach developed in the CRISP project: the methodology to assess systems from the perspective of four different, though interrelated dimensions. These dimensions are referred to as the STEFi dimensions (Security, Trust, Efficiency and Freedom infringement) and the methodology integrates these in its evaluation phase. The methodology acknowledges and addresses the complexity of assessing security systems by identifying potential conflicts between the various assessment dimensions and related criteria and by providing an approach to resolve these conflicts in specific situations. The methodology does not single out technical, legal, social or economic aspects, but integrates these in a multidimensional and multi-stakeholder assessment.
Efforts have been undertaken to present the evaluation methodology as clear as possible, including examples of evaluation questions and requirements for further understanding on the application of the evaluation. This allows the CWA to be used and tested in specific contexts in which security systems are implemented. The CWA is considered as a useful first concept of a checklist which can be further refined by the testing results. Moreover, the CWA stimulates awareness amongst end-users to consider the four STEFi dimensions in the field of security systems.
The CWA may be used as a standardization resource in the development of the certification scheme. Third-party certification should mostly appeal to large companies and installations. However, for smaller companies, such as SMEs, that may not be interested to go all the way to have a certified system, the CWA can still be useful when regarding as a tool for self-assessment, by self-evaluating the compliance of an installed security system according to the STEFi dimensions. Security systems in development cannot be certified, but the CWA may again be useful in this phase, allowing for evaluation of the system in development and consequently improving the design of a system.
The CEN Workshop Agreement (CWA) is published as CRISP deliverable, but will also be published as an official CEN-CENELEC standardisation document (expected publication date: May 2017). The CWA will then be available on the CEN-CENELEC website as well as via the members of CEN-CENELEC, i.e. the national standardisation bodies. Arrangement to make the CWA freely available has been taken care of, allowing any interested stakeholder to consult this document, spreading the knowledge beyond the project lifetime and via other sources.
The CEN-CENELEC procedures apply to the CWA, which consequently means that the document will be reviewed after three years. The review may result in: to keep the CWA as it is for another three years if the CWA is still up to date; to withdraw the document in case the CWA does is not valid anymore; or to revise/update the document if new insights have been collected and/or changes occurred in the context.
8 ROADMAP AND IMPLEMENTATION PLAN
Roadmaps are used to strategically plan and describe the steps to achieve outlined outcomes and goals. The CRISP roadmap is modelled on the basis of guidance from the IEA and thus includes the sections goals; milestones, priorities and a timeline; barriers and gaps; and implementation plan and action items. Furthermore, the roadmap includes an outlook chapter to visualise the future for the CRISP scheme.
At the beginning of its work on the roadmap, the CRISP consortium specified its mission, which was already presented at the beginning of this report. Based on this mission, the overall goal of the CRISP project was the production of an innovative assessment procedure, which leads to the certification of security technologies. It aimed to include social/societal aspects as key elements in the evaluation and certification methodology, which results in a future pan-European certification scheme and fosters harmonisation of the European security market. On this basis, the following sub-goals were defined:
(a) Develop a high-quality scheme and trustworthy CRISP certificates
(b) Develop a cost-effective CRISP scheme
(c) Achieve synergies with regulatory demands
(d) Become recognized and accepted by certification bodies
(e) Achieve backing-up by insurance companies
(f) Start gaining acceptance and penetration in appropriate regions
Based on a market analysis, the CRISP scheme shall be piloted within the area of video surveillance systems as a starting point. In total, CRISP’s roadmap includes 14 steps:
(1) Duration of the CRISP project
(2) Development of the CWA
(3) Building interest among stakeholders to establish a CRISP organisation
(4) Establishment CRISP organisation
(5) Completion CRISP’s pilot scheme
(6) Development Evaluation Tool
(7) Building interest among certification bodies to offer pilot and final scheme
(8) Pilot phase
(9) Building of close relationship with certification bodies, regulators, policy makers
(10) Marketing campaign
(11) Development of CRISP standards
(12) Development of the extended scheme and preparation of its accreditation
(13) Accreditation of the CRISP scheme and certification bodies
(14) Certification based on extended scheme
Furthermore, the CRISP consortium defined twelve milestones in three stages to represent interim performance targets for achieving the goals, assigned to specific time periods on the CRISP timeline that started in 2016 and is extended to beyond 2028:
Stage 1:
• M1: Completion of CRISP‘s CWA
• M2: Establishment of the CRISP organisation
• M3: Completion of a pilot scheme for video surveillance systems based on CRISP’s CWA
Stage 2:
• M4: Issuing the first pilot certificate
• M5: Early adoption of the CRISP pilot scheme for video surveillance by ten certification bodies from EU Member States
Stage 3:
• M6: Finalisation of the CRISP standards
• M7: Completion of the CRISP scheme extended to other security systems
• M8: Confirmation of the accredibility of the CRISP scheme
• M9: Accreditation of the CRISP scheme
• M10: First accreditation of certification bodies, which want to provide CRISP certification
• M11: Issuing of the first final CRISP certificate
• M12: Market penetration by 10% in Europe at least in the area of video-surveillance systems
The first stage includes three milestones. The CRISP project finished with the CWA in March 2017 (M1). The next milestone will be the specification of the members of the CRISP organisation (M2). The final milestone of the first stage (M3) refers to the CRISP organisation completing the pilot scheme based on the CWA and the innovative CRISP methodology. In parallel with this development , it is planned to develop an evaluation tool which shall be also completed at M3. The three milestones also represent the targets that show the process towards achieving the sub-goals a to c. (The preparation of M7, the completion of the final CRISP scheme will also build on these principles.)
The next “piloting” stage, which includes the issuing of the first certificate in the area of video surveillance systems (M4), relates to the goal of recognition and acceptance of the CRISP scheme by certification bodies (sub-goal d) and to the sub-goal of gaining early adopters and penetration (sub-goal f). M5 refers to the early adoption of the CRISP pilot scheme for video surveillance systems by ten certification bodies from EU Member States and serves as a target of achieving the above mentioned sub-goal c, as well as achieve backing up by insurance companies (sub-goal e). Success in the piloting phase will be shown by the market share of the CRISP scheme in the pilot area in the following years.
The last stage represents milestones related to the further development and extension of the CRISP scheme to be offered to other security systems rather than being limited to the pilot market of video surveillance systems. The milestones represent targets in relation to all of CRISP’s goals, as in the previous two stages, only now addressing the development and uptake of the extended CRISP scheme, available for other security systems and based on developed CRISP standards.
It is expected that CRISP standards will be developed by 2023 (M6), originating from the CWA developed as part of the project. Standardisation activities take time and therefore this process will run for five years in parallel with the pilot testing. M7 concerns the completion of the CRISP scheme extended to other security systems, whereas the following two milestones represent the accreditation of the CRISP scheme (M9) after receiving confirmation of the accredibility of the CRISP scheme (M8). The next two milestones refer to the accreditation of the first certification body wishing to offer CRISP certification and the issuing of the first CRISP certificate based on the final scheme by one of these organisations. The last milestone (M12) signals that the CRISP scheme reached the desired market share of 10%, at least in the area of video surveillance systems.
The roadmap extends beyond the duration of the CRISP project and outlines the role of a CRISP organisation that will take over the further development and implementation the CRISP scheme. The milestones are thus split between the CRISP consortium, the future CRISP organisation and other parties, which want to offer CRISP certification in the future. The milestone of the former was the completion of the CWA, which is already realized. The milestones assigned to the CRISP organisation consider, for example, the introduction of the CRISP scheme, the facilitation of pilot activities and the finalisation the CRISP standard(s) while the accreditation as well as the issuing of CRISP’s first pilot and final certificates build on combined efforts of the future CRISP organisation and the future providers of the relevant certificates.
For the CRISP scheme to succeed, external stakeholders are highly relevant, especially for consultation purposes, as well as promotion and support of the new scheme. The roadmap defines these groups as well as their roles and responsibilities for specific milestones and actions. It is clear that, for the purposes of building a strong and accepted scheme, the security industry, regulators, DPAs, certification, standardisation and accreditation bodies are identified as crucial in gaining broad acceptance and ensuring the take-up of the CRISP scheme by demonstrating support, offering and seeking certification and participating in standards development. The European Commission and Member States will also play an important role in facilitating the legal and regulatory framework, within which the CRISP scheme will operate.
The CRISP consortium realised the importance of acknowledging potential barriers, such as public acceptance, market structural barriers, regulatory limitations, technology limitations, gaps in knowledge or other barriers, that can potentially hinder the achievement of the goals and milestones. The barriers and gaps were extensively discussed with stakeholders and are addressed.
In addition to a roadmap, an implementation plan and a list of defined action items was also developed, so that the implementation of the CRISP scheme can be robustly planned. The key elements of the CRISP implementation plan are based upon the CRISP organisation that will take on the management of the CRISP scheme, to whom this roadmap is in fact directed to. The CRISP consortium has prepared selection criteria for such a CRISP organisation. Furthermore, the implementation plan identifies the resources needed by the chosen organisation to implement the CRISP scheme and action items, including securing financial and human resources to ensure adequate conduct, education, promotion, risk management and further development of the CRISP scheme. Action items for external stakeholders are also identified and arranged in order of priority and by stakeholder groups. Action items for the CRISP organisation include:
• Seek early engagement of insurance companies, consultants, installers to ensure their interest in the CRISP scheme
• Ensure early engagement of the public and other stakeholders, such as end users, manufacturers, data protection agencies, etc.
• Complete the CRISP scheme appropriately by ensuring that it provides significant added value for the end users
• Establish communication with stakeholders for appropriate standardisation activities at EU level
• Establish appropriate communication with experts on European and Member States’ regulation and legislation to identify best options for the characteristics of the scheme and to update the results of the CRISP project
• Ensure early engagement of regulators to ensure an appropriate development and refinement of the requirements of the scheme as well as greater promotion and acceptance of the CRISP scheme
• Negotiate the use of the CRISP scheme and mutual recognition agreements with European certification bodies
• Conduct activities to sign licensing agreements with certification bodies
• Conduct an appropriate marketing campaign to identify and highlight the added value of the CRISP scheme for end users and describe it properly to different markets of end users )
• Create an appropriate training programme for evaluation bodies, certification bodies etc.
Action items for external stakeholders to support the CRISP scheme are:
For the European Commission:
• Publically advocate for and support the CRISP scheme and its uptake in the EU Member States to facilitate positive framework conditions for the emergence of the CRISP organisation
• Develop a proposal for legislation regarding security certification with minimum requirements (a proposal for legislation)
For the industry/industry representatives, associations
• Partake in CRISP’s standards development
• Support the formation of the scheme to ensure that it is fit-for-purpose
For European standards bodies
• Develop (a) standard(s) based on CRISP’s CWA
• National bodies: agree on starting such a (standardisation) project, to work on the common document, seek agreement on its content later
• Members of relevant TCs and WGs: check the need for additional standards in the CRISP context and initiate relevant activities where necessary
For European certification bodies
• Promote the CRISP scheme to the customers
• Undertake training in assessment and evaluation methods for the CRISP scheme
• Participate in the further development of the CRISP scheme
• Partake in CRISP’s standards development
• Get CRISP accreditation after the completion of the relevant standard(s)
For European accreditation bodies
• Monitor that there are no national differences in CRISP-related certification
For Regulators
• Oversee the CRISP certification so that it complies with legislation and regulation to the highest possible degree
• Participate in the development of the CWA and the CRISP standards
For Member States
• Conduct market surveillance, ensure conformity, and inform the public
• Examine and identify whether the scheme posts any challenges to the national regulatory/legislative framework and care for appropriate solutions, if/where necessary
9 ENHANCING CONFIDENCE AND ACCEPTABILITY IN THE NEW CERTIFICATION MEASURES
Dedicated efforts have been made in enhancing confidence and acceptance of the proposed work amongst the key stakeholders.
First, efforts were undertaken to build confidence and enhance acceptance of the new certification measures, specifically in an EU-wide security standards and certification process. The CRISP consortium reached out to different stakeholder groups, presenting the CRISP project output and engaging them vigorously to provide feedback which was used to improve the results. Here the efforts made to engage and gather input from the regulators and certification bodies, together with the manufacturers and the experts involved in the relevant European standardisation working groups, are the most important since they are a crucial part of the EU wide standardization and certification processes. Additionally, , the project invested many efforts to engage stakeholders in the CWA development process as the CWA is one of the main outputs of the project, by itself leverages EU wide acceptance of CRISP methodology.
Secondly, measures were promoted to ensure the success of the EU-wide security standards and certification scheme. This has been addressed by extensively communicating to all the stakeholders the benefits of the CRISP innovative approach to holistic evaluation certification of security systems and the pan-European approach of the measures. Engagement of the stakeholders on as many occasions possible, by presentations, face-to-face communications and events, and inclusion of feedback in the development of the building blocks of the CRISP approach can also be seen as an excellent measure to ensure that their views and expectations are well built into the proposed solutions, thereby ensuring acceptance and success of the solutions that were designed in collaboration.
Furthermore, key stakeholders were engaged and involved in the process of EU-wide security standards and certification scheme making. The consortium actively approached the different stakeholders to get them acquainted with the CRISP project and to get them involved in the finalization of the outputs, in their local languages, by the local partners that have established reputation in their respective field. A direct link can be made here to the process of the CWA development, where the consortium used all the activities related to the promotion of the CRISP approach in specific stakeholder groups, to also draw them into the standardization process of the CWA development. The consortium was successful in this task, since it managed to bring around the table a diverse group of experts representing organizations from all stakeholder groups coming form 10 EU Member States, which give further credibility and value to the CWA output.
Finally, advocates among the key stakeholders were identified that could facilitate gaining confidence and enhance acceptance of the new evaluation and certification measures. All partners have developed excellent relationships with the respective experts, by informing them about the activities and the progress, meeting with them, involving them in the CWA process and gathering their input, as well as by inviting them to the events and activities of the project. Project partners have identified the advocates among stakeholders that represent some of the most important market players in the EU, the relevant EU and international working groups in standardization, consumer organizations, regulators and certification bodies, and they also kept a close tie with them for further promotion and dissemination of the CRISP project results.
Strong engagement with different stakeholders was achieved; partners have efficiently used resources to reach the stakeholders in a great number of EU Members States, resulting in close to perfect outreach to all EU Member States at the level of all activities, also making efforts to reach the smaller ones.
The CRISP partners prepared four briefing papers, which were aimed at presenting the CRISP project, approach and methodology to different stakeholder groups:
• For regulators
• For manufacturers
• For certification bodies
• For the public and consumers
The briefing papers aimed to enhance confidence in the CRISP evaluation methodology and raising awareness specifically within the above stakeholder groups. The approach included targeted communication to organizations and individuals (phone calls and emails) across Europe. The briefing papers were emailed to specific stakeholders (a mailing list consisted of 138 specifically selected stakeholders and organisations) and advertised on twitter and the project website.
This targeted communication included emails and telephone calls resulted in strong ties between the CRISP project and partners from 7 consumer organizations in different European countries, extensive communication with 62 individuals within the stakeholder group regulators and policy makers, and established a close working relationship with manufacturers from 8 EU member states.
More on dissemination activities of the project in general can be found in the next section.
Potential Impact:
EXPLOITATION RESULTS
The CRISP project mission was to develop an innovative evaluation and certification methodology for the CRISP certification scheme for security systems. The exploitation results consist of the following building blocks for the CRISP certification scheme:
• CRISP methodology (D5.1 D5.2)
• CEN Workshop Agreement, where the evaluation methodology is recorded, exemplified with evaluation criteria (D7.2)
• CRISP certification manual and roadmap (D6.1 and D6.2)
• CRISP exploitation plan (D7.4)
• Briefing papers on the proposed certification scheme and roadmap:
o For regulators
o For manufacturers
o For certification bodies
o For the public and consumers
These building blocks will allow the future scheme owner to take up and further develop this, finalising the development of the CRISP certification scheme and operationalize it.
POTENTIAL IMPACT
The major potential impacts of the CRISP project are:
• Enhancing the knowledge base of evaluation and certification of security systems
• Stimulate awareness amongst end-users in social, economic and legal aspects in evaluation and certification of security systems
• Providing a level playing field for the security industry in Europe
• Enhancing the competiveness of the European security industry
• Enhancing trust of stakeholders and especially end-users and citizens in certification of security systems
• Fostering dialogue on security certification at all levels
• Facilitating implementation of relevant EU legislation related to security systems, in particular the data protection regulation
These potential impacts are briefly addressed in the following paragraphs.
• Enhancing the knowledge base of evaluation and certification of security systems
The project is highly interdisciplinary and draws upon expertise in the diverse disciplines and security domains. As the integration of social and legal aspects in the methodology is very innovative, the urgent need to involve and engage with stakeholders from different categories was recognised, and this has been incorporated throughout the project.
The early involvement of and frequent interaction with stakeholders (including an Advisory Board) also had a positive influence on the quality of the research work as well as the support for the work, as feedback and review from the stakeholders was included throughout the project.
In the CRISP project an innovative approach to evaluation and certification of security systems was developed. The most innovative part of the methodology is the assessment of systems from the perspective of four different, though interrelated dimensions. These dimensions are referred to as the STEFi dimensions (Security, Trust, Efficiency and Freedom infringement) and the methodology integrates these in its evaluation phase. This is an innovative approach as certification has, to date, primarily focused on the evaluation of technical requirements for security systems (the security dimension) or singled out other relevant dimensions (e.g. privacy or data protection in the freedom infringement dimension). The methodology described in this CWA, however, is not (over)simplifying the complexity of assessing security systems but acknowledges and addresses this complexity by identifying potential conflicts between the various assessment dimensions and related criteria and by providing an approach to resolve these conflicts in specific situations. The methodology does not single out technical, legal, social or economic aspects, but integrates these in a multidimensional and multi-stakeholder assessment.
The project developed collective capability, by consolidating the knowledge and expertise on security standardisation and certification. The research results are published, widely disseminated, and publicly available.
• Stimulate awareness amongst end-users in social, economic and legal aspects in evaluation and certification of security systems
Advancing the state of the art in security standardisation and certification, the project offers a methodology that not only takes into consideration the technical aspects, but encourages the inclusion of social, economic and legal aspects in a multi-dimensional and multi-stakeholder environment. This innovative approach on evaluating security systems should provide guidance to stakeholders that are interested in the inclusion of all of these dimensions in design and operation of security systems.
Next to using the CWA for certification, the CWA can also be regarded as a tool for self-assessment, by self-evaluating the compliance of an installed security system according to the STEFi dimensions. For security systems in development, the CWA may be useful in evaluating the system in development and consequently improving the design of the system. The CWA may stimulate awareness amongst end-users to consider the four STEFi dimensions in the field of security systems.
• Providing a level playing field for the security industry in Europe
The current certification market for security systems is highly fragmented as different requirements are set at the national levels and a variety of different certification schemes is applied. The CRISP approach is complementary to the already existing standards and certification schemes, not competing with them. The results of the project will facilitate a more harmonised playing field for the European security industry by providing the basis for a pan-European certification scheme for security systems. The future scheme is intended to be accepted across Europe, which will also enhance competitiveness by reducing commercialisation costs for the industry.
• Enhancing the competitiveness of the European security industry
The European security industry is an important industrial sector, however, due to a fragmented approach by differing national legal requirements and governmental policies and still a lack of European standards for a range of components and systems, the sector is not as competitive world-wide as could be. The results of the CRISP project will contribute to more cost-effective operations for the security industry by opening the European market.
• Enhancing trust of stakeholders and especially end-users and citizens in certification of security systems
The results of the CRISP will enhance trust of stakeholders in certification of security systems because in addition to technical requirements that ensure that a sufficient level of security is achieved also requirements related to societal concerns such as safety, cost efficiency and freedom infringement are taken into account in the evaluation and certification of security systems.
• Fostering dialogue on security certification at all levels
Since security evaluation and certification is an iterative process, CRISP fostered dialogue on security certification at all levels, particularly amongst:
- The stakeholders that can opt for CRISP evaluation/certification: security industry, installers and end-users of security systems, also via industry associations
- The certification bodies, i.e. providers of certification
- The regulators, who gain from CRISP evaluation/certification efforts of contributing to compliance with regulatory demands
- The public and consumers, as the ones being scrutinized by security systems and having interest in safeguarding their rights during surveillance
Not only workshops addressing all stakeholders, but also some dedicated workshops for specific stakeholder groups have been organised. Furthermore, briefing papers for each of the above-mentioned stakeholder group were developed. In the briefing papers special attention was given to build confidence and enhance acceptance of the proposed scheme, in order to stimulate interest in the future steps after the project.
As the certification scheme will need to be further developed and finalised and this is out of the scope of the CRISP project, documentation is included to guide the future CRISP organisation/scheme owner in taking the next steps. Next to the roadmap and implementation plan, the certification manual and exploitation plan are specifically addressed to the future scheme owner.
The value of the future certification scheme for the four main stakeholder categories is voiced and validated with the data protection/regulatory demands as main driver for security industry/ installers of systems and certification bodies.
• Facilitating implementation of relevant EU legislation related to security systems, in particular the data protection regulation
The CRISP methodology supports compliance with European legislation: the CRISP scheme will contribute to and promote compliance with relevant legislation, in particular with the General Data Protection Regulation provisions, and consumer protection legislation, as well as with relevant technical standards. By employing a multi-dimensional approach it will offer insight to the regulators on dimensions that are not in their primary focus, but nevertheless significantly impact the security solution in question. A trustworthy certification, issued by appropriately accredited certification bodies, also brings direct benefits to supervisory bodies in their supervisory procedures, as it is possible for them to inspect the documentation behind the issued certification which aids effectiveness of their procedures.
MAIN DISSEMINATION ACTIVITIES
Specific activities focused on enhancing confidence in the new certification measures were undertaken, and within this remit different stakeholder groups were approached with targeted information to promote and boost interest in the CRISP methodology. This has been described as part of the S&T results. This section focuses on the dissemination and communication of the project and project results. Overall, the project included stakeholder contact in some form, through research and validation activities, e.g. interviews, surveys, and workshops, through which the profile of the CRISP project was raised and its findings promoted.
The CRISP consortium successfully promoted the project, findings and events throughout the three years and awareness amongst key stakeholder groups was high. Due to the innovative approach to certification, stakeholders expressed a keen interest in the project findings, and in the future of the CRISP methodology. This was especially apparent during the final conference where questions about the future CRISP scheme were frequent.
The project dissemination strategy included a variety of approaches and tools that partners used in conjunction to reach as many relevant audiences as possible. This included publications and attendance at third party events, website, social media, electronic newsletters and CRISP led events, such as workshops, roundtable event and the CRISP final conference.
Website and social media
The CRISP website at http://crispproject.eu was set up at the start of the project. The website holds all information on the project, the project consortium, related projects and all public project deliverables. The project partners also used the website to communicate information about project events and new deliverables. The project website will be maintained for a year after the project ends, to ensure that all materials continue to be available to those who wish to use the project deliverables for further research or innovation. All project deliverables will be uploaded to Zenodo to ensure that they are available for future use, after the website is taken down.
Traffic to the website was tracked by using Google Analytics. Throughout the project there have been over 11 thousand visits to the website from just under 9 thousand individual users. In total 1,493 downloads have were made from the website during the project. The deliverables on the overview of standards and certification in Europe and internationally have consistently been at the top of downloads throughout the project, demonstrating that this knowledge is very relevant and useful to stakeholders working within the field of security certification. In April 2015 the CRISP twitter account (@CRISP_project) was set up to communicate project news, events and deliverables could be further communicated. At the end of the project the twitter account has a very good following of 155 followers and has tweeted 313 times. A Slideshare account for the CRISP project was also set up through which to disseminate slides from external and internal events. The Slideshare enables greater sharing and promotion of CRISP findings in short and easily digestible format. The CRISP Slideshare account now hosts 15 slide presentations from project events, third party conferences and from the CRISP final conference. For the final year of the project (March 2016 - April 2017) slides from the account have been viewed 721 times in total and downloaded 61 times.
The project has a very strong contact list of 380 contacts that the partners have added to throughout the project. These contacts have received 9 electronic newsletters to inform them about the progress of the project, key deliverables and upcoming events.
CRISP Events
Twelve stakeholder events were held by the CRISP partners during the project, these included workshops (stakeholder, validation and scenario), roundtable with Data Protection Authorities, and a Final Conference. All events were attended by experts in security, data protection, certification, standardisation and accreditation. In addition, a number of events were also attended by security industry, and industry association representatives. The events served to raise the profile of the CRISP project, and allowed the partners to disseminate findings and validate them with the help of the attending experts.
The CRISP Final Conference was held on 16th March at the BAO Congress Centre in Brussels, Belgium. The conference was co-located with the 6th Plenary of the Community of Users on Secure, Safe and Resilient Societies, and related Thematic Workshops, held by the European Commission DG HOME. The conference consisted of two parts, a plenary session in the morning and two topic specific workshops in the afternoon. The plenary session consisted of presentations on the CRISP results from the project coordinator, as well as externa presentations on the broader security industry, policy and data protection context and how CRISP results can prove useful to address specific issues. The CRISP partners and experts then took questions from the audience and facilitated discussion on the future of the CRISP scheme. The two afternoon workshops consisted of 1) an in-depth presentation of the STEFi methodology and the CRISP CWA, and 2) standardisation and certification basics, and a discussion on how standards are used within the security field.
Attendees could sign up for the whole event or choose to attend either plenary session or workshop only. The final conference was attended by 52 participants from various European countries and stakeholder groups, e.g. certification bodies, industry, standardisation bodies, security academics, data protection authorities, and accreditation bodies. All presentation slides from the conference are available from the project SlideShare account.
Publications and presentations at third party events
The CRISP partners attended 49 external events in Europe, and internationally through which they presented CRISP results through presentations and posters. Through these events the CRISP project has been promoted to audiences in certification, standardisation, security, disaster resilience, privacy, data protection, cyber security and surveillance studies. A number of papers have been published and at least three papers are forthcoming, along with two book chapters. CRISP partners have also published in magazines and via conference proceedings.
SUCCESS FACTORS FOR FURTHER EXPLOITATION
The CRISP project holds the opinion that the CRISP certification scheme shall be set up and maintained by the future scheme owner in cooperation with interested stakeholders. For the acceptance of the scheme by all relevant stakeholders and to realise its full potential, the following preconditions and key success factors for the future CRISP certification scheme is summarised; the exploitation plan provides further detailed information.
The following preconditions have been put forward:
• Commitment and active support to the certification scheme by all relevant stakeholders;
• Consider the best practices of other relevant certification schemes and existing European and international standards as input for the development of the certification scheme;
• Organize the commitment of the regulatory and political authorities and other relevant stakeholders by involving these as members of a future Advisory or Steering Board which acts as an external monitoring and oversight group to the CRISP organisation;
• Endorsement of the CRISP certification scheme by the regulators, EC and the Member States would result in faster implementation and uptake of the scheme, since compliance with regulatory demands, especially data protection, is an important driver for the market CRISP certification scheme is targeting at;
• Starting with pilot studies during the development of the certification scheme to use the outputs of the pilots for improving the eventual certification scheme.
Crucial factors which need to be taken into account by the future CRISP organisation in the process of finalising the CRISP certification scheme are as followed, as they present key considerations in the formulation of a successful scheme:
• Quality of the scheme - simplicity, openness, transparency and accreditation
The legal standing of the proposed CRISP scheme must be clear and should be open to any interested organisation and its details should be accessible by consumers. The certification bodies offering certification according to the future CRISP certification scheme will need to be accredited in order to achieve the necessary level of trustworthiness, quality and acceptance. Interactions with national accreditation bodies are necessary to facilitate their future task of accrediting certification bodies based on the relevant documents of the CRISP scheme. Finally, the certification procedures shall be as simple as possible.
• Support, acceptance and promotion from key stakeholder groups
The support, acceptance and/or promotion from the different stakeholders, such as the European Commission, national authorities, the industry and security industry bodies (e.g. Euralarm), regulators and certification bodies is the main precondition for the development of the certification scheme.
• Appropriate Management
The proposed CRISP certification scheme must be kept up to date to ensure its relevance, resulting in a scheme that is kept flexible to adapt to technological innovations and any economic and societal changes. In addition, oversight mechanisms must be in place to assure consistency of the scheme, relevance of criteria and validity of certifications. The scheme management must oversee enforcement and assessment processes to ensure that all requirements of the scheme are complied with.
• Impartiality
Impartiality is one of the main aspects associated with CRISP certification and ultimately crucial for the success of the CRISP certification scheme and the certificates issued. The independence of personnel can be safeguarded with impartiality agreements between the scheme owners and the personnel.
List of Websites:
CRISP project website:
www.crispproject.eu
Contact details CRISP coordinator:
Ronald Boon, Netherlands Standardisation Institute (NEN): R.Boon@nen.nl