Final Report Summary - TEMONAS (TEchnology MONitoring and ASsessment)
Executive Summary:
TEMONAS is the acronym for TEchnology MONitoring and ASsessment. This Project Title is a direct reference to the call topic theme and describes an integrated methodology and IT-tool enabling the evaluation of the technology status of hydrogen and fuel cell technologies as managed by the FCH-JU and its major competitive and incumbent opponents. Guiding principle of TEMONAS is to enable objectified and transparent evaluation of technology status per se, in comparison with other technologies, or monitoring the respective progress of the development of the technology. These functions have been realized in an integrated IT-tool, specifically tailored for the needs of the FCH-JU. TEMONAS uses state-of-the-art database technology enabling secure multi-user operations. Its range of functions includes data entry with corresponding quality monitoring and validation routines, data selection methodologies, evaluation methodologies including assessment and benchmarking which can be used to develop complex multi-criteria decision aid aggregated results as well as monitoring the development of such metrics over time.
The routines can be customized per user or user group and the respective user group privileges. Such evaluations can then also be stored for later reference and, of course, be plotted and the result data exported in various formats.
Complete functional logging within the data base tool ensures traceability of all inputs and outputs.
The project was completed in 21 months and is now in use by the FCH-JU.
Project Context and Objectives:
The Joint Undertaking for Fuel Cells and Hydrogen (FCH-JU) is a large-scale cooperative research venture involving an industry group and currently representing more than 60 industrial corporations, a research association of about 60 institutional members and the European commission who will jointly invest nearly €1 billion into accelerating the commercialization of fuel cell and hydrogen technologies. The starting point sent targets were developed by European technology platform in a bottom-up approach resulting in three key documents i.e. strategic research agenda, deployment strategy and an implementation plan status 2006. The scope of the implementation plan covered a 7 billion euro program; however, due to European Commission cash contribution, which was assumed to be around EUR 1 Bio. for the planning period, ending up at just under €500 million, choices in the focus of research, technology development and demonstration had to be made once the exact funding became clear.
One way to design such programs is to agree on technology roadmaps. Technology roadmaps are an important element of the so-called scientific management culture first introduced into the public management domain in the USA e.g. by NASA for their space program administrations or in the department of defense during the time of Robert McNamara and later broadly applied to strategic technology fields in the industry government programs of MITI in Japan.
Another example of such scientific approaches is the generation of readiness levels created by NASA as risk management tools. The most prominent of these readiness levels is the technology readiness level TRL, which has seen broad use by other sectors outside space program management or aerospace and has become more or less generally accepted definition norm for technology maturity The concept has been expanded over the years to also address other risk fields such as manufacturing readiness or systems integration readiness, and even commercialization readiness, etc. Although the readiness levels may represent an attractive form of managing certain project categories, their combination has come under some scientific debate more recently and they represent but one attribute of a technology roadmap, which, of course, needs to further in depth and width.
In an ideal case such a roadmap is designed by projecting future performances of a given new technology based on past but state-of-the-art known performance. These performances are then compared to the trajectory for the existing or incumbent technology and its performance thereby allowing the formulation of quantitative and qualitative Targets for the new technology. The program design now needs to address the identified gaps between such target performance and actual performance of the new technology. It will typically do so by publishing the targets and inviting scientific and industrial consortia to propose pathways towards reducing or eliminating the gap, and subsequently selecting individual projects addressing one or more dimensions using different technology approaches to achieve a solution. At the predefined milestone dates the performance of the new technology can then be compared to the planning assumptions and targets and management decisions can be made regarding the continuation of both individual projects and the complete program.
On a broad level this process is similar to Gateway Management or Stage Gate Processes employed e.g. in the automotive industry for product development (Peters, 2010) In the case of the FCH-JU the stage gate concept is reflected in the midterm review of individual projects which formally represents a continuation/discontinuation decision point for each project as well as a programme level review already listed in the regulation on establishing the FCH-JU.A number of deviations between purely industrial product development and such a cooperative research venture do, however, exist. They are mainly due to the fact that cooperative research ventures (i) operate at very early stages with high levels of uncertainty, where learning from failures might still represent an important contribution and (ii) typically have more than one decision-maker, in particular since the decision is made by officers of the funding agency requiring an impartial and transparent process thus often relying on external expert reviewers (peer review) who thus exert significant influence on the decision.
Another complication comes from the fact that in today's advanced technology world it is quite unlikely that a new technology or its variant will actually outperform existing technology in every aspect. It is much more likely that the technology may e.g. offer better performance for example in environmental aspects, but may actually be more expensive to own or operate. Decision making thus requires the ability to make broad assessments incorporating a multitude of performance or attribute dimensions. This so-called multi-criteria decision aid has a long tradition in complex project decisions such as location of nuclear plants. While the literature on such Multi-Criteria Decision Aid tool is quite abundant and lists avariety of methodologies. None of them have been reported to be fully integrated into Technology Management Solutions so far.
A general problem occuring when working in very early stages of technology development work or trying to monitor emerging technologies or – as was required by the specifications of the call – ascribe a relative ranking of “proximity-to-commercialization” is that there may be insufficient data for proper algorithm based outputs, either due to the fact that serious development work has just started and it will take years before a sufficient amount of results are published in the open literature to enable broader conclusions or because the data is too divergent to draw any conclusion.
Some industrial technology leaders thus advocate using expert input in developing technology roadmaps. Again, the issue may be the size of the new technology community and the potential related lack of experts without conflicts of interests. Furthermore, expert input tends to be in the form of reports or presentations always formatted along the preferences of the individual experts very often rendering those inputs difficult, if not impossible, to compare. On the other hand, it is a broadly observed phenomenon that innovations do not succeed based only on their technical/technological quality, but also a broader “fitting” to a variety of factors. Thus, considerable effort has been expended, for example, to study the so-called NTB (non-technical barriers) to innovation, not least with significant involvement of the European Commission institutions; therefore, a base for a new approach seemed available.
With the advent of new and even more complex technologies such as those summarized under the heading of “nanotechnologies” new processes were developed to address the broad implications such technologies might have on to not only industry and economic stakeholders but society or nature as a whole. This process is often referred to as Technology Assessment or TA, and represents the version of strategic Technology Management with the highest possible level of societal involvement but is quite complex.
The objectives for the project thus were:
Develop a practical secure multi-user database tool implementing and seamlessly integrating the necessary methologies to overcome the gaps identified in the existing solutions:
1. Data entry to address inconsistent data descriptions and parameter names as well as inaccurate results, different levels of information reliability and confidentiality.
2. Design and implement a data QM process including Validation/authorisation and clear transpararent resonsibilities with full logging of any modifications to ensure transparency.
3. Develop a methodology to assess, benchmark and monitor the Progress of individual projects and technologies both from within the hydrogen domain and in broader competing and incumbent technologies in a focused as well as a holistic approach.
4. Select and seamlessly implement a methodology for multi-criteria comparison
5. Develop a methodology to include expert opinions in a structured format to enable both a determination of "proximity-to-commercialization" and an "emerging technology radar function"
6. Develop the necessary query, result Integration and reporting Tools.
Project Results:
The main S&T foreground of the TEMONAS project is the TEMONAS Software. It provides the FCH-JU with a management tool that successfully addresses all of the five objectives defined.
At its core is an object oriented database, implemented in an RDBMS using an appliaction builder provided as Background by one of the consortium partners.
A variety of objects can be defined in the TEMONAS solution such as Program/Call/Topic as well as Targets, Sources and Research Organisations. The central object is the Research Object (RO) which is smallest discrete unit of which identifyable and relationable performance result is achieved and reported. This RO can be at a variety of aggregation levels ranging from material to complete systems, such as vehicles. Multiple objects can be aggregated into a "meta-type" Aggregated Research Object (ARO) representing a more categorical abstract RO with performance data developed using statistical processing of all ROs aggregated. Thus broad comparisons between technologies are possible, e.g. Compact HFCV vs. Compact Gasoline ICE car or one type of membranes against others.
The data entry is supported by wizard solutions and paramter masks. It includes special routines to address issues like multiple parameters being used and reported to describe the same ontological attribute or adress imprecise performance data, such as durability greater than xy hours while maintaining reasonable results during queries. A strict data responsibility chain including mandatory authorisation and user profile dependent access to confidential data is built into the system.
Analysis is seamlessly escalatable from simple single performances of single research objects to complex multi-criteria comparisions using the MACBETH methodology. An additional module, the Expert Judgment Mapping enables expert input using qualitative and ranking scales to determine a variety of maturity assessments and the status of non-technical barriers and a description of market status thus enabling an analyst to draw conclusions regarding both emerging technologies and proximity-to-commercialization. For all evaluations suitable graphic representations were selected in cooperation with the FCH-JU and implemented as ready-to-use templates. Besides this the TEMONAS Solution contains a powerful graphic engine to enable expert users to create advanced graphic data presentations beyond those pre-defined. All Outputs can be stored inside the database but may also be exported in formats enabling a seamless importing into widely used software packages.
Potential Impact:
The impact of TEMONAS at this point is of an indirect nature as will enable a transparent data based decision making for the FCH-JU. As future framework programs may also aim for this level of transparancy its indirect impact can, however, be quite significant.
Early results of user workshops and dissemination activities indicate a strong interest from both industrial users and funding agencies.
Dissemination activities were focussed on the community of innovation and technology program managers from the hydrogen community as well as addressing the managerial professional society:
Informative items were designed and prepared for awareness at public events including:
• Leaflets and Flyers,
• Posters and
• Memory sticks.
These have been submitted under deliverable D7.22 – Project CD.
Conferences
TEMONAS was presented at 7 international conferences and Events as well as the FCH-JU Review Days, 3 in 2012 and five in 2013 within the project Duration plus one more shortly after the project closing. This needs to be evaluated against a project Duration of 21 months with first results from methodological work only becoming available after Mid 2012. One or two partners were able to represent the consortium at each of the conferences.
2nd International Conference on Leadership, Technology & Innovation Management, Istanbul, TR, October 11-13, 2012
Fuel Cell Seminar 2012, Connecticut USA Connecticut, November 5-8, 2012,
FCH JU Review Day, Brussels November 28-29, 2012
2013
TISS Conference 2013, Mumbai, IN, Jan 22-25, 2013
PACITA Conference, PragueMarch 15-16, 2013
Fuel Cell and Hydrogen Conference 2013,Birmingham March 21, 2013
Hypothesis conference, EdinburghJune 11-12, 2013
Final Workshop, Brussels, May 22, 2013
Exploitation:
The consortium has decided to not only continue to maintain and update the solution as supplied to the FCH-JU but to actively market it with a concrete marketing strategy. In Phase 1 starting with the formal completion of the TEMONAS project, this will be done as a consortium with different partners taking different roles and client responsibilities in line with their strengths and access. A preliminary agreement on pricing and royalties distribution has been found if such marketing efforts are successful.
Target groups include:
- Cooperative RTD consortia
- Industrial organisations with a broad technology portfolio or extreme technology leadership challenges
- Research laboratories
- Funding agencies in both the private and public domain
List of Websites:
www.temonas.eu
The TEMONAS website was set up for online information on the project and tool. Efforts were made to increase the amount of tool related information on the website as the tool began to take shape, so as to arouse interest among the online visitors for the tool enough for them to decide whether they want the tool or not. In line with the open access policy, all reports listed as public are accessible through this website, however, this requires registration prior to the download to be able to monitor the interest and prevent obvious cases of mis- or abuse.
