Final Report Summary - ADHOPHTA (Adopting Hospital Based Health Technology Assessment in EU)
Executive Summary:
The problem
Hospitals are the main entry point of new technologies. Knowledge and tools to evaluate these are often lacking in hospitals. Some high-value innovations never reach clinical practice, while other technologies with no added value do. This highlights the importance of supporting hospital managers and head of clinical departments in making sound decisions on investment in health technologies.
The project Aim
The project 'Adopting hospital based health technology assessment in EU' (ADHOPHTA) aims to bolster the use and impact of high-quality health technology assessment (HTA) in hospital settings. The ultimate goal is to facilitate the adoption of health technologies with proven value in hospitals and to keep costly pseudo-innovations without proven benefit at bay.
To achieve this goal, the team performed a critical analysis of existing hospital-based HTA initiatives (HB-HTA) and a research on the principles informing good practices for HB-HTA. The next step was to make pragmatic knowledge available, as well as tools to facilitate the adoption of hospital-based HTA initiatives and improve the quality of existing ones.
Work done
The project began with a literature search on informational needs and decision making processes for the adoption of technology in hospitals, followed by the development of an open questionnaire, which was administered face-to-face to 53 hospital and clinical managers. The results were used to design a large-scale, web-based survey sent to 339 hospital and clinical managers. Next 38 case studies were performed to characterise the decision-making process for adopting different types of health technologies, including medical devices, medical equipment, drugs and procedures.
Additionally, a literature review was performed to identify good practices in national, regional and current experiences in hospital-based HTA. In parallel, a business health care excellence model was selected (EFQM) and adapted to HB-HTA, to build a framework for principles that should govern good practices of HB-HTA. A Delphi survey was conducted with 43 international experts to validate the principles found.
To determine the level of collaboration among national, regional and hospital-based levels of HTA, the team interviewed 24 HTA professionals in 9 countries, supplemented with detailed case-studies in Finland and Norway. The findings showed that in general these professionals have a positive attitude towards collaboration and informally engage in it frequently. To improve collaboration, multidisciplinary participation and mutual trust and respect are essential. A collection of collaboration strategies and activities was elaborated.
In the second phase of the project, the project developed a portfolio of current patterns and types of hospital-based HTAs units, which led to the identification of four main generic models of HB-HTA units. It was seen that hospitals with HB-HTA units are more efficient in the process of introducing health technologies. In parallel a checklist to help in undertaking high quality HTA at hospital level was produced. This checklist was applied to a sample of HB-HTA reports showing that current products are of moderate-high quality and there is room for improvement. This, along with the findings on informational requirements for hospital decision-makers, resulted in the production of a new AdHopHTA mini-HTA template. A set of 15 guiding principles for good practices in HB-HTA units were identified (9 considered as core).
The overall findings of the project were collected in a handbook that can inspire hospitals in understanding the concept and help them to set up HB-HTA units. This handbook is accompanied by a toolkit, also based on the guiding principles for good practices, designed to support implementation of HB-HTA units or improve them. A third important project result has been a database containing hospital-based HTA products produced by the partners in the consortium.
Project Context and Objectives:
The problem
Hospitals are the main entry point of new technologies. Knowledge and tools to evaluate these are often lacking in hospitals. Some high-value innovations never reach clinical practice, while other technologies with no added value do. This highlights the importance of supporting hospital managers in making sound decisions on investment in health technologies.
The project Aim
AdHopHTA aims to bolster the use and improve the impact of high-quality HTA in hospital settings to facilitate the adoption in hospitals of those health technologies with proven value. To achieve this aim AdHopHTA first performs a critical analysis of existing hospital based HTA initiatives. To better evaluate technology in hospital settings, AdHopHTA will subsequently make available pragmatic knowledge and tools to facilitate adoption of new hospital-based HTA initiatives and to improve the quality of existing ones. AdHopHTA will support an adequate ecosystem where formal coordination among existing hospital-based HTA initiatives is fostered and liaison with national and regional HTA agencies can flourish
Project Results:
During the first reporting period the following WP tasks “WP1- Innovation Uptake and Hospital HTA Models” and “WP2-Assessing Innovations through HTA in Hospitals” run in parallel. Therefore, some research activities were conducted jointly. The results obtained were one of the materials used as input for WP5.
Several activities were conducted in this period. A literature review in their targeted areas (i.e. decision-making and informational needs) was the basis for subsequent research. A joint open questionnaire was designed and administered face to face to 53 hospital and clinical managers. Its results helped to design a large scale web-based survey sent to 339 hospital and clinical managers from partner’s countries (49% response rate). 38 case-studies were performed aimed to characterize the decision-making process for adopting different types of HTs (medical device, medical equipment, drug and procedures). Main results at the end of the first reporting period included:
• Presentation and use of evidence, economic factors and resource needs, and organizational factors are the most frequently mentioned barriers&facilitators for innovation uptake
• Information that hospital managers value most are clinical evidence, economic aspects, safety and organizational aspects
• Strategic aspects for hospital is a new highly demanded informational need not present today in standard National HTA
• Ethical and legal information is quoted as non-relevant, contrary to the usual requirements of national HTA
• More relevance to hospital budget impact than societal cost-effectiveness as required by National/Regional HTA;
• Hospitals with HTA units/programs had more structured and efficient processes for adopting HTs than those without HTA function.
• Decision-making behaviour in up taking innovations at hospitals with HTA units/programs follows a “managerial Science” pattern under the Contingency Decision-making Framework, which is characterized by a high certainty of both consensus on the problem and understanding of the solution.
During the second reporting period, WP1 and WP2 had different specific tasks which led to different results that were also used as inputs for WP5.
In WP1 an “Analysis of organisational models in leading HB-HTA initiatives in Europe” (Task 1.3) was performed. A semi-structured interview based on the framework of the EFQM model, adapted to HB-HTA, was built, and seven interviews with the heads of the HB-HTA units/programs among the AdHopHTA partners with a HB-HTA unit were performed. Partners from Spain, Italy, Turkey, Denmark, Switzerland and Finland participated in the study. Finally, the sample was complemented with the interview of the HB-HTA unit in New Zealand. Data analysis allowed providing an extensive description of the HB-HTA functions, in terms of leadership style, external and internal strategy, people and workforce, products, processes and services, methods used as well as performance. Main results include:
• most of the units have a formalised position within the hospital organisational charts and their strategies are mainly aligned with hospital HTA strategies;
• staff of HB-HTA units is not fully devoted to its activities; nevertheless, “mature” units are more willing to use full-time contracts than those that have been established more recently;
• the stage of maturity seems to have an impact on organisational arrangements and formalisation of the processes. Mature units hold a formal position within their respective hospitals’ organizational chart, follow formal procedures in informing decision makers, have formal recruiting processes for personnel and are keener to organise training programmes for collaborators and external customers. Newly established units show a lower level of formalisation of procedures for assessing technologies and managing personnel;
• professional profiles usually present in all units are clinicians, health economists and public health specialists. Medical doctors are the usually the heads of units;
• the involvement of the units in the adoption process is almost always mandatory, even if the results produced are only advisory;
• components of the HTA process (i.e. the assessment of health technologies) are similar in all the contexts and do not relate to the age of units;
• HB-HTA units act as a formal filter in addressing the choice of adoption of the HTs assessed;
• from the micro characteristics of studied HB-HTA units, 4 main organisational models have emerged: independent group (provides support for managerial decisions in fairly informal way), integrated-essential HB-HTA unit (small HB-HTA units with limited staff involving many stakeholders and ‘allies’), stand-alone HB-HTA unit (formalised and specialised internal HB HTA unit ; not strongly influenced by national or regional HTA), integrated-specialised HB-HTA unit (HB-HTA unit formally collaborating with national or regional HTA).
As regards to WP2, in this second period, a check list for quality assessment of HB-HTA reports was developed based on the following activities: A systematic literature review of characteristics defining high quality in HTA reports, previous AdHopHTA research on needs of hospital decision makers, and inputs from experienced HB-HTA professionals. A convenience sample of reports from HB-HTA producing organisations from 9 European countries was collected. Quality was assessed by considering whether a given criteria from the checklist had been explicitly addressed in the HB-HTA reports (yes, no, not relevant). A quality score was calculated as the proportion of positive ratings, i.e. the number of “yes”-ratings divided by the total number of criteria in the checklist. Main results were:
• the quality checklist consisting of 26 items was produced for the assessment of quality of current HB-HTA reports;
• great variation was observed between HB-HTA reports in terms of: (i) overall quality score (0.50-0.92); (ii) comprehensiveness – total no. of pages (5-54 pages), and; (iii) staff effort invested – total no. of weeks (0.6 – 14.3)
• Although most of HB-HTA reports has a moderate-high quality score; none of the 9 HB-HTA reports met all of the quality criteria – there is a potential for improvement
• while most of the HB-HTA reports were well structured, included a reference list and described both the problem of interest and results within most domains, there was still a significant potential for improvement in terms of description of for instance conflicts of interests, the quality of included information, the patients’ experience and the strategic implications of a new health technology. Discussion of findings in the reports is another area with improvement potential.
• HB-HTA can be done in many ways – of varying quality and comprehensiveness and with a varying use of resources
WP3 dealt with exploring bridging activities between Hospital HTA and National/Regional HTA Agencies. Activities performed include two case studies (Norway and Finland) with formalised collaboration; 24 surveys from national, regional, and hospital key HTA professionals on current and desired collaboration; and a literature review. Main findings include the prominence of informal and voluntary current collaborations; the positive attitude towards collaboration, and a wide range of collaborative areas, including mutual strategic and political support. Main barriers identified include: lack of knowledge and culture of HB-HTA, lack of regulations on this area, differences in the methodological standards and timings between HB-HTA and National/Regional HTA. Multidisciplinary participation, mutual trust and respect are essential to improve collaboration. A portfolio of key elements for success and areas of collaboration has been also produced.
The results of this WP fed the work that was done in WP5.
This WP3 was completed during the first reporting period.
The focus of WP4 was to update HTA best practices at national / regional level and explore current hospital-based HTA practices. A literature review found best practices in National/Regional HTA and practices in HB-HTA. In parallel, a business health care excellent model was identified (EFQM model) and adapted to HB-HTA to be used as a framework for developing principles that should govern HB-HTA good practices. The EFQM model has nine main excellence criteria (i.e. leadership, strategy, people, partnership & resources, processes, products & services, customers results, people results, society results, business results), that should be accomplished by several defined key elements. The adaptation of the EFQM model consisted in identifying these specific key elements for HB-HTA; this was made through the results from the literature review and a dry-run with one HB-HTA partner institution. The result was a first proposal of a framework which was discussed and re-shaped with project partners that run a HB-HTA unit/program. A focus group with the Advisory Committee of the project was performed to validate and complement this first framework. All these activities resulted in the identification of forty-two key elements that should feed the principles for HB-HTA good practices. Finally, a Delphi survey with these 42 key elements was sent in two rounds to 43 international experts in HTA and HB-HTA to assess their relevance for hospital-based HTA. The results showed that all elements were considered relevant.
Therefore, a final (3rd) round of the Delphi Survey was carried out asking respondents to rate each of the 42 key elements according to when the element had to be implemented in a HB-HTA unit: deployment (starting to set up), improvement, and optimisation (reaching excellence). If, for a given key element, answers were equally distributed among the three levels of development, then each level would receive exactly a third of the votes. This was an indication that experts, on average, did not associate that specific key element with a particular level of development of the HB-HTA function. Statistical analyses were performed to find out if the level of development for each particular key element had a percentage of answers significantly higher than one third. The significance level was set at 5% (p≤0.05). In addition, two open-questions were added to the questionnaire of the third round in order to collect specific views of experts on patients’ involvement as well customisation of the HB-HTA process.
The results of the Delphi study provided a basis to extract recommendations for the initial deployment of a HB-HTA function. Mainly, experts considered leadership an important asset to build a HB-HTA function and mission, vision, and values of the function were seen as a prerequisite. Furthermore, a strategy taking into account the specific hospital context needed to be designed beforehand. Finally, experts recognised the specificity of the skills required to conduct HB-HTA and the necessity to hire and retain qualified people.
Results from Delphi contributed to guide the development of the final set of guiding principles for good practices to be included in the handbook and the toolkit.
Another objective within this WP was to identify and revise current EU incentivising policies affecting HTA. As policies of reference, the following were identified: Funding of the EUnetHTA (European Network of HTA Agencies), the Directive 2011/24/EU (point out at the creation of a stable EU HTA Network) and the 7th FP funding of specific HTA projects. However, only the latter recognised the specificity and importance of HB-HTA.
Therefore, there is a need to consider HB-HTA under the on-going HTA EU policies.
WP5 aimed at building final consensus on HB-HTA good practices, their deployment in practice (T5.1) and their validation (T5.2). To achieve these objectives multiple methods were applied. Firstly, a literature review in medical databases and grey literature was conducted to inform the development and design of the handbook and the toolkit.
Secondly, partners carried out a joint analysis and discussion of the outputs of WP1, WP2, WP3 and WP4 that led to the elaboration of the handbook, which include a final set of guiding principles for good practices in HB-HTA units, among other information obtained as a result of the research performed during the project. The definition of a set of guiding principles for HB-HTA good practices was important since was to be included as a chapter in the handbook as well as constitute the basis for the development of the toolkit.
Furthermore, both the handbook and toolkit went through a three step validation process with: (i) face-to-face interviews with a convenience sample of respondents who answered the open questionnaire in the first period of the project (one hospital manager and one clinical director) selected from the nine AdHopHTA partners' countries; (ii) Advisory Committee of the AdHopHTA project, and; (iii) 10 global HTA leaders at a validation workshop. The validation process led to the ultimate refinement of the AdHopHTA handbook and toolkit based on the feedback of participants.
The handbook of hospital-based HTA consists of information (from evidence) and knowledge (from partners’ experience) supporting the development of an evidence- and knowledge-based decision-making process for management of HTs in hospitals. The handbook is a body of knowledge of current characteristics of HB-HTA in Europe and includes information on what HB-HTA is and its impact in hospitals, how hospitals are using HB-HTA to manage health technologies, current organisational models of HB-HTA, what information is relevant for hospital decision-makers when performing HB-HTA, how to perform assessment reports at the hospital level (including the new AdHopHTA mini-HTA template) as well as characteristics and quality of current HB-HTA reports, current collaboration experiences with national/regional HTA agencies, and the principles that should guide good practices in setting and running a HB-HTA unit (which includes the process of assessing technologies).
The handbook also includes recommendations to EU for reaching a comprehensive HTA ecosystem in the EU, through the inclusion of Hospital-based HTA. Additionally, a possible business canvas for a future hospital-based HTA network was provided. Moreover, specific recommendations for EU member states and different stakeholders are provided on how to consider or use and implement hospital-based HTA in their constituencies.
The toolkit consists of a set of tools that provide guidance and facilitate the pragmatic application of the guiding principles for good practices in HB-HTA units. It is a web-based, interactive resource with 34 tools. It enables self-assessment, as starting point, of any given HB-HTA activity and thus, helps to tailor and ease the setting-up and running HB-HTA units. The toolkit also offers a collection of suggested solutions to potential problems that may emerge while setting up and running an HB-HTA unit.
In addition to other regular dissemination activities, WP6 had a focus on the development of a HB-HTA database. Discussions on the characteristics of this database were held with partners for defining its needs and requirements. Analysis of current available repositories of HTA products (eg EUnetHTA Pop database) and evidence-based products have been performed. The final database includes 219 entries of HB-HTA reports.
Regarding dissemination activities, this WP focused on raising worldwide awareness of AdHopHTA project and results, during and beyond the life of the project. This included extensive communication and dissemination activities as well as making hospital based HTA products easily available to interested organisations. Dissemination actions performed comprised, among others, the set-up and maintenance of the AdHopHTA website; the development of promotional material which was distributed in relevant
scientific conferences; presentations by partners at national and international invited conferences; and teaching activities. Additionally, a contact database of potentially interested institutions and individuals in AdHopHTA results was created to support dissemination of end results. A final website with the three AdHopHTA products (i.e. handbook, toolkit, and database) as well as useful materials (for instance infographics, power-point presentations) has been produced succinctly presenting key findings of the project. These products and materials are available at www.adhophta.eu.
Potential Impact:
The different activities conducted in the project have led to an accurate picture of the current situation of HB-HTA units, their characteristics and the expectations of decision-makers around them. Thus, the research of WP1 shed new light on how HB-HTA units carry out their activity in Europe. The description of current HB-HTA organisational models can be seen as a reference for HTA experts and professionals in improving their own HB-HTA units or setting-up a new one at hospital level. The recognition of the existence of a sort of evolution path for HB-HTA functions could be also useful to manage the evolution and development of an HTA unit at hospital level. These key observations for hospital managers and executives interested in HTA have been included in the handbook.
Hospital decision-makers need accurate, relevant and timely inputs for decision-making, but these objectives may be in conflict. According to the outputs of WP2 striving to achieve the highest level of quality may as well have a price in terms of resource use, i.e. the higher the quality of information the greater the use of time. This finding is key for current and future professionals conducting HB-HTA assessments. Performing HB-HTA assessments and reporting this work on time need to look for a break-even point between the timely answer to decision-maker request and the use of the tools that guarantee a high quality report (i.e. AdHopHTA quality checklist and the AdHopHTA mini-HTA template).
Findings of WP4 enriched the comprehension of the language and managerial practices that are usual at hospitals. This insight led to the elaboration of a first set of pragmatic guiding principles that pointed at better ways of approaching decision-makers, thus paving the way for real impact in hospital settings. This is an important step forward with “impact on European HTA policies”, also in alignment with current European directives (Directive on the application of patients’ rights in cross-border healthcare) and strategies towards a more systematic evaluation of technologies across the healthcare system. The handbook of HB-HTA brings comprehensive information and knowledge to guide those who want to embark on or improve their HB-HTA activity. Wider adoption of HB-HTA would result in better decision making in the adoption of HTs at hospital level. This is a fundamental component for the sustainability of healthcare systems. This handbook is seamlessly complemented by the toolkit for HB-HTA. Devised to provide a meaningful aid in setting up an HB-HTA unit or improving its performance, the toolkit follows the guiding principles for good practices developed by the AdHopHTA project. It incorporates targeted tools and suggested solutions to problems that may emerge at any stage of the development of HTA activity. With an easy-to-use and interactive design, it is expected that clinical and hospital managers will appreciate its utility to guide them through complex assessment process as well as supporting the deployment of HTA activities in their hospitals. Moreover, one of the tools is a self-assessment test, which positions the starting-point of the hospital and provides with a tailored implementation plan to concentrate on gaps in capabilities and resources towards a successful, fully-operating HB-HTA unit.
AdHopHTA partners, and collaborators, have shown that HB-HTA units have a positive impact in promoting health care quality and efficiency in hospitals. For example, one specific assessment on laboratory use produced a saving of $371K and a 10% reduction in unnecessary tests in one year; savings that will be able to be used in more beneficent clinical procedures and technologies. Additionally, considering that all tests have false positive and false negative results, decreasing unnecessary tests will avoid that people will be exposed to additional diagnostic techniques and/or unnecessary treatments. Other HB-HTA units have also shown big savings in their overall work. For example, 16 HB-HTA reports resulted in an annual estimating saving of US$ 3 M; and in another hospital €4M in net present values for 12 recommended technologies and efficiencies of €13.6 M in net present value for 11 not recommended technologies. These are examples of specific hospitals, if more hospitals will use HB-HTA efficiencies in health care, while maintaining health care quality, will be bigger allowing health care systems to distribute limited resources more wisely. (Please refer to AdhopHTA handbook of HB-HTA for more details and references)
List of Websites:
http://www.adhophta.eu/
Dr. Laura Sampietro-Colom
FUNDACIÓ PRIVADA CLÍNIC PER A LA RECERCA BIOMÉDICA
+34 93 227 91 54
LSAMPIET@clinic.ub.es
The problem
Hospitals are the main entry point of new technologies. Knowledge and tools to evaluate these are often lacking in hospitals. Some high-value innovations never reach clinical practice, while other technologies with no added value do. This highlights the importance of supporting hospital managers and head of clinical departments in making sound decisions on investment in health technologies.
The project Aim
The project 'Adopting hospital based health technology assessment in EU' (ADHOPHTA) aims to bolster the use and impact of high-quality health technology assessment (HTA) in hospital settings. The ultimate goal is to facilitate the adoption of health technologies with proven value in hospitals and to keep costly pseudo-innovations without proven benefit at bay.
To achieve this goal, the team performed a critical analysis of existing hospital-based HTA initiatives (HB-HTA) and a research on the principles informing good practices for HB-HTA. The next step was to make pragmatic knowledge available, as well as tools to facilitate the adoption of hospital-based HTA initiatives and improve the quality of existing ones.
Work done
The project began with a literature search on informational needs and decision making processes for the adoption of technology in hospitals, followed by the development of an open questionnaire, which was administered face-to-face to 53 hospital and clinical managers. The results were used to design a large-scale, web-based survey sent to 339 hospital and clinical managers. Next 38 case studies were performed to characterise the decision-making process for adopting different types of health technologies, including medical devices, medical equipment, drugs and procedures.
Additionally, a literature review was performed to identify good practices in national, regional and current experiences in hospital-based HTA. In parallel, a business health care excellence model was selected (EFQM) and adapted to HB-HTA, to build a framework for principles that should govern good practices of HB-HTA. A Delphi survey was conducted with 43 international experts to validate the principles found.
To determine the level of collaboration among national, regional and hospital-based levels of HTA, the team interviewed 24 HTA professionals in 9 countries, supplemented with detailed case-studies in Finland and Norway. The findings showed that in general these professionals have a positive attitude towards collaboration and informally engage in it frequently. To improve collaboration, multidisciplinary participation and mutual trust and respect are essential. A collection of collaboration strategies and activities was elaborated.
In the second phase of the project, the project developed a portfolio of current patterns and types of hospital-based HTAs units, which led to the identification of four main generic models of HB-HTA units. It was seen that hospitals with HB-HTA units are more efficient in the process of introducing health technologies. In parallel a checklist to help in undertaking high quality HTA at hospital level was produced. This checklist was applied to a sample of HB-HTA reports showing that current products are of moderate-high quality and there is room for improvement. This, along with the findings on informational requirements for hospital decision-makers, resulted in the production of a new AdHopHTA mini-HTA template. A set of 15 guiding principles for good practices in HB-HTA units were identified (9 considered as core).
The overall findings of the project were collected in a handbook that can inspire hospitals in understanding the concept and help them to set up HB-HTA units. This handbook is accompanied by a toolkit, also based on the guiding principles for good practices, designed to support implementation of HB-HTA units or improve them. A third important project result has been a database containing hospital-based HTA products produced by the partners in the consortium.
Project Context and Objectives:
The problem
Hospitals are the main entry point of new technologies. Knowledge and tools to evaluate these are often lacking in hospitals. Some high-value innovations never reach clinical practice, while other technologies with no added value do. This highlights the importance of supporting hospital managers in making sound decisions on investment in health technologies.
The project Aim
AdHopHTA aims to bolster the use and improve the impact of high-quality HTA in hospital settings to facilitate the adoption in hospitals of those health technologies with proven value. To achieve this aim AdHopHTA first performs a critical analysis of existing hospital based HTA initiatives. To better evaluate technology in hospital settings, AdHopHTA will subsequently make available pragmatic knowledge and tools to facilitate adoption of new hospital-based HTA initiatives and to improve the quality of existing ones. AdHopHTA will support an adequate ecosystem where formal coordination among existing hospital-based HTA initiatives is fostered and liaison with national and regional HTA agencies can flourish
Project Results:
During the first reporting period the following WP tasks “WP1- Innovation Uptake and Hospital HTA Models” and “WP2-Assessing Innovations through HTA in Hospitals” run in parallel. Therefore, some research activities were conducted jointly. The results obtained were one of the materials used as input for WP5.
Several activities were conducted in this period. A literature review in their targeted areas (i.e. decision-making and informational needs) was the basis for subsequent research. A joint open questionnaire was designed and administered face to face to 53 hospital and clinical managers. Its results helped to design a large scale web-based survey sent to 339 hospital and clinical managers from partner’s countries (49% response rate). 38 case-studies were performed aimed to characterize the decision-making process for adopting different types of HTs (medical device, medical equipment, drug and procedures). Main results at the end of the first reporting period included:
• Presentation and use of evidence, economic factors and resource needs, and organizational factors are the most frequently mentioned barriers&facilitators for innovation uptake
• Information that hospital managers value most are clinical evidence, economic aspects, safety and organizational aspects
• Strategic aspects for hospital is a new highly demanded informational need not present today in standard National HTA
• Ethical and legal information is quoted as non-relevant, contrary to the usual requirements of national HTA
• More relevance to hospital budget impact than societal cost-effectiveness as required by National/Regional HTA;
• Hospitals with HTA units/programs had more structured and efficient processes for adopting HTs than those without HTA function.
• Decision-making behaviour in up taking innovations at hospitals with HTA units/programs follows a “managerial Science” pattern under the Contingency Decision-making Framework, which is characterized by a high certainty of both consensus on the problem and understanding of the solution.
During the second reporting period, WP1 and WP2 had different specific tasks which led to different results that were also used as inputs for WP5.
In WP1 an “Analysis of organisational models in leading HB-HTA initiatives in Europe” (Task 1.3) was performed. A semi-structured interview based on the framework of the EFQM model, adapted to HB-HTA, was built, and seven interviews with the heads of the HB-HTA units/programs among the AdHopHTA partners with a HB-HTA unit were performed. Partners from Spain, Italy, Turkey, Denmark, Switzerland and Finland participated in the study. Finally, the sample was complemented with the interview of the HB-HTA unit in New Zealand. Data analysis allowed providing an extensive description of the HB-HTA functions, in terms of leadership style, external and internal strategy, people and workforce, products, processes and services, methods used as well as performance. Main results include:
• most of the units have a formalised position within the hospital organisational charts and their strategies are mainly aligned with hospital HTA strategies;
• staff of HB-HTA units is not fully devoted to its activities; nevertheless, “mature” units are more willing to use full-time contracts than those that have been established more recently;
• the stage of maturity seems to have an impact on organisational arrangements and formalisation of the processes. Mature units hold a formal position within their respective hospitals’ organizational chart, follow formal procedures in informing decision makers, have formal recruiting processes for personnel and are keener to organise training programmes for collaborators and external customers. Newly established units show a lower level of formalisation of procedures for assessing technologies and managing personnel;
• professional profiles usually present in all units are clinicians, health economists and public health specialists. Medical doctors are the usually the heads of units;
• the involvement of the units in the adoption process is almost always mandatory, even if the results produced are only advisory;
• components of the HTA process (i.e. the assessment of health technologies) are similar in all the contexts and do not relate to the age of units;
• HB-HTA units act as a formal filter in addressing the choice of adoption of the HTs assessed;
• from the micro characteristics of studied HB-HTA units, 4 main organisational models have emerged: independent group (provides support for managerial decisions in fairly informal way), integrated-essential HB-HTA unit (small HB-HTA units with limited staff involving many stakeholders and ‘allies’), stand-alone HB-HTA unit (formalised and specialised internal HB HTA unit ; not strongly influenced by national or regional HTA), integrated-specialised HB-HTA unit (HB-HTA unit formally collaborating with national or regional HTA).
As regards to WP2, in this second period, a check list for quality assessment of HB-HTA reports was developed based on the following activities: A systematic literature review of characteristics defining high quality in HTA reports, previous AdHopHTA research on needs of hospital decision makers, and inputs from experienced HB-HTA professionals. A convenience sample of reports from HB-HTA producing organisations from 9 European countries was collected. Quality was assessed by considering whether a given criteria from the checklist had been explicitly addressed in the HB-HTA reports (yes, no, not relevant). A quality score was calculated as the proportion of positive ratings, i.e. the number of “yes”-ratings divided by the total number of criteria in the checklist. Main results were:
• the quality checklist consisting of 26 items was produced for the assessment of quality of current HB-HTA reports;
• great variation was observed between HB-HTA reports in terms of: (i) overall quality score (0.50-0.92); (ii) comprehensiveness – total no. of pages (5-54 pages), and; (iii) staff effort invested – total no. of weeks (0.6 – 14.3)
• Although most of HB-HTA reports has a moderate-high quality score; none of the 9 HB-HTA reports met all of the quality criteria – there is a potential for improvement
• while most of the HB-HTA reports were well structured, included a reference list and described both the problem of interest and results within most domains, there was still a significant potential for improvement in terms of description of for instance conflicts of interests, the quality of included information, the patients’ experience and the strategic implications of a new health technology. Discussion of findings in the reports is another area with improvement potential.
• HB-HTA can be done in many ways – of varying quality and comprehensiveness and with a varying use of resources
WP3 dealt with exploring bridging activities between Hospital HTA and National/Regional HTA Agencies. Activities performed include two case studies (Norway and Finland) with formalised collaboration; 24 surveys from national, regional, and hospital key HTA professionals on current and desired collaboration; and a literature review. Main findings include the prominence of informal and voluntary current collaborations; the positive attitude towards collaboration, and a wide range of collaborative areas, including mutual strategic and political support. Main barriers identified include: lack of knowledge and culture of HB-HTA, lack of regulations on this area, differences in the methodological standards and timings between HB-HTA and National/Regional HTA. Multidisciplinary participation, mutual trust and respect are essential to improve collaboration. A portfolio of key elements for success and areas of collaboration has been also produced.
The results of this WP fed the work that was done in WP5.
This WP3 was completed during the first reporting period.
The focus of WP4 was to update HTA best practices at national / regional level and explore current hospital-based HTA practices. A literature review found best practices in National/Regional HTA and practices in HB-HTA. In parallel, a business health care excellent model was identified (EFQM model) and adapted to HB-HTA to be used as a framework for developing principles that should govern HB-HTA good practices. The EFQM model has nine main excellence criteria (i.e. leadership, strategy, people, partnership & resources, processes, products & services, customers results, people results, society results, business results), that should be accomplished by several defined key elements. The adaptation of the EFQM model consisted in identifying these specific key elements for HB-HTA; this was made through the results from the literature review and a dry-run with one HB-HTA partner institution. The result was a first proposal of a framework which was discussed and re-shaped with project partners that run a HB-HTA unit/program. A focus group with the Advisory Committee of the project was performed to validate and complement this first framework. All these activities resulted in the identification of forty-two key elements that should feed the principles for HB-HTA good practices. Finally, a Delphi survey with these 42 key elements was sent in two rounds to 43 international experts in HTA and HB-HTA to assess their relevance for hospital-based HTA. The results showed that all elements were considered relevant.
Therefore, a final (3rd) round of the Delphi Survey was carried out asking respondents to rate each of the 42 key elements according to when the element had to be implemented in a HB-HTA unit: deployment (starting to set up), improvement, and optimisation (reaching excellence). If, for a given key element, answers were equally distributed among the three levels of development, then each level would receive exactly a third of the votes. This was an indication that experts, on average, did not associate that specific key element with a particular level of development of the HB-HTA function. Statistical analyses were performed to find out if the level of development for each particular key element had a percentage of answers significantly higher than one third. The significance level was set at 5% (p≤0.05). In addition, two open-questions were added to the questionnaire of the third round in order to collect specific views of experts on patients’ involvement as well customisation of the HB-HTA process.
The results of the Delphi study provided a basis to extract recommendations for the initial deployment of a HB-HTA function. Mainly, experts considered leadership an important asset to build a HB-HTA function and mission, vision, and values of the function were seen as a prerequisite. Furthermore, a strategy taking into account the specific hospital context needed to be designed beforehand. Finally, experts recognised the specificity of the skills required to conduct HB-HTA and the necessity to hire and retain qualified people.
Results from Delphi contributed to guide the development of the final set of guiding principles for good practices to be included in the handbook and the toolkit.
Another objective within this WP was to identify and revise current EU incentivising policies affecting HTA. As policies of reference, the following were identified: Funding of the EUnetHTA (European Network of HTA Agencies), the Directive 2011/24/EU (point out at the creation of a stable EU HTA Network) and the 7th FP funding of specific HTA projects. However, only the latter recognised the specificity and importance of HB-HTA.
Therefore, there is a need to consider HB-HTA under the on-going HTA EU policies.
WP5 aimed at building final consensus on HB-HTA good practices, their deployment in practice (T5.1) and their validation (T5.2). To achieve these objectives multiple methods were applied. Firstly, a literature review in medical databases and grey literature was conducted to inform the development and design of the handbook and the toolkit.
Secondly, partners carried out a joint analysis and discussion of the outputs of WP1, WP2, WP3 and WP4 that led to the elaboration of the handbook, which include a final set of guiding principles for good practices in HB-HTA units, among other information obtained as a result of the research performed during the project. The definition of a set of guiding principles for HB-HTA good practices was important since was to be included as a chapter in the handbook as well as constitute the basis for the development of the toolkit.
Furthermore, both the handbook and toolkit went through a three step validation process with: (i) face-to-face interviews with a convenience sample of respondents who answered the open questionnaire in the first period of the project (one hospital manager and one clinical director) selected from the nine AdHopHTA partners' countries; (ii) Advisory Committee of the AdHopHTA project, and; (iii) 10 global HTA leaders at a validation workshop. The validation process led to the ultimate refinement of the AdHopHTA handbook and toolkit based on the feedback of participants.
The handbook of hospital-based HTA consists of information (from evidence) and knowledge (from partners’ experience) supporting the development of an evidence- and knowledge-based decision-making process for management of HTs in hospitals. The handbook is a body of knowledge of current characteristics of HB-HTA in Europe and includes information on what HB-HTA is and its impact in hospitals, how hospitals are using HB-HTA to manage health technologies, current organisational models of HB-HTA, what information is relevant for hospital decision-makers when performing HB-HTA, how to perform assessment reports at the hospital level (including the new AdHopHTA mini-HTA template) as well as characteristics and quality of current HB-HTA reports, current collaboration experiences with national/regional HTA agencies, and the principles that should guide good practices in setting and running a HB-HTA unit (which includes the process of assessing technologies).
The handbook also includes recommendations to EU for reaching a comprehensive HTA ecosystem in the EU, through the inclusion of Hospital-based HTA. Additionally, a possible business canvas for a future hospital-based HTA network was provided. Moreover, specific recommendations for EU member states and different stakeholders are provided on how to consider or use and implement hospital-based HTA in their constituencies.
The toolkit consists of a set of tools that provide guidance and facilitate the pragmatic application of the guiding principles for good practices in HB-HTA units. It is a web-based, interactive resource with 34 tools. It enables self-assessment, as starting point, of any given HB-HTA activity and thus, helps to tailor and ease the setting-up and running HB-HTA units. The toolkit also offers a collection of suggested solutions to potential problems that may emerge while setting up and running an HB-HTA unit.
In addition to other regular dissemination activities, WP6 had a focus on the development of a HB-HTA database. Discussions on the characteristics of this database were held with partners for defining its needs and requirements. Analysis of current available repositories of HTA products (eg EUnetHTA Pop database) and evidence-based products have been performed. The final database includes 219 entries of HB-HTA reports.
Regarding dissemination activities, this WP focused on raising worldwide awareness of AdHopHTA project and results, during and beyond the life of the project. This included extensive communication and dissemination activities as well as making hospital based HTA products easily available to interested organisations. Dissemination actions performed comprised, among others, the set-up and maintenance of the AdHopHTA website; the development of promotional material which was distributed in relevant
scientific conferences; presentations by partners at national and international invited conferences; and teaching activities. Additionally, a contact database of potentially interested institutions and individuals in AdHopHTA results was created to support dissemination of end results. A final website with the three AdHopHTA products (i.e. handbook, toolkit, and database) as well as useful materials (for instance infographics, power-point presentations) has been produced succinctly presenting key findings of the project. These products and materials are available at www.adhophta.eu.
Potential Impact:
The different activities conducted in the project have led to an accurate picture of the current situation of HB-HTA units, their characteristics and the expectations of decision-makers around them. Thus, the research of WP1 shed new light on how HB-HTA units carry out their activity in Europe. The description of current HB-HTA organisational models can be seen as a reference for HTA experts and professionals in improving their own HB-HTA units or setting-up a new one at hospital level. The recognition of the existence of a sort of evolution path for HB-HTA functions could be also useful to manage the evolution and development of an HTA unit at hospital level. These key observations for hospital managers and executives interested in HTA have been included in the handbook.
Hospital decision-makers need accurate, relevant and timely inputs for decision-making, but these objectives may be in conflict. According to the outputs of WP2 striving to achieve the highest level of quality may as well have a price in terms of resource use, i.e. the higher the quality of information the greater the use of time. This finding is key for current and future professionals conducting HB-HTA assessments. Performing HB-HTA assessments and reporting this work on time need to look for a break-even point between the timely answer to decision-maker request and the use of the tools that guarantee a high quality report (i.e. AdHopHTA quality checklist and the AdHopHTA mini-HTA template).
Findings of WP4 enriched the comprehension of the language and managerial practices that are usual at hospitals. This insight led to the elaboration of a first set of pragmatic guiding principles that pointed at better ways of approaching decision-makers, thus paving the way for real impact in hospital settings. This is an important step forward with “impact on European HTA policies”, also in alignment with current European directives (Directive on the application of patients’ rights in cross-border healthcare) and strategies towards a more systematic evaluation of technologies across the healthcare system. The handbook of HB-HTA brings comprehensive information and knowledge to guide those who want to embark on or improve their HB-HTA activity. Wider adoption of HB-HTA would result in better decision making in the adoption of HTs at hospital level. This is a fundamental component for the sustainability of healthcare systems. This handbook is seamlessly complemented by the toolkit for HB-HTA. Devised to provide a meaningful aid in setting up an HB-HTA unit or improving its performance, the toolkit follows the guiding principles for good practices developed by the AdHopHTA project. It incorporates targeted tools and suggested solutions to problems that may emerge at any stage of the development of HTA activity. With an easy-to-use and interactive design, it is expected that clinical and hospital managers will appreciate its utility to guide them through complex assessment process as well as supporting the deployment of HTA activities in their hospitals. Moreover, one of the tools is a self-assessment test, which positions the starting-point of the hospital and provides with a tailored implementation plan to concentrate on gaps in capabilities and resources towards a successful, fully-operating HB-HTA unit.
AdHopHTA partners, and collaborators, have shown that HB-HTA units have a positive impact in promoting health care quality and efficiency in hospitals. For example, one specific assessment on laboratory use produced a saving of $371K and a 10% reduction in unnecessary tests in one year; savings that will be able to be used in more beneficent clinical procedures and technologies. Additionally, considering that all tests have false positive and false negative results, decreasing unnecessary tests will avoid that people will be exposed to additional diagnostic techniques and/or unnecessary treatments. Other HB-HTA units have also shown big savings in their overall work. For example, 16 HB-HTA reports resulted in an annual estimating saving of US$ 3 M; and in another hospital €4M in net present values for 12 recommended technologies and efficiencies of €13.6 M in net present value for 11 not recommended technologies. These are examples of specific hospitals, if more hospitals will use HB-HTA efficiencies in health care, while maintaining health care quality, will be bigger allowing health care systems to distribute limited resources more wisely. (Please refer to AdhopHTA handbook of HB-HTA for more details and references)
List of Websites:
http://www.adhophta.eu/
Dr. Laura Sampietro-Colom
FUNDACIÓ PRIVADA CLÍNIC PER A LA RECERCA BIOMÉDICA
+34 93 227 91 54
LSAMPIET@clinic.ub.es