Final Report Summary - ADVANCE_HTA (Advancing and strengthening the methodological tools and practices relating to the application and implementation of Health Technology Assessment (HTA))
Executive Summary:
ADVANCE-HTA aims to contribute to advances in the methods and practices for HTA in European and other settings by involving the wider stakeholder community in areas actively and heavily debated given their implications for decision-making and resource allocation. ADVANCE-HTA aims to make a number of contributions in six distinct areas, which carry significant policy implications for resource allocation (see Figures 1 and 2 in attached report). These are:
• First, the issue around value for money and the different approaches surrounding current thresholds for resource allocation, where ADVANCE-HTA will systematically explore alternative means of assessing value for money and trace the implications for the conduct of HTA and the use of cost-effectiveness data to inform decision-making.
• Second, the concept of value assessment, and the factors that need to be considered and incorporated beyond cost effectiveness, such as burden of disease, disease severity, quality of the data and evidence produced and the implications these are having on the continuous assessment of new health technologies and relative effectiveness. ADVANCE-HTA will aim to explore new tools and methodologies in this domain, for example Multi-Criteria Decision Analysis, and investigate their adoption and implementation.
• Third, to improve the quality of the evidence required for and the methods associated with the assessment of rare diseases by relying on new data providing a more realistic understanding of the socio-economic benefits of orphan drugs. In this context, ADVANCE-HTA it will develop and validate a framework to support decision-making relating to orphan drugs for rare diseases, by means of a Multi-Criteria-Decision framework.
• Fourth, to improve the robustness of the evidence on the elicitation of preferences by deriving these in more realistic settings, by drawing on the wider EU citizenship and from within the patient community. ADVANCE-HTA will create new data that will incorporate patient-relevant values into widely used tools of quality of life measurement, such as the EQ-5D.
• Fifth, to advance the debate on the suitability of current HTA tools across different categories of medical devices, including diagnostics, ADVANCE-HTA will consolidate the current methods for assessing HTA in medical devices in different settings, address their suitability to appraise different types of medical devices, including diagnostics and propose how current tools can be modified or adapted in order to arrive at more robust methods of assessment.
• Sixth, to improve the implementation and capacity building of HTA, also incorporating improvements as outlined above in settings outside Europe, where HTA is beginning to be considered explicitly in decision-making. ADVANCE-HTA will create a framework for HTAs at different levels (national, hospital [mini-HTAs]) by benchmarking with evidence from countries that have developed such frameworks.
Additional activities within ADVANCE-HTA contribute to furthering the debate on future developments in HTA by bringing together the research being conducted across the Consortium. Further, extensive dissemination of the results has been carried out by effectively linking policy makers, stakeholders and patient networks to the research evidence on HTA.
ADVANCE-HTA impacts a range of stakeholders and activities. The methodological advances in HTA are likely to influence developments in areas such as value-based pricing of medical technologies, or other areas of HTA (e.g. HTA in medical devices). Capacity building activities in Latin America and Eastern Europe have transferred the accumulated expertise to countries that are new to HTA, while involving experts from other settings on the debate surrounding HTA and the health care resource allocation debate.
Overall, ADVANCE-HTA aims to broaden the spectrum, complement and address areas of intense methodological debate in the application, use and implementation of HTA. It also aims to improve HTA methods, which can be taken further by competent authorities nationally whilst supplementing the work of supra-national bodies (e.g. EUnetHTA) towards a common understanding of choices in health care decision-making.
Project Context and Objectives:
The Advance-HTA Project was based on a collaboration between 13 partner agencies and funded by the European Commission's Research Framework Programme (FP7) during the three year period 2013-15. Advance-HTA explored the different processes and varying criteria used for assessing clinical cost-effectiveness across Europe, and how frameworks such as Multi-Criteria Decision Analysis (MCDA) might be used to further enhance the value and acceptability to all stakeholders of current ‘mainstream’ approaches to cost-effectiveness evaluation as the central element in modern Health Technology Assessment. The Project, in addition, sought to determine the extent to which disease rarity impacts on assessments of the incremental cost per QALY ratios (the ICERs) for orphan drugs, and the ways in which preferences relating to health related quality-of-life differ between the general population and patients who have actual experience of given conditions. Two further work streams were designed to cast light on the degree to which, and ways in which, HTA is used in appraising the value of medical devices, and the establishment of both governmental and private sector HTA capacity in ‘emerging’ economic settings like those of Eastern Europe and the Americas, excluding the US. Finally, this LSE-led initiative sought to identify and promote public consideration of the wider policy implications of the research findings generated via the above sets of research activity.
Work Package 1: Value for money (WP Lead: LSHTM)
Most European countries have Health Technology Assessment (HTA) procedures for informing drug reimbursement decisions. Depending on the drug and the country assessing it, the decision can be Favourable, Favourable with restrictions or Non-Favourable. The main objective of this research was to determine the factors that might lead to different drug reimbursement decisions in ten European countries, on Cancer, Rheumatoid Arthritis and Multiple Sclerosis drugs (2002-2015). More specifically, the main areas of focus for this work package were:
• To expand knowledge and understanding of the advantages and disadvantages of the different approaches to assess cost-effectiveness and the main factors defining HTA decision-making and to construct a database of HTA decisions of different European countries;
• To consolidate the lessons from existing research, and to identify what new research would be of greatest value to improve HTA decision-making processes; and
• To advance the methodology in HTA by systematically exploring alternative means of assessing value for money and tracing the implications for the conduct of HTA and the use of cost-effectiveness data to inform decision-making regarding which health technologies to adopt, by empirically identifying the main factors driving HTA decision-making across countries
Work Package 2: Value Assessment in HTA (WP Lead: LSE)
This work package relates to the methodological and empirical aspects of value assessment for new medical technologies as well as advancing our thinking on methods and their implementation for the same. The need for a WP that re-visits the issue of value has become clear in light of the limitations of cost-effectiveness analysis to adequately account for other dimensions of value (which are often considered on an ad hoc basis), as well as the wide variability in coverage recommendations by different HTA bodies for the same technology – indication pair, indicating that there are other factors at play beyond the strict dimensions of economic evaluation. The WP’s specific objectives were:
1. To understand the parameters of value in HTA appraisals from an international perspective;
2. To explore how factors such as disease severity, burden of disease, distinguishing between levels of innovation, and the quality of the available evidence can be incorporated more explicitly – and in a quantifiable way, in the HTA process;
3. To explore how alternative analytical frameworks, such as Decision Analysis, can be used to elicit value;
4. To conduct case studies in specific disease areas by using alternative analytical tools and by explicitly incorporating all identified parameters of value.
Work Package 3: HTA and Rare Diseases - Assessing the Societal Value of Orphan Drugs (WP Lead: ISS)
There were 4 key areas that WP4 focussed on.
• First, to understand whether rarity has an impact on the assessment of the incremental cost per QALY of orphan drugs compared with drugs for more common diseases and whether rarity impacts mainly on the incremental costs or on the incremental QALYs;
• Second, given the difficulties of conducting randomized clinical trials and the progressive nature of many rare diseases, to identify alternative robust sources of data to accumulate knowledge on the effectiveness and societal value of orphan treatments, which can be used for policy decisions;
• Third, to determine in what way(s) do current processes for assessing and appraising drugs need to be adapted to make them suitable for orphan drugs, and whether all elements of societal value can be adequately reflected in existing decision-making procedures.
• Fourth, to develop and validate a framework to support decision-making relating to orphan drugs for rare diseases, by means of a Multi-Criteria-Decision framework.
Work Package 4: HTA and Quality of Life Measurement (WP Lead: IER)
The key objectives of this work package were:
• To determine whether the preferences towards health related quality of life differ between the general population and defined patient groups, including those benefitting from personalised treatments;
• To examine the causes of the established differences between general population and patient group preferences;
• To define advantages and disadvantages of the value sets drawn from the general population and patient populations respectively of cost-effectiveness.
Work Package 5: HTA and Medical devices (WP Lead: TUB)
The work carried out in WP5 aimed to structure existing knowledge on and improve understanding of the particular challenges characterizing the assessment of MDs. Furthermore, it intended to set the groundwork for bridging knowledge and implementation gaps and providing best practice recommendations for HTA institutions and policy-makers alike. Specifically, the key objectives were:
• Develop a Taxonomy of Medical Devices (MDs) based on existing classifications and nomenclatures and test its plausibility and usefulness;
• Identify and compare current HTA methodologies, processes and practices across EU Member States;
• Clarify and supplement earlier findings, and to trace methodological and procedural challenges and trends.
Work Package 6: HTA in Emerging Settings (WP Leads: EASP & PAHO)
WP6 consisted of 3 parts:
• To identify the use and capacity for using HTA in emerging settings (Eastern Europe, Region of the Americas) including not only decision making bodies, but also some other institutions (universities, private companies, etc.). This includes the identification of HTA mechanisms and techniques apply in different emerging countries/regions;
• To encourage the use HTA for decision making through the development of formal guidelines for HTA methodologies in the context of Emerging Countries (South-) Eastern Europe, Region of the Americas, etc.) that make more transparent the use of these tools; and;
• To enable and facilitate an effective exchange of economic evaluations and evidence, reducing duplication of effort, between countries in Eastern Europe/Latin America and EU countries that have strong HTA tradition.
Work Package 7: Potential Implications - Lessons learnt, recommendations and implementation (WP Lead(s): UCLM & LSE)
This final deliverable of Advance-HTA (D7.1) aims to contribute to the debate on future developments in Health Technology Assessment, the advances in a number of aspects of HTA implementation, the policy implications these are having and their contribution to the debate of efficiency in resource allocation in health care systems and the related health system sustainability. The specific objectives of this WO are threefold:
• To provide a concrete analysis of the policy implications of the research results for HTA in the domains researched upon (part I of the report);
• To identify how the research results can be incorporated in the decision-making process and implemented at national, supra-national and international level and what implications are there for specific stakeholders; and
• In synthesizing the results of this project, part II of this report explores the wider conceptual/theoretical, economic, social, political, R&D, and policy implications for HTA and the relevance of the findings of Advance-HTA in this context.
Work Package 8: Dissemination of Project Objectives and Results (WP Lead: LSE)
The main objective of this work package was to effectively link policy makers, stakeholders and patient networks to the research evidence on HTA. The dissemination strategy determined the target groups for different results and the most adequate dissemination mode: publications, face-to-face and electronic dissemination. Dissemination was continuous throughout the life of the project and is also expected to continue after the end of the project through publications in the peer review literature and symposia among others. A number of concrete dissemination activities have been undertaken during the lifetime of the project which are discussed in section 4.1.4.
Overall, Advance-HTA’s findings indicate that further investment needs to be made in developing Health Technology Assessment tools for use in areas outside the narrowly defined pharmaceutical sector, in order to further increase the productivity of health care as a whole. They may also need to be adapted to accommodate the reality that drugs do not normally act alone to achieve the best possible outcomes, albeit the discovery of optimum combinations and administration strategies can – as experience in spheres such as the treatment of child leukaemias has shown – be a long drawn-out process.
However, from a practical perspective it is presently the case that HTA is centrally concerned with a relatively narrow form of medicinal product evaluation, pricing, purchasing and use. This paper reflects this fact. It initially offers an outline of the origins of health economic evaluation in Europe and the United States, and highlights some basic economic aspects of pharmaceutical research, development, manufacturing and supply. It then explores from a public policy perspective a spectrum of topics arising from the Advance-HTA research. These centre on issues like the value of further involving patients and their representatives in determining whether or not treatments are affordable, and the need for well-balanced national and international policies aimed at incentivising public interest focused private investment in pharmaceutical research and development.
Project Results:
Work Package 1 (WP Lead: LSHTM, UK)
Even if European countries have common objectives for HTA systems, the procedure is not homogenous. The operative processes and the organisations work differently across countries. Moreover, evidence shows that there are differences in the final decisions across countries, despite the clinical evidence reviewed being largely the same, and the countries - while not of equal wealth - are of broadly comparable levels of economic development. The main objective of WP1 was to determine the factors that might lead to different drug reimbursement decisions in ten European countries, on Cancer, Rheumatoid Arthritis and Multiple Sclerosis drugs. This goal was achieved through a specially designed taxonomy and an econometric analysis. In a preliminary analysis (Maynou and Cairns, 2015 ), a number of hypotheses were tested that could explain the differences in cancer drug reimbursement decisions across ten European countries. While the results showed that the HTA system characteristics, the drug particularities and the socioeconomic situation can explain some of the differences between countries, a fuller explanation requires a model which determines cancer drug reimbursement decisions incorporating a wide range of health system characteristics and specific characteristics of the individual drugs. The WP proceeds in building a taxonomy, then a database, performing econometric analysis before presenting results and policy implications.
1. Taxonomy
A detailed analysis of the drug reimbursement systems in ten European countries was conducted. Research focused on: Belgium, France, Germany, the Netherlands, Poland, Portugal, Spain, Sweden and the United Kingdom (England and Scotland analysed separately ). These countries were selected because they each have a well-defined HTA process and publicly available information on their drug reimbursement decisions. This first analysis involved: (1) a review of policy documents and relevant literature and examination and querying of the study country decision-making bodies’ websites and (2) a discussion with experts knowledgeable regarding the process in each of the countries. In this second process, we drew heavily on Advance-HTA Consortium members representing some of the studied countries.
This analysis was essential to identify potentially important system-wide factors. This taxonomy (see Table 1 in the appendix) describes the main characteristics of the drug reimbursement system in each country and the main features of each drug-indication pair. Moreover, time dependent variables are also included in the classification to capture the socioeconomic situation of each country. This taxonomy can be classified into three groups, the system-wide variables (organisational, process and method), the product-specific variables (general characteristics and country-specific) and the time dependent variables. Table 1 defines the taxonomy variables. This taxonomy is used to populate the econometric model with explanatory variables.
2. Database
The sample included the reimbursement decisions for Cancer, Multiple Sclerosis and Rheumatoid Arthritis drugs from January 2002 to April 2015 appraised in the ten selected countries (Belgium, France, Germany, Netherlands, Poland, Portugal, Spain, Sweden, England and Scotland ). The dataset contained the outcome of the decision, the date and all the variables of the taxonomy. The decision outcome described the final decision regarding the adoption of the technology: Non-Favourable, Favourable with restrictions and Favourable. In order to create the database, the reimbursement decisions were taken, when possible, from national websites. However, for some countries, these decisions were not publicly available or some details were missing. These limitations were overcome with the collaboration of some partners, namely, UPEC (France), AHTAPoL (Poland), EASP and UCLM (Spain) and TLV (Sweden). The database comprised 1077 decisions, of which 62% were Favourable, 13% were Non-Favourable, and 26% were Favourable with Restrictions (see Table 2 in the appendix). Of these decisions, 79% related to Cancer drugs, 9% related to Multiple Sclerosis and 12% to Rheumatoid Arthritis.
3. Econometric analysis
The objective of the econometric analysis was to determine empirically which of the taxonomy variables were associated with a higher or lower probability of reimbursement (Favourable decision). The dataset was designed as a particular case of a hierarchical model within the family of multi-level models. It considered the decisions for each drug-indication in ten European countries from January 2002 to April 2015. The primary interest was to observe the effect of the explanatory variables (defined in the taxonomy section) on the probability of reimbursement. Although this variable was unobserved, it could be approximated through a categorical variable corresponding to the final decision: 0 = Non-Favourable; 1 = Favourable with restrictions; and 2 = Favourable. In order to take into account different time periods, the spatial distribution and the heterogeneity of the model, a Hierarchical, Random-Effects Ordered Probit was considered to be the best model specification and was used in the analysis. The inference was made through the Bayesian framework, the Integrated Nested Laplace Approximation (INLA).
4. Results
The results of the econometric model pointed to a number of policy relevant findings. In terms of the system-wide variables, the model showed that less stakeholder involvement in the HTA process was associated with a higher probability of reimbursement. However, the requirement of economic evaluation for all drugs, an external review of the evidence, the manufacturer being the initiator of the process, price based on a(n) (external) reference pricing and the budget impact requirement were related to a lower probability of reimbursement.
According to the results of the model for product-specific variables, a drug-indication with a NICE-Favourable decision was linked to a higher probability of this drug-indication being accepted in another country. A drug considered cost-effective by NICE/SMC was associated with a higher probability of reimbursement. The other product-specific variables included in the model were not statistically significant. One of the specific objectives was to observe if the therapeutic area was an additional determinant of the probability of reimbursement. However, the relevant variable was not statistically significant.
Finally, the results also showed the effects of the time-dependent variables included in the model. In particular, the rate of female mortality and percentage of population older than 65 were positively and significantly associated with the probability of reimbursement. However, higher proportion of population below 14 years old was associated with a lower probability of reimbursement. The crisis dummy showing the time effect from 2008 onwards was not statistically significant. However, the interaction between the crisis dummy and the Managed Entry Agreement (MEA) variable shows that a MEA in the crisis period was associated with a higher probability of reimbursement. The remaining time-dependent variables included in the model did not have a statistically significant association with the probability of reimbursement.
A further finding was that according to previous literature and our data collection, differences existed in the final reimbursement decision across the ten countries analysed. As a result, a key finding of our study documented the differences in decisions across the ten European countries. The database showed that for the same drug-indication pair there were different reimbursement decisions for the selected countries. However, the specifications of the econometric model were able to control for the differences across the HTA systems and the socioeconomic conditions of each country, as the random effect used to allow for country variation was not significant.
Moreover, in order to overcome one of the main limitations (i.e. meaning of decision outcome), we carried out two further econometric analyses. In the first, France and Germany were excluded because both countries’ drug assessment is used to inform the pricing decision procedure and not to accept or reject the new technology for coverage purposes. As this procedure differs from the remainder of the sample of countries, which take a decision on whether to accept or not the drug in their benefits catalogue, a decision was made to run the model without taking into account the decisions of France and Germany. Comparing the original model (all countries) with the new model, the main finding was that the new model was more efficient in terms of the DIC and the CPO (both, goodness of fit of the Bayesian approach). The main differences of this new model were that the initiator and the time dependent variables were not statistically significant.
The second analysis adopted a different approach in an attempt to solve the previous limitation. Instead of directly removing both countries, this approach tried to capture this difference by controlling for the pricing system in each case. In this model, the pricing decision variable was removed, but some interactions between the main explanatory variables and the pricing systems were included. The aim of this analysis was to observe different effects depending on the pricing system used in each decision. The main finding was that there was a statistically significant effect when the price-setting is undertaken by the manufacturer (such as in England, Scotland, Sweden, Netherlands and Poland), compared to the baseline category, i.e. price-setting based on negotiation (France, drug non-adding value, and Germany, drug adding value). In other words, the pricing system of each country has an effect on the decision outcome.
5. Policy implications
A number of policy implications have emerged from this work package. First, the research conducted implied a better understanding of the main determinants of drug reimbursement decisions. As a result, policy-makers can be aware of the factors that may drive the HTA decisions. In a new or reformed HTA process, these analyses could help defining the HTA procedure. Second, based on this analysis, differences in drug reimbursement decisions across countries can be better explained to the HTA community. In particular, patient groups and health care professionals can benefit from this information and better understand why some drugs are approved while others do not. Third, from the data collection procedure, it can be determined which of the study countries have a more transparent and accessible drug reimbursement system. The policy implication is that differences in transparency and accessibility of decisions should be addressed, if there is to be greater understanding of the differences in access to these medicines across countries. In other words, if policy-makers are interested in why decisions are different across countries, non-transparency is an important barrier. Finally, during the data collection process, experience was acquired in finding the HTA documentation on the websites and/or internal databases. A pathway can be created for each country, which can be very useful for the HTA community and would be of interest not only to policy-makers, HTA agencies or Departments of Health, but patient groups, health professionals and researchers.
Work Package 2 (WP Lead: LSE, UK)
This WP has researched the domain of value assessment of medical technologies and provides insights on different modalities and frameworks. The WP is associated with the development of 2 methodological frameworks, one that explains a significant part of the unexplained heterogeneity in coverage recommendations across HTA settings and another - using decision analysis principles - that incorporates a wider selection of value dimensions and stakeholder preferences to provide a more holistic perspective on value assessment. Both frameworks have been piloted and tested and evidence is provided on their performance.
1. Value assessment drivers and criteria
A systematic literature review was conducted taking into account evidence from England, Scotland, Sweden, Poland, France, Germany, Italy and Spain showed that, overall, all countries assess similar types of evidence, however the specific endpoints used, their level of provision and requirement (i.e. extent of guidelines provided and any flexibility around them), the way they are incorporated (e.g. explicitly vs. implicitly) and their relative importance varies across countries. The main evidence assessed could be divided into four clusters of evidence: (a) burden of disease; (b) therapeutic and safety impact; (c) innovation level; and (d) socioeconomic impact. In addition, efficiency parameters such as cost-effectiveness and budget impact are considered, but these are essentially derived through the initial assessment of the main evidence and thus could be regarded as different analysis approaches. Furthermore, ethical value characteristics such as equity are also taken into account in many countries, but they usually act as a type of ‘fundamental principles’ on which the complete evaluation, including the committees’ judgements, should be based on. The relative importance of the different criteria, or their overall and relative importance on the overall decision is in the majority of cases, unclear (e.g. the relative importance of overall survival versus the relative importance of quality of life). The analysis has shown that the value of new medical technologies is multi-dimensional and not limited to clinical benefit and cost dimensions.
A number of the value dimensions mentioned above such as the burden of disease and its overall socioeconomic implications are not adequately reflected in the assessment process. These value dimensions are not always considered, and, if they are, this is done in an implicit and non-systematic manner or an ad-hoc basis. Consequently, the methodological framework of these value assessment approaches remains inadequate and at best partial mainly because the criteria used to assess evidence and determine value are valid but incomplete or inadequate. The gap between the assessment of evidence and the appraisal of technologies seems to give rise to a fragmented decision-making process. These results highlight the limitations of current value assessment approaches mainly with regards to the incomplete and implicit set of evaluation criteria considered.
2. Criteria driving HTA decisions: A Retrospective Analysis Framework
Beyond the state of the art in value assessment, this WP has aimed to understand the criteria driving HTA decisions based on a retrospective analysis of these in four countries (England, France, Scotland, Sweden). A methodological framework was developed and subsequently piloted and tested, that enables to systematically identify and compare the decision-making criteria across countries and therapy areas (see Figure 1 in the appendix). This allowed for an enhanced understanding of these decision processes in different settings. Each step undertaken to set up and test this framework was described in detail to ensure its transparency and also transferability to third parties, by means of a detailed coding manual and case study template. Its application to analysing a sample of drugs in several countries facilitated the identification of taxonomy of criteria that may have contributed to the decision process, as well as of similarities and differences across countries. This was then used to raise awareness about how value assessments are conducted in different settings and identify those cases where differences are a consequence of a particular process captured by this framework, or by limitations in applying HTA. A sequential exploratory mixed-methods qualitative design was used to develop the framework, dividing the research process into three distinct phases. First, case study analyses were piloted in order to identify the range of criteria accounted for and the structure of the decision-processes, such that they were comparable across countries. Second, the structure of analysis was used to establish the coding manual and case study template, which formed the basis of the methodological framework, used to inform the qualitatively thematic coding of all drugs and countries. This enabled for coding to be homogeneous, systematic and flexible given the iterative process adopted. Third, qualitative data was quantitatively analysed in order to understand and measure the level of agreement at each stage of the decision-process and reasons for differences, as well as identify agency-specific preferences (risk and value preferences). Data sources consisted in the publicly available documents containing the specific HTA recommendation, which will be referred to as the “HTA report”. Based on the assumption of transparency, the information reported in the HTA reports was considered to reflect the key reasons around the final decision.
The different phases of the decision-making process included in the methodological framework comprised (a) the clinical and economic evidence submitted by the manufacturer or produced by the HTA body, (b) the assessment of the evidence including the uncertainties raised, stakeholder input and “other considerations” influencing the decision, also as part of the deliberative process, and (c) to what extent these actually influenced the final HTA recommendation usually in the form of a positive or restricted recommendation or a negative recommendation for coverage.
The main originalities of this research are: (a) the applicability of a new methodological framework to other countries and disease areas given the iterative and flexible approach adopted; and (b) its innovative design through the integration of both a vertical component (decision-making criteria) and a horizontal component (reasons for differences across countries). The vertical component entails coding each individual drug-indication assessment processes in a systematic, homogeneous and comprehensive manner (based on an existing and flexible coding manual) in order to quantitatively devise - through correspondence analysis - agency-specific preferences in the type of evidence, concerns (e.g. risk preferences), and “other considerations” (e.g. value preferences) identified. The horizontal component of applying the framework enables to capture how these processes compare across countries, whether the criteria had a positive or negative influence on the decision, where the criteria put forward came from (e.g. stakeholder input), and whether the criteria were one of the main reasons for the final recommendation. Quantitative analysis (through Cohen’s kappa scores) of these allowed to measure the degree of agreement across countries in the criteria accounted for.
3. Application of the Retrospective Analysis Framework: Evidence from Oncology and Orphan Drugs
The Retrospective Analysis Framework was applied to a sample of cancer and central nervous system treatments. Twenty-one drug-indication pairs were systematically analysed and compared (13 cancer, and 8 central nervous system (CNS) across four study countries – England, Scotland, France and Sweden resulting in 74 drug-indication pair observations across the 4 countries [N=74]). 14/21 drug-indication pairs received diverging recommendations: 11 out of 13 in cancer indications, and 3 out 8 in CNS indications received diverging recommendations.
When comparing the evidence, the same primary trials were appraised for the cancer drugs (and therefore the clinical assessments made were considered comparable), and a high level of heterogeneity in the evidence appraised was found in the CNS sample, particularly for four of the drugs (Alemtuzumab, Fingolimod, Retigabine and Natalizumab). In terms of the economic assessments (for NICE, SMC and TLV), they were deemed partially comparable when the same types of models and comparators were appraised. The primary clinical trials considered were mainly phase III trials, with the exception of two cases that included phase II trials (2/8 primary trials in the CNS sample) and one case that included indirect comparisons. The comparators used were either another treatment, standard of care, placebo, or no other treatment. Important differences were seen in the level of reporting these same primary trials, which partially explained some of the differences across countries (e.g. Eltrombopag, Mifamurtide, Trabectedin, Erlotinib, Rituximab, Pazopanib and dimethyl fumarate).
Focusing on the interpretation of the evidence presented, a number of value judgments were made about this evidence. One of the main contributions of this research was that it was possible to capture these additional (softer) dimensions of decision-making and their influence on the decision. This was facilitated through the double coding conducted for all uncertainties identified with: (a) whether the evidence that was considered was accounted for by the other agencies, (b) whether this concern was addressed or not (and through what means), and (c) whether it was one of the main reasons put forth for the decision. On this basis, it was possible to measure how different HTA agencies dealt with evidence considered by all by comparing the types of issues raised. For example, the similarities in the concerns identified were even lower, with 2.5% of the concerns raised for the cancer drugs common across all agencies (n=162 cumulative number of concerns) and 2.7% for CNS drugs (n=11 cumulative number of concerns), highlighting a lack of common pattern in evaluating the evidence. Further, uncertainties were addressed through various means: stakeholder input was used to confirm the plausibility of a (n)(uncertain) clinical claim; or the uncertainties were raised but nevertheless considered acceptable by Appraisal Committees. Raising awareness about how these issues were dealt with across settings may provide insights on alternative ways to address these through cross-country learning. Social value judgments were captured by coding all the emerging “other considerations”, together with their influence on the decision and whether they were provided by the different stakeholders (through double coding). The main contribution of this research from applying the framework is that it enables to differentiate elicited social value judgments (and preference explicitly given) from those that were not elicited but were nevertheless raised by the decision-makers.
The last step was to examine how the evidence and its interpretation influenced the decision. Findings show that there is generally not a single main reason for having different HTA recommendations, but a number of reasons and circumstances that can be identified at each step of the process. The reasons for a certain decision or for different decisions across countries are, therefore, a combination of these three components rather than one criterion further emphasising the complexity of HTA processes and the multiplicity of criteria beyond clinical-cost-effectiveness that feed into decision-making processes as identified by this research.
4. Using MCDA principles to measure overall value in the context of HTA: The Advance Value Framework
Given the variety of criteria that feed into decision-making processes and the fact the clinical & cost-effectiveness analysis may account for only part of a new intervention’s overall value, this component of the WP departed from these principles and developed a model based on decision analysis, more specifically, multiple criteria decision analysis (MCDA). This was undertaken in an attempt to determine whether and how additional dimensions of value and stakeholder preferences could be incorporated in the decision-making process explicitly rather than implicitly.
In the context of HTA the MCDA process developed could be divided into five distinct phases (see Figure 2 in the appendix), involving (a) problem structuring, (b) model building, (c) model assessment, (d) model appraisal, and (e) development of action plans. Problem structuring involves an understanding of the problem to be addressed (i.e. defining the aim of the analysis, as for example ranking a set of drugs for a particular indication based on their overall value). This includes key concerns, envisaged goals, relevant stakeholders that may participate in or contribute to decisions, and identification of uncertainties in terms of a new technology’s clinical evidence and its quality. In the phase of model building criteria selection takes place, while being consistent with a set of assumptions, followed by the phases of model assessment and model appraisal, which involve the construction of decision makers’ value judgements across the criteria, aiming to help decision makers identify the most preferred solution(s) as well as the intensity of individual preferences. Finally, given that the outcome of the analysis needs to inform decision-making, action plans need to be shaped involving a clear pathway for result implementation. In the case of HTA this could involve the policy-making actions relating to coverage that take place following the evaluation of a new technology.
The criteria selection and their hierarchical organisation in the form of a tree, known as value tree, were completed in parallel during the model-building phase following a four-stage process aiming to limit subjectivity to a minimum through an iterative process of evidence identification, supplementation, revision, and validation. This involved (a) a systematic review of the value assessment literature in the context of HTA; (b) a second more targeted examination of the methodological literature by focusing on known initiatives in the wider health care decision-making context while also capturing some grey literature, including the benefit-risk assessment of new drugs from a licensing perspective, pricing and coverage decisions from a payer perspective and patient access of orphan medicines from a social responsibility perspective; (c) a consultation with HTA experts that acted as Partners and Scientific Advisory Board members of the Advance-HTA project; and (d) a number of dissemination activities through the Advance-HTA project. Criteria were structured in the form of a tree to reflect the values under consideration, offering an organised overview of the various concerns at hand. Ultimately, the resulting value tree (Advance Value Tree©), consisted of criteria clusters relating to five key groups of parameters that can be explicitly quantified and assessed (see Figure 3 in the appendix): (a) burden of illness, (b) therapeutic impact, (c) safety profile (d) innovation level, and (e) socioeconomic impact.
The Advance Value Framework© is a new value framework that has been developed specifically based on MCDA principles and adapted to the needs of HTA. It addresses the following methodological aspects: (a) with regards to the Decision Context, it adopts a societal perspective; however, the choice of perspective can be adapted to local circumstances and choices by the decision maker. (b) With regards to Model Building, it assesses the value of a new medical technology through the set of criteria outlined as part of the Advance Value Tree©; specifically, there is incorporation of social value judgements relating to burden of disease, innovation level and socioeconomic impact, in addition to therapeutic impact and safety, all of which can be captured explicitly; (c) in terms of model assessment it implements a combination of (i) a value measurement method relating to multi-attribute value theory (MAVT), and, more precisely, an indirect technique for the elicitation of preferences in the form of value functions (MACBETH), (ii) a swing-weighting technique, and (iii) an additive linear aggregation approach. This combination of techniques is used because of their comprehensiveness and methodological robustness, but also their ability of reducing ambiguity and motivational bias.
5. Application of the Advance Value Framework: Case Studies on Cancer
The framework was applied and the value tree was adapted in the context of two case studies. In both case studies the decision support software M-MACEBTH, based on the MACBETH approach, was used to build the value tree, elicit the value functions for the different criteria in order to score the options, weigh the criteria through a swing weighting approach, aggregate the scores and weights using an additive aggregation, and conduct sensitivity analysis.
An experimental case study was conducted as a first proof of concept, applying the methodological framework in practice, for assessing the value of alternative options for the treatment of metastatic colorectal cancer (mCRC) following first line chemotherapy. For this purpose, a decision conference workshop was organised with the participation of a wide range of stakeholders. Metastatic colorectal cancer was chosen because of its severity, the availability of several expensive alternative treatment options, and the fact that it has been the topic of appraisals by several HTA agencies, including the National Institute for Health and Care Excellence (NICE). The framework was used to rank three therapies (Cetuximab, Panitumumab, and Aflibercept with FOLFIRI).
In a value scale of 0 to 100, where 0 corresponded to a lower reference level reflecting “satisfactory performance” and 100 to a higher reference level reflecting “best performance”, Cetuximab scored the highest overall value score of 51.8 followed by Panitumumab with an overall value score of 45.2 and Aflibercept in combination with FOLFIRI with an overall value score of 17.4. These numbers represent a value index score reflecting the performance of the alternative therapies across all the criteria of interest while being adjusted for their relative importance. In terms of total weights across criteria clusters (out of 100), therapeutic impact cluster attributes totalled overall a relative weight of 46.5 (three attributes), the safety profile cluster a relative weight of 23.3 (single attribute), the innovation level cluster attributes had a relative weight of 18.6 (four attributes), and the socioeconomic impact cluster had a relative weight of 11.6 (single attribute). By incorporating the total purchasing costs of the different drugs (including their administration costs), their overall value scores versus total costs plot was produced. By using rounded up total cost figures and dividing them with overall value scores, the costs per MCDA value unit were calculated to be £348, £598, and £1,693 for Cetuximab, Panitumumab and Aflibercept in combination with FOLFIRI, respectively. Therefore, in terms of value-for-money Aflibercept in combination with FOLFIRI is shown to be dominated by Panitumumab, both of which are shown to be dominated by Cetuximab, which is associated with the highest overall value score and the lowest cost.
A second proof of concept case study was conducted, applying the methodological framework for assessing the value of treatment options used in the management of metastatic prostate cancer (mPC) following first/second line chemotherapy. In contrast to the first case study, which elicited preferences from a wide range of stakeholders, this case study utilised the value judgements of an actual HTA agency. For this purpose, a decision conference workshop was organised with the participation of assessors from the Swedish Dental and Pharmaceutical Benefits Agency (Tandvårds- & Läkemedelsförmånsverket, TLV). Metastatic prostate cancer was chosen because of its severity, the availability of several and expensive alternative treatment options, and the fact that it has been the topic of appraisals by several HTA agencies, including TLV and NICE. The treatments under consideration were Enzalutamide, Abiraterone and Cabazitaxel.
Enzalutamide scored the highest overall value score of 58.8 followed by Abiraterone with an overall value score of 18.9. Cabazitaxel scored an overall value score just above zero (0.3) mainly because of its “negative” performance in the treatment discontinuation attribute. In terms of total weights across criteria clusters (out of 100), the therapeutic impact cluster had a relative weight of 44.5 (3 attributes), the safety profile cluster had a relative weight of 33.3 (2 attributes), the socioeconomic impact cluster had a relative weight of 14.8 (single attribute), and the innovation level cluster had a relative weight of 7.4 (two attributes). By incorporating the total purchasing costs of the different drugs (including their administration costs), their overall value scores versus total costs plot was produced. By using rounded up total cost figures, the costs per unit of MCDA value were calculated to be £419, £1,159, and £89,348 for Enzalutamide, Abiraterone and Cabazitaxel, respectively. In terms of value-for-money Cabazitaxel was shown to be dominated by Abiraterone, while also being very close to be dominated by Enzalutamide (£500 difference). Enzalutamide on the other hand was associated with a higher cost (£2,500 difference) and a higher value score (39.86 difference) compared to Abiraterone.
6. Recommendations based on primary and secondary evidence
The Retrospective Analysis Framework has shown that both the similarities and differences in HTA recommendations across settings are not random, but can be explained through the parameters developed in the framework. Important policy implications have also arisen. First, the framework captures the full range of criteria accounted for during decision processes, systematically and in a comparable manner. Second, it raises awareness about cross-country differences and potential methodological improvements; this is relevant in light of on-going efforts to harmonize HTA processes at European level. Third, a better understanding of the criteria driving these processes and how value is assessed in different settings can be useful for all relevant stakeholders involved in the process. For example, it enables the identification of the criteria across all drugs, therapy areas, indications, countries, which could be useful for other applications such as within multiple criteria decision analysis. It allows an understanding of how scientific and social value judgments (e.g. elicited or not) are applied and how the types of agency-specific preferences are factored in. The application of “other considerations” can contribute towards greater accountability for reasonableness and help define social values. Finally, it sheds light on the type of input needed from stakeholders (e.g. patient or clinical experts) and how stakeholders may influence the process.
In the case of the Advance Value Framework and compared to economic evaluation, the resulting aggregate metric of value emerging from the MCDA process is more encompassing in nature as multiple evaluation criteria are informing its composition. By adopting this value index metric for each option as the benefit component and incorporating the cost of purchasing the different options, the incremental cost per incremental value ratio(s) (ICVR) could act as the basis for resource allocation in a way comparable to that of an incremental cost effectiveness ratio (ICER): options with lower ICVRs would be interpreted as more valuable and efficient and would as a result be prioritised.
The resulting value index and its value scores would be context-specific reflecting stakeholder preferences: first, the value index incorporates value judgements and preferences for a set of options based on a group of criteria, all of which are informed through stakeholder input. Second, the MCDA process, as proposed, respects stakeholder preferences in individual settings, whilst reducing decision heterogeneity by introducing clarity, objectivity and greater transparency about the criteria based on which decisions can be shaped. Third, MCDA has the potential to be used in assessing the value of health care interventions in the context of HTA, mainly because of its robustness in terms of the multiplicity of criteria that can be used to assess value, its flexibility in terms of differential weightings that can be applied to the identified criteria, and its encompassing nature in terms of extensive stakeholder engagement.
Given that MCDA is a departure from conventional HTA approaches, a roadmap would be needed on how MCDA could be factored in current HTA practices. First, any efforts of MCDA implementation should start by building a multi-disciplinary research team with the appropriate technical expertise as part of an education phase, spanning the fields of decision analysis, medical and life sciences, health economics, and statistics. Second, a number of hypothetical pilot studies could be carried out in a testing phase, which would act as testing exercises, in order for the HTA agencies to gain a first-hand experience on the technical aspects of the MCDA process and exploring a variety of MCDA methods and techniques in the first instance. Third, actual case studies could subsequently be conducted as part of a transition phase, using evidence from past health technology appraisals. Value judgements and preferences could be elicited from actual members of an agency’s appraisal committees while acting as participants in decision conferences taking the form of facilitated workshops. The results could be used to highlight any differences in recommendations with past appraisals, realising the benefits and insight of the methods used as part of real practice. Finally, the MCDA approach could become fully operational as part of an execution phase, initially running in parallel with any existing formal appraisals.
Work Package 3 (WP Lead: ISS, Italy)
• WP3 sought to understand whether rarity has an impact on the assessment of the incremental cost per QALY of orphan drugs compared with drugs for more common diseases and whether it impacts mainly on the incremental costs or on the incremental QALYs. Additionally, it seeks to develop and validate a framework to support decision-making relating to orphan drugs for rare diseases, by means of a Multi-Criteria-Decision framework, more specifically using a discrete choice experiment (DCE) and, through that, canvassing views of decision-makers and patients.
1. HTA of Orphan Drugs: How are Countries Dealing with Rarity?
Through the application of a methodological framework (Retrospective Analysis Framework) developed within WP2, the systematic comparison of 35 drug-indication pair appraisals enabled a better understanding of how different HTA bodies conduct value assessments for orphan drugs, and whether orphan drugs have a special status. The methodological framework adopted a mixed methods research design comprising two stages: (1) qualitative in-depth analysis of the decision-making processes; and (2) quantitative identification of agency-specific risk and value preferences through correspondence analysis, and agreement levels across countries through Cohen’s kappa scores. The framework was developed on the basis of in-depth case study analyses and the creation of a coding manual, and was applied to the same sample of orphan drugs in four countries (England, Scotland, Sweden and France (N=35)). Orphan drugs were selected because they generally undergo the same process as drugs to treat non-rare diseases, but are by definition not cost-effective because of the challenges in producing robust evidence due to small patient population and their heterogeneous nature.
Analysis of the study drugs on the basis of the analytical framework resulted in the following: (a) identification of reasons for differences at each stage of the process across countries, and (b) identification of criteria driving HTA decisions for this sample of drugs. First, levels of differences across countries were quantified using Cohen’s kappa scores, and a difference and difference matrix was created to highlight those cases where differences at each stage of the process were associated with differing HTA outcomes. Second, agency-specific risk and value preferences were quantitatively derived using correspondence analysis and were defined as the criteria driving HTA decisions in each country.
The application of this model on orphan drug decision processes enabled to understand the factors, considerations and criteria (whether these are HTA setting-specific, evidence-specific, or related to further considerations) that contribute to decisions about coverage in different policy settings for orphan drugs. Specifically, the evidence accounted for in the decision was compared and instances where different levels of evidence were associated with different HTA recommendations across countries were identified. The issues about the evidence presented and raised by the HTA bodies were also coded and categorized per type of concern (Nuncertainty=124). Agreement in the concerns raised (whether or not they raised the same concern) was poor to less than expected by chance (-0.3< κ <0.08); and in cases when the same concern was highlighted, agreement was moderate to less than expected by chance (-0.50< κ < 1.0) depending on the agency in question. Agency-specific risk preferences were also identified through correspondence analysis showing whether one country/agency was relatively more likely to raise a concern than another, also explaining some of the differences in HTA recommendations seen. This contributed to the identification of the types of concerns raised and how they were addressed across countries, facilitating cross-country learning particularly when the issues dealt with relate to limitations in HTA methodological approaches.
The scientific and social value judgments made that influenced these processes were also identified (Njudgments=125), and pertained to characteristics related to living with a disease or having a treatment. A classification framework was developed based on the literature and findings from this study, differentiating for (non-elicited) social value judgments and (non-quantified) scientific value judgments. Results also highlighted whether and how these value judgments influenced the decisions, and if they were based on stakeholder input. This contributed to a better definition of the determinants of social value, improving the lack of accountability for reasonableness and their transparency and consistency across decisions, and furthering the debate as to whether orphan drugs have a special status.
HTA bodies were subsequently interviewed to validate the findings and collect further insights about a number of methodological issues identified during the comparison. An interview topic guide was developed based on previous findings with open-ended questions. Qualitative thematic data analysis was applied to the interview transcripts using the Framework Approach. Eight HTA body representatives were interviewed between March and June 2015. Evidentiary requirements and approaches to dealing with uncertainty were discussed around: (a) trial design, (b) population and duration, (c) comparators, (d) relevant endpoints and (e) economic modelling. There seemed to be agreement that decisions regarding orphan drugs are made in a context of greater uncertainty. The threshold of acceptable uncertainty varied by country and was generally not related to the risk of not marketing the drugs. The acceptability of surrogate endpoints was not consistent across countries nor were the validation requirements. Different mechanisms were used to modulate the ICER in cases of uncertainty (e.g. Patient Access Schemes). Some countries require higher evidentiary standards for greater clinical claims, which may be more challenging for orphan diseases. The most common social value judgments identified related to innovation, disease severity and unmet need. Trivial differences were seen in the way these concepts were defined and accounted for across countries. Although agreement was seen in evidentiary requirements or preferences, there were subtle differences in the circumstances where uncertain evidence may be considered acceptable, possibly explaining differences in HTA recommendations.
This component of the WP showed substantial differences across countries in their ways of assessing value and in the different means to deal with uncertainty (e.g. stakeholder input, other considerations). The nature of the evidence presented was typical of the type of evidence generated for orphan conditions, namely relatively small trial populations, surrogate and subgroup data, lack of comparators or non-phase III trials. Although this lower level of evidence is specific to – but not limited to – rare conditions, this component of the WP showcased the different ways seen in dealing with it. In some instances, more flexibility was given because of the situation of the disease (e.g. recognised difficulties in recruiting sufficient patient numbers, orphan status) or through other means (e.g. managed entry agreements). There were also other circumstances, where this had a negative effect on the decision. Mixed scenarios were seen across and within countries, emphasising to a greater extent the complexity of these processes, where it was not possible to conclude whether orphan drugs deserve special status in practice. Despite this, findings did highlight various ways identified in dealing with these issues related, but not limited, to rare diseases.
2. A DCE to understand decision-makers’ perceptions
In order to understand whether rarity has an impact in assessing the cost effectiveness of orphan drugs compared with drugs for more common conditions and whether rarity impacts mainly on the incremental costs or on the incremental QALYs, we used the evidence collected on costs and effectiveness/quality of life across a set of orphan conditions in order to quantify the socio-economic costs and Health Related Quality of Life (HRQOL) of patients affected by rare diseases. A methodological analysis has been carried out in order to understand how various factors might alter relevant cost-effectiveness thresholds. A systematic literature review was performed to identify the attributes to conduct a discrete choice experiment (DCE) survey across several countries and on decision makers and patients with rare diseases to identify the criteria leading HTA decisions related to rare diseases. This study revealed that attributes such as improvement in health, cost of treatment, side effects, waiting time, severity of illness, availability of other treatments and value for money should be considered to better capture societal preferences and their implications for HTA. Taking into account this information, a DCE survey was designed and applied to a representative sample of 199 decision makers, using an online questionnaire, across five EU Member States (England, France, Germany, Italy and Spain). The data obtained from the DCE survey was used to estimate the weights associated with each of the criteria by means of regression models. Logic and probit model coefficients have been tested. Choice data were used to consider the relative importance of changes across attribute levels, and to model utility scores and relative probabilities for the full set of combinations of attributes and levels. The five country models show preferences for some attributes over others. “Cost of the treatment” (Spain, France, England and Italy), “improvement in health” (Germany, France, England and Italy), “value for money” (England, Germany and France), “availability of other treatment” (England and France), “waiting times” (Germany, Spain and Italy), “side effects” (Italy) and “beginning of life” (Germany) are the attributes receiving the highest attention, while “importance of the disease” (France, England and Germany), “value for money” (Spain), “availability of other treatment” (Italy and Spain), “waiting times” (France and England) “beginning of life” (Italy) and “side effects” (France and Spain) were found to be less important. Further analysis could be made in terms of finding classes of respondents with similar patterns using latent class models when sample sizes allow this.
3. A DCE to understand patient perceptions
A similar DCE survey was undertaken amongst patients by furthering the knowledge and testing empirically the effectiveness and societal value of orphan drugs in patients with two rare diseases: cystic fibrosis and haemophilia. Patient registers can help in the assessment of effectiveness and societal value. A specific DCE questionnaire available online was elaborated to collect information from 54 patients with cystic fibrosis and haemophilia in Italy about the societal values for orphan drugs with a view to further eliciting preferences across a number of rare conditions. The findings provide evidence about how patients with cystic fibrosis and haemophilia think that decision should be made in Italy when considering which health technology scenarios are more appropriate to receive funding. “Improvements in health”, “the cost of treatment” and “value for money” were the attributes receiving greatest attention from patients with rare diseases, while “importance of the diseases” and available of other treatment” were less important for patients with rare diseases. The findings presented provide valuable evidence for future designs following similar approaches. The DCEs conducted in this study provided valuable insights regarding benefit valuation and are therefore useful as additional information to complement QALYs when assessing health care interventions.
Work Package 4 (WP Lead: IER, Slovenia)
Recent literature does not provide an irrefutable answer on whether differences between patient and general public preferences exist. There are some empirical studies demonstrating no significant differences between patient and general public sets, while others have shown that although preferences were dependent on whether they were derived from the general population or from patients, QALY gains were similar between groups. The evidence of the differences between the sets has however been growing. In this work package hypothetical values of the general public to hypothetical values of the patients were compared. This was a departure from debating experience-based vs. hypothetical health state valuation. Specifically, the WP generated new datasets from: (a) face-to-face interviews with Spanish patients with a diagnosis of breast cancer or rheumatoid arthritis using the EQ-5D-5L, and (b) surveyed patients on quality of life across countries and diagnoses.
1. Preferences drawn from general and patient populations
The advantages and disadvantages of the preferences drawn from the general population and patient populations have been defined based on literature review. According to the groups arguing in favour of preferences being drawn from the general public, their major advantage is the fact that they better represent taxpayers and potential patients. Besides, they are supposed to be behind the “veil of ignorance”, meaning that they are blind to their own self-interest, therefore establishing principles of justice which would ensure fair distribution of resources. The review, however, revealed that this argument is questionable and does not sustain theoretical justification: a thick veil is in conflict with the requirement that members of the community must be at least partially informed about the health status in order to be able to attach utilities to it. The next argument connects to the issue of adaptation - utilities elicited from patients are supposed to be higher than those from general population due to adaptation to disease. Adaptation has many sources, two of them being cognitive denial and suppressed recognition. They both are seen as negative aspects of the process of adaptation as they relate to difficulty of admitting how poor a patient’s health really is. The other sources of adaptation are skill enhancement, activity adjustment and substantive goal adjustment, which are seen as positive for patients. In case adaptation is comprised primarily of cognitive denial or suppressed recognition of full health, it becomes an inappropriate influence on preferences and health care allocation priorities. The other arguments to support general population preference elicitation seem to be more practical: certain patient groups are unable to attach utility to health states, there seem to be not enough patients in certain diseases as well as patient preferences cannot be used for comparisons across interventions due to their differences. On the other hand, the most obvious and most commonly mentioned argument which supports the use of patients’ preferences is that the general public lacks the experience with impaired health that forms a valid basis for valuation. This problem and its effects on ratings can never be entirely overcome. Furthermore, it is supposed to be easier and far less expensive to survey a convenience sample of patients than it is to survey a probability sample of an entire nation’s population.
2. EQ-5D-5L administered to breast cancer and rheumatoid arthritis patients
Based on HRQoL data, acquired from face-to-face interviews of 611 breast cancer and rheumatoid arthritis patients in Spain, following the Euroqol (EQ-5D-5L) protocol and achieved with the direct assistance and involvement of Euroqol, a database was created. The analysis of the data revealed that patients, regardless of the disease, have similar preferences towards hypothetical health states, yet different to the general population. Comparison of health states ranked by severity (from best to worst) shows that eight better health states are valued higher by the patient population, while the last eleven states hold lower average values compared to the average values in the general population. While the difference between the utilities did not differ a lot for health states where the patients’ valuations were higher, the difference was bigger for those health states, where the patients’ valuations were lower (worse health states). The average absolute difference between general and patient population is 0.232 while the average absolute difference between RA and BC patient group is 0.073.
The acquired data also enabled the examination of the causes of the established differences between general population and patient group preferences. We found statistically significant differences between patients’ valuations and general public valuations on three health dimensions: mobility, pain or discomfort, and anxiety or depression. Patients consider problems related to mobility less problematic, while problems related to pain/anxiety are seen as more problematic. The greatest disagreement is on the dimension of anxiety or depression. The results suggest that not all health dimensions defined by EQ-5D are equally well understood by the general public. A possible explanation for the differences in the valuation that is suggested by the regression analysis is that problems with mobility, self-care, and usual activities are easier to imagine and understand by the general population. The easiness to understand and visualize problems on these three dimensions might explain why differences do not exist or are not as large as those with pain/discomfort and anxiety/depression dimensions. The problems on last two dimensions, with their non-tangible nature, are more difficult to imagine to general population behind a veil of ignorance.
In order to define the advantages and disadvantages of patient and general population preferences, we examined the agreement of arguments given in the literature with our results. Our results show quite the opposite picture to the findings from the literature reviews. We found no significant difference in preferences between the groups of patients with different diseases. Therefore, the most commonly cited advantages of general population preferences seem to be irrelevant: the small number of patients in certain patient groups, patient being unable to answer and absence of comparison possibilities across patient groups are no longer a valid argument. The results have also shown that adaptation in the case of HRQoL no longer possess a normative problem, as the effects of adaptation on hypothetical health states valuation are trivial. In our opinion and suggested by the results of the analysis the differences exist not because of adaptation but because patients have experience with the impaired health and can more accurately imagine hypothetical health states. For that reason, preferences based on patient population will not exhibit higher utilities and hence lower priority with respect to treatment, but on the contrary, with the exception of mobility, receive more priority.
Regarding the argument stating that the general population is the preferred source of preferences because they are behind a ‘‘veil of ignorance’’ and blind to their own self-interest, one would expect to observe the highest differences on a health dimension with largest prevalence discrepancy (usual activities). However, no statistically significant differences were observed on that dimension between general population and patients. It seems that a “veil of ignorance” is too thick and fails the requirement that community must be at least partially informed about the health status of individuals as the general public lacks the experience with impaired health that forms a valid basis for valuation. The argument of easier and less expensive survey of convenience samples of patients does not hold in our research as we studied hypothetical health state utilities elicitation by patients and not their own health state valuations. In order to elicit utilities for hypothetical health states, sample size is not smaller and the elicitation process seems to be even more expensive due to special training required for interviewers eliciting preferences from patients.
As far as it is safe to claim that preferences based on patient and population values differ, further data would be needed to determine differences in health state utilities for all health states. The results of our analysis suggest that the differences in preferences stem from patients being more able to accurately imagine “non-tangible” dimensions of health states (anxiety or depression and pain or discomfort) than the general population with less experience in various health states. Consequently it is recommended to use patient based value sets for resource allocation. A departure from general population value sets to patient-based value sets would result in a change of priorities. According to the patients, treatments of conditions/health states that mostly affect mobility should receive less funding, while treatments for diseases, where the dimensions of pain/discomfort and anxiety/depression are more relevant, should receive higher funding compared to the current decision-making processes where preferences of the general public are used to set priorities.
In order to further extend and test the results of our study the whole value set based on patient population preferences should be calculated and tested in a pre-selected HTA model for diseases where health states have various stress on dimensions: either on mobility or on pain/discomfort and anxiety/depression. The differences in impact of using both preference sets should be closely observed and differences determined. As the elicitation of preferences from patients is much more demanding and difficult as the elicitation of preferences from general population, possible cross mapping could be considered in order to deter the need for patient preference elicitation. A question remains open of comparability of patient preferences across countries. Further research is needed to determine whether patterns and relations to general population sets across countries are the same as in Spain.
3. Main findings from the online QoL survey
767 web-survey responses, across 38 countries and 122 disease areas demonstrated that patients had an average utility loss of 24% compared to the general population, although disease-specific health state utility (HSU) variations were also highlighted. The survey results regarding average utility losses mirrored the results reported earlier in the context of face-to-face interviews among patients. Blood cancer and asthma patients had an average utility loss of 12%, in contrast to Ehlers Danlos Syndrome (EDS) and Rheumatoid Arthritis (RA) patients for whom utility losses amounted up to 53% and 27% respectively. The highest utility loss was exhibited by EDS patients, shaping a recommendation for health-care professionals to take a greater interest in understanding the widespread manifestations of this rare disease.
QoL deterioration was largely attributed to the domains of Pain/Discomfort and Usual Activities, where problems of all severity levels comprised 78% (16% severe and/or extreme) and 71% (15% severe and/or extreme) of all responses for each domain respectively. On the contrary, 76% of individuals were problem-free in the dimension of Self-Care. Discrepancies on the extent to which each EQ-5D domain impacted upon QoL outcomes across diseases, suggested that similar EQ-5D utilities (i.e. HSU of 0.7 for both breast cancer (BC) and Myelodysplastic Syndrome (MDS) patients) reflected different severities within domains depending on the disease group under investigation (i.e. anxiety was the most and least problematic domain among BC and MDS patients, respectively).
Country-specific variation on QoL preferences was also observed, such as a statistically significant (p<0.05) health utility difference between France (0.53) and Denmark (0.78). Inherent cultural differences, country-specific health expenditure and health behaviour likely impacted upon a population’s QoL, and should be considered when developing strategies to address health inequalities both within and between countries.
It was also demonstrated that EQ-5D-5L missed important health aspects in approximately 51% of a general, chronically ill population, even though this percentage fluctuated between therapeutic areas. The representativeness of EQ-5D-5L was largely questioned by EDS (60%) and cancer (52%) patients, whereas lower percentages were observed when asthma and RA patient groups reported on important QoL aspects not captured by EQ-5D-5L (45% and 42% respectively). 17 additional QoL dimensions were identified, the most common being fatigue (19.5%) and medication side effects (9%). Finally, 57% of patients reported that aspects of their illness that had a big impact on their health changed over the course of their disease, and of these aspects fatigue was also the most commonly reported (17% of responses). Fatigue can be related to both disease and treatment and has been associated with multidimensional consequences ranging from functional impairment, interference with relationships and performance of daily activities. However, its importance as a bolt-on item to the existing EQ-5D has been controversial, underscoring the need to understand better why patients report it as being highly important to their wellbeing. Overall, it was shown that EQ-5D-5L in HRQoL measurement raises inconsistencies in capturing QoL attributes in disease-specific patient populations, however further research is needed to clarify how patients’ disease-specific utilities map to generic tools and ensure that these capture aspects important to patients.
Work Package 5 (WP Lead: TUB, Germany)
The work carried out in WP5 has provided important insights into the suitability of existing HTA assessment tools for medical devices (MDs), taking into account similarities as well as differences across different device categories. Moreover, it has clearly identified areas where knowledge and tools are lacking or can be expanded.
1. Current European HTA practices with regard to MD assessment
A systematic identification of HTA institutions across Europe involved in HTA of MDs as well as a systematic review along with a website search for specific information was carried out. The systematic search encompassed the collection of information on (a) institutional methodologies, (b) processes and (c) practices, including a search for and analysis of methodological documents.
From the identified 84 institutions involved in HTA in Europe, 47 institutions are actively involved in the commissioning or performing of HTA reports on MDs (assessment and/or appraisal). Sufficient information was not publicly available for all institutions. For example, details on how technologies are prioritized could be obtained from less than half of the institutions. There still exists variability in the understanding of what the term “MD” entails, which is also reflected in different structural, procedural and methodological elements among institutions. Regarding the institutions with publicly available HTA reports on MDs, our results show that the number varies from 1 to more than 200 assessment reports per institution surveyed.
Although a large number of methodological documents (n=52) were identified, only five institutions developed specific documents (n=9) for the assessment of MDs. Three of the nine identified MD-specific documents are intended for other stakeholders rather than for the institution’s own assessments. Interestingly, five out of nine separate documents focus on diagnostic technologies (including tests). Similarities between documents for internal use are mainly related to the type of preferred evidence and outcome parameters to be considered, appraisal of evidence quality and stakeholder involvement. Differences identified refer mainly to the assessment base and the comparator used, and thus largely reflect the different types of devices addressed (diagnostic vs. therapeutics). Institutions such as NICE and HAS address additional parameters, for example learning curves or utilization setting, as crucial elements that also need to be considered. Only few institutions made separate provisions for the assessment of MDs in their general methodological documents (e.g. HIQA). This reflects that certain methodological evaluation steps, which are described in a general methodological paper, apply to all types of technologies including MDs. Of those, some have separate sections in their documents dedicated to MDs (e.g. IQWiG) concentrating primarily on diagnostic technologies.
Despite growing recognition of their importance and related international initiatives, such as the EUnetHTA Core Model, specific tools for the assessment of MDs are rarely developed and implemented at national level. Separate additional signposts incorporated in existing general methods guides may be sufficient for the evaluation of MDs.
Although HTA methodology already considers different approaches depending on the therapeutic or diagnostic nature of technologies, other elements such as device-operator interaction, duration of patient-device contact, level of device activity or combined diagnostic/therapeutic functionality require further methodological discussion.
2. Developed taxonomy: testing and usefulness
To form a comprehensive taxonomic model the classifications included in the EU licensing process for medical devices (Directives 90/385/EEC, 93/42/EEC, 98/79/EC), the international classification for medical devices used by the Global Medical Device Nomenclature (GMDN) as well as the categorization employed in the OECD Classification of Health Care Functions and elaborated on by the EUCOMED project were analyzed in combination. A matrix in table format (see Figure 4 in the appendix) was created based on relevant aspects from the existing classification schemes, incorporating the elements of risk and role/functionality of device types. The matrix further incorporates an additional distinction between the diagnostic or therapeutic nature of devices, which can be crucial for HTA purposes.
1006 reports of different types and length addressing 1234 technologies produced by 32 European institutions from 17 countries were collected to test the plausibility of the normative taxonomic model. Overall, the distribution of identified reports on the matrix generally confirms that the taxonomy is plausible. Only one report was identified for one grey field where no HTAs were expected. The majority of reports in the sample addressed technologies from the green fields, considered of high relevance. Relatively few reports were available for the red fields considered of low relevance. Some correlation to risk level and type of device is apparent, at least for devices that are implanted or serve to assist medical professionals.
Within a survey including 16 European HTA institutions, interviewees were asked about the usefulness of the taxonomy, especially for their institutions’ own work. Many of the interviewees stated that the taxonomy could be useful. However, only four found that it could be used directly by their institutions, while seven stated that it would not be directly applicable for them. Four interviewees generally saw the taxonomy as a helpful tool but did not specify for whom. Finally, two interviewees assessed the taxonomy as not useful or were unsure about its added value. Suggestions for refinement and further development of the model were also given.
Based on the described plausibility testing in conjunction with the opinions from HTA institutions illustrated above, we believe that the taxonomy developed in this WP can be useful for HTA institutions and decision makers (e.g. MoH, insurers) alike. It can serve as a support tool to (a) select topics for assessment and (b) identify certain aspects/particularities that require tailored (methodological) approaches. Work on detailed recommendations on these aspects continues beyond the end of the project.
3. Methodological considerations with regard to the developed taxonomy
In total, fifty-five HTA reports from the large HTA report pool mentioned above, covering 6 case studies (e.g. PET/CT) were extracted and analysed with regard to specific methodological aspects to explore similarities and differences across taxonomic positions. These reports were published between 2010 and 2014 and produced by a total of 15 institutions.
With regards to dimensions of evaluation and using the EUnetHTA Core Model domains as reference, we identified that for MDs of lower risk categories safety and economic aspects seemed to be of lesser importance. For implants (e.g. cochlear implants, artificial joints), social, ethical and legal aspects seemed to be of higher importance. Organizational aspects (e.g. learning curves) appeared to be of greater relevance for taxonomic positions, which entail a MD used within a procedure.
Regarding the level of evidence a lack of high quality evidence in most cases was reported. Reports assessing implants more frequently identified high quality studies. Almost all reports, irrespective of the case study, were based primarily on direct evidence and on independent research rather than manufacturer submissions. Most institutions included both primary and secondary evidence. The majority of assessments considered an active comparator, while placebo or sham interventions were used mainly in reports of low risk devices.
With regards to the endpoints, all case study reports considered mortality, morbidity, QoL and safety. In the evaluation of ‘Risk class I’ MDs mortality seemed to play a minor role, in contrast to reports regarding ICDs. Most institutions used existing checklists/tools to appraise the evidence (e.g. QUADAS 2) but also a lack of fitting checklists, e.g. for surgical RCTs, was mentioned.
Based on the information included in the reports, the majority of institutions included stakeholders in the various stages of evaluation, except evaluations of ‘Risk class I’ MD. The types of stakeholders involved varied by agency. The type of approach for economic evaluation was agency-specific (e.g. simple mention of costs, literature review or own economic evaluation) and depended on the agency remit and potentially also financing streams for MDs.
This analysis of HTA report showed that there were some differences across taxonomic positions with regard to aspects such as acceptable level of evidence or chosen comparators in current HTA practice. Furthermore, input from HTA institutions provided in the interviews supported that a common understanding on methodological requirements and specific tools as well as further research (e.g. GRADE for prognostic studies) was vital and needed, including both the regulatory and the HTA-doers’ side.
We formulated ‘pointers’ addressing certain methodological aspects that could be applicable to specific device types (and, therefore, taxonomic cells) and could/should be considered along with the regular methodological approach adopted by each institution. Recognizing that larger samples and additional, more in-depth analysis would be required to reach unequivocal conclusions, we consider these recommendations work in progress. Three examples of such ‘pointers’ are presented below.
The first example relates to the EUnetHTA Core Model elements. The EUnetHTA Core Model™ for Diagnostic Technologies was frequently mentioned by interviewees as a methodological basis. It can serve as a basic understanding on how to assess diagnostics. The Core Model™ iteration specific to Medical and Surgical Interventions and Screening Technologies can also be used as the basis for assessing related technologies. Based on our findings, the following aspects could be taken into account for specific taxonomic positions: First, for MDs of lower risk categories: safety and/or economic aspects might not be fully touched upon; however, closer inspections of these elements maybe of importance, for example, for low cost MDs broadly used in the healthcare system; second, for certain implanted devices and devices that require user skills, social, ethical and legal aspects have to be considered in greater detail; third, taxonomic positions, which entail a MD used within a procedure require that organisational aspects have to be explicitly considered (e.g. by using OSTEBA’s checklist).
The second example relates to the selection of an active or inactive (placebo/sham intervention) comparator to be considered on a case-by-case basis . In principle, the use of an active comparator is generally recommended. Pragmatically, however, there are cases where placebo/sham interventions may be acceptable for comparison. For example, in the assessment of MDs of lower risk classes, placebo or sham interventions can be chosen without major ethical concerns or limitation in the resulting evidence. Similarly, in circumstances where an active comparator is not ethical and/or feasible, a placebo control arm might also be appropriate. In contrast, high-risk MDs should have active comparators to demonstrate comparative effectiveness and safety.
The third example relates to the quality assessment of studies. We identified certain quality appraisal checklists and tools for specific study designs that are used quite often in current HTA practice. These differ mainly between therapeutic and diagnostic technologies. Based on our results, QUADAS 2 is indicated for diagnostic studies and also recommended by EUnetHTA . For therapeutic studies, tools vary depending on study type and domain addressed (e.g. effectiveness). Non-randomized controlled studies (NRS) of lower level of evidence were often accepted in existing HTA reports. While it is acceptable to take these into account for some types of technologies (see above), we strongly recommend the use of a quality appraisal tool to evaluate NRS studies before they are included in the body of evidence. EUnetHTA recommends the instrument ‘ACROBAT-NRSI’ by the Cochrane Collaboration as the most suitable for this purpose, a recommendation confirmed by the interviews conducted in this work.
4. Regulatory recommendations based on primary data collection
Although WP5 mainly focused on the methodological implications of MD assessment, relevant potential regulatory changes also merit attention. While a broad range of institutions with different levels of experience participated in the survey carried out within this work, a uniform emerging statement was that regulatory changes needed to be made in order to facilitate and strengthen their work. Interviewees wished to have stricter regulation for MDs, which includes evidence-based requirements on effectiveness and safety for receiving marketing authorization (CE mark). More specifically, these changes should entail:
• A singular system for marketing authorization across EU countries as it already exists for drugs;
• A requirement for data including high quality studies such as RCTs but also studies addressing organizational aspects (e.g. learning curves);
• A transparent and publicly available system/database that provides an overview of all licensed MDs at the European level including marketing authorization documents, prices, instructions and indications for use as well as other information;
• Mandatory trial registration and publication: Obligation for studies on MDs to be documented in a central register and to be published, once results are available, regardless the nature of these results.
In addition, interviewees emphasized that more attention and focus on MDs on behalf of decision-makers, including a more structured and systematic planning for their introduction to the market was necessary. The need for greater interaction of all parties including manufacturers and patients but also between regulatory and reimbursement bodies was also raised. Finally, initiating a discussion about what needs to be assessed before reimbursement decisions for MDs are taken was seen as crucial. Given that European regulation is currently being revised, there is an opportunity for insights from our work, especially the opinions of 16 HTA institutions, to contribute to the discussion and hopefully help ameliorate the current situation.
Work Package 6 (WP Leads: EASP, Spain & PAHO, United States)
This WP had two key objectives, notably to map and understand the HTA activities in two geographical regions (Latin America and Eastern Europe), and to create a toolbox that brings the collective wisdom on current and likely future HTA policies and practices to settings where HTA is currently not used.
1. Mapping HTA policies and practices in Eastern Europe and Latin America
This exercise included a mapping of HTA activities in Central, Eastern and South-Eastern Europe (CESEE) and Latin America and the Caribbean (LAC) countries. This exercise encompassed the design of two surveys: (i) diagnosing capacity in HTA and (ii) HTA decision-making processes.
In relation to capacity to perform HTA, in CESEE a total of 41 responses were obtained (response rate 19.8%) and 31 from LAC (response rate 67.8%). The main activities of respondents were health policy (65.5%) and HTA (62.1%), followed by organization and management of health care (51.7%) and research (48.3%) (non-mutually exclusive options). Significant heterogeneity was observed in the characteristics of respondents (e.g. affiliation, responsibility, HTA experience). Similarities in the barriers faced by the institutions to which respondents were affiliated were observed and the most common barriers faced by those performing HTA were skills training and sources of funding in CESEE, while skills training and institutional support were the most cited in LAC. In CESEE countries, the majority of respondents reported having partnerships with domestic and international institutions. Requests to perform HTA were generally received from government (62.1%) and private companies (31%). Some respondents indicated that, at that moment, they did not perform HTA, because of a lack of political interest and funding. In LAC countries, 22% of respondents indicated that their institutions undertake HTA, while 15% reported coordinating HTA activities, and being involved in pricing and coverage/reimbursement decisions. However, a majority of respondents (30%) indicated that they participated in other activities, including policy development, regulation, HTA training, research on the impact of sanitary measures, public health research, and epidemiological surveillance.
In relation to the HTA decision-making process, data was obtained in a number of key dimensions as follows: first, technology selection and type of technology assessed for decision-making process: For both CESEE and LAC countries, the most cited criteria for selecting a technology for assessment were the frequency of the clinical condition (prevalence, incidence) and/or disease burden (mortality, morbidity and quality of life). For CESEE countries, budget impact was the third most common criteria, while this position was occupied by ethical, legal or social implications (equity criteria included) in LAC. The second dimension was potential conflicts of interest (CoI) addressed when undertaking HTA; conflict of interest may significantly affect the outcome of the HTA process. In many settings staff involved in HTA have to confirm in writing that they do not have CoI, whereas in other settings, respondents indicated that there were no established procedures to manage CoI, or that they did not know the procedure. The third dimension was how HTAs are ultimately linked to decision-making; countries with legislation establishing that HTA must be considered in the decision-making process as mandatory are Slovakia, Latvia, Greece, Poland, Czech Republic, Estonia, Slovenia, Bermuda and Dominican Republic (although in some cases respondents highlighted that there is no link and the decisions are not informed by HTA). The legislation in Estonia, Poland and Surinam also considers HTA to support coverage decisions as recommendation. Countries such as Croatia, Lithuania, Russia, Hungary, Belize, Costa Rica, El Salvador, Jamaica, Panama and Saint Lucia did not have specific legislation at the time of the survey stating that coverage decisions should be informed by HTA reports, but HTA reports have been used to support policymaking. In Serbia, Honduras, Costa Rica, Dominican Republic, Saint Maarten and Trinidad and Tobago there is no link, and the decisions are not informed by HTA. The fourth dimension of interest concerned stakeholder involvement in the decision-making process; Latvia, Poland, Czech Republic, Lithuania, Russia, Estonia, El Salvador, Suriname, Saint Lucia stated that HTA entities involve stakeholders in the HTA process, and submissions of evidence are required from stakeholders. In Poland, stakeholders are involved in the HTA assessment and appraisal stages, but not in decision-making. Only the HTA organization from the Czech Republic allows stakeholders to comment on HTA at the draft stage. The fifth dimension examined was the type of HTA sources that could de distinguished according to the institution that drafted the report; three such sources were identified, notably, internal reports produced by the Ministry of Health, Social Security, or HTA agency (which are always or often taken into consideration in decision-making in Barbados, Bermuda, St. Maarten, Slovakia, Latvia, Poland, Czech Republic, Croatia, Lithuania, Estonia, Bulgaria and Russian Federation; by contrast, in Slovenia, Honduras and Trinidad & Tobago, decision-makers never take any input from internal reports), reports commissioned externally by the Government (St. Maarten, Panama, Lithuania and Bulgaria take into consideration these reports regularly) and information presented by the industry requesting the technology being publicly subsidized or incorporated in a benefits package (Panama, Latvia, Czech Republic and Croatia considered this type of report for the decision-making process. Finally, a number of other dimensions were examined, notably: requirements for economic evaluations (required in El Salvador, Latvia, Poland, Czech Republic, Rumania, Estonia, Bulgaria, Slovakia and Hungary); and the timelines for HTA completion (ranged from 90 to 365 days).
2. Toolbox on Best Practices in HTA
The other key objective of this WP concerned the development of a toolbox outlining best practices and including recommendations for HTA, based on the information gathered in earlier phases of the research. The Toolbox was divided into five sections:
I. Healthcare system and HTA: Offers a general view of what HTA is and its use. It provides information about the functioning of HTA systems in specific settings, illustrating diverse HTA practices.
II. Building the HTA function: Offers examples of types of HTA agencies and their performance. In order to assist in the harmonization of methods and information sharing, a selection of HTA networks is shown. Recommendations, examples, and tools aiding the analysis of social or ethical values are compiled.
III. HTA products: Discusses types of evidence to inform decision-making, and the structure and contents of a typical HTA report. It offers the key elements to read and understand the different typologies of HTA reports, with hyperlinks to relevant repositories.
IV. Beyond HTA: Focuses on multiple Criteria Decision Analysis (MCDA) as a tool to facilitate priority-setting and decision-making.
V. Decision implementation: Offers tools and examples of the adaptation of guidelines, disinvestment & reinvestment decisions with country examples, and monitoring and evaluation of HTA impact.
Work Package: WP7 (WP Leads: UCLM, Spain and LSE, UK)
Work package 7 synthesized the results of the previous six work packages in horizontal and vertical terms. While the horizontal aspect outlined the results of each of the previous work packages, the vertical aspect underscored a number of dimensions related to HTA, the resulting policy implications and the broader socio-economic context in which Advance-HTA operated, notably: (a) the theoretical and conceptual considerations addressed by Advance-HTA; (b) the importance of appropriate service user involvement in HTA evaluations; (c) the orphan medicine treatments and the value of HTA in industrial policy formation; and (d) the implications for global health and health care development. These are briefly discussed below.
1. Theoretical and conceptual considerations addressed by Advance-HTA
Some of Advance-HTA’s results in general show that differences in the evaluation criteria underlying how products such as innovative medicines are assessed can (over and above issues such as variations in national per capita GDPs) consistently and significantly influence whether or not a treatment is judged cost-effective in one country as opposed to another. This does not mean that the work undertaken by health economists and others involved in HTA leads to arbitrary decisions. But it does mean that if more or less value is ascribed to, say, the scientific originality of a therapeutic innovation (and so, implicitly, the unknowable but potentially important long term returns to communities from publicly or privately funded biomedical research investment) relative to the immediately demonstrable benefits generated for individual patients, then differing purchasing and treatment supply decisions will result. Similar points apply to the use or non-use of ‘adjusting’ factors such as disease severity and rarity or ‘end of life’ care applications.
From a pan-European policy perspective this raises social equity concerns and a number of wider economic and industrial policy questions, alongside more immediate health sector resource allocation issues. One important challenge for the future relates to ensuring a better balance between achieving short-term equity in health care areas where there is already evidence available relating to the promotion of allocative efficiency and the inevitably more uncertain task of defending public interests in long-term innovation, and the eventual achievement of fundamentally improved technologies for preventing and treating conditions such as cancers and neurodegenerative disorders. These are different goals, which in some circumstances conflict. Similar questions and possible consequences arise in fields ranging from how affordability criteria are set in relation to ‘cost per QALY’ and allied utility measures in different nations and/or contrasting spheres of social and economic activity in a single country, through to – not least in the rare disease and cancer care contexts – whether or not rationing choices should be made on the basis of Benthamite utilitarianism as opposed to Rawlsian concepts of social justice. The latter are more likely to ascribe premium values to treatments that benefit minority populations than the former.
Other important Advance-HTA findings range from new observations showing that patients tend to ascribe higher relative values to physical pain and mental distress reduction than observers who have not experienced relevant disease states directly, through to revealing the limited extent to which HTA techniques have to date been applied to the evaluation, pricing and purchasing of medical devices as compared to that of medicines and vaccines. Its existence stems from the relative lack of centrally accessible data on medical devices use and the outcomes attributable to them, and the fact that their employment is often intimately associated with other forms of hospital spending. The special focus of HTA on drug evaluations is also linked to the (in some ways questionable) Western pharmacological tradition of evaluating medicines as discrete molecular entities, as opposed to items that act in combination with not only additional drug treatments but many other medical, surgical, nursing, psychological and wider social inputs.
Advance-HTA’s findings indicate that further investment needs to be made in developing Health Technology Assessment tools for use in areas outside the narrowly defined pharmaceutical sector, in order to further increase the productivity of health care as a whole. This stretches beyond conventional technologies and includes diverse health technologies such as combination therapies and companion diagnostics, among others. They may also need to be adapted to accommodate the reality that drugs do not normally act alone to achieve the best possible outcomes, albeit the discovery of optimum combinations and administration strategies can – as experience in spheres such as the treatment of child leukaemias has shown – be a long drawn-out process.
2. The importance of appropriate service user involvement in HTA evaluations
From a public policy perspective some of the strongest findings of the Advance-HTA research discussed here stem from observations made in relation to preference elicitation, and in the rare disease treatment context. These include the discovery that people who have experienced conditions such as physical pain or depression (and perhaps other psychiatric states such as, for example, severe anxiety disorders) are likely to value their alleviation more highly than those without such direct experiences. At the same time reductions made in the degree of QoL lost recorded via instruments like the EQ-5D-5L in the context of problems that can more readily be accommodated, such as being unable to conduct self-care tasks autonomously, are likely to be less highly valued by patients with relevant experiences than they are by ‘naïve’ observers.
Although the likely impact of such variations on quality of life estimations should again not be over-stated, such variations are potentially significant with regard to the ways that HTA research might best be conducted. This is particularly so when they are combined with findings that demonstrate that in areas such as rare disease research it is often the case that patients and their family members have deeper and more accurate insight into their conditions than many clinicians, let alone academic or directly State employed economists.
Given this, the key policy related conclusions drawn here are not only that there are opportunities for technical improvements in the ways in which the quality of life related impacts of health care interventions are measured, but also that closer patient involvement in the governance and direction of HTA and other forms of health care research and evaluation remains an important priority. At worst, patient involvement can be little more than a form of ‘box ticking’. But done well it is likely to add significantly to the quality of HTA findings, and the ways in which they are understood and implemented.
3. Orphan medicinal treatments and the value of HTA in industrial policy formation
Advance-HTA’s findings with regard to rare disease treatment and the affordability and access to orphan medicines highlight the fact that if evaluations include criteria which relate to their low prevalence and variables such as the severity of distress and other exceptional burdens that inheritable monogenic disorders can impose on not only individuals but entire families and, occasionally, on specific ethnic/racial minorities, then they are more likely to be judged affordable than if more general assessment methodologies or processes are used.
In linked areas like some areas of cancer care (and probably in future some forms of dementia treatment) patient access is also likely to be affected by the existence or otherwise of additional HTA evaluation modifying factors. These could relate to not only the value of demonstrating social solidarity in situations where only relatively small numbers of people are involved, but also the social fact that at given points in history some disease may be popularly seen as deserving greater investment than others. Such prioritisation judgement may be driven by subjectively perceived fears or objectively based beliefs that some sorts of innovation are more likely to prove viable than others, despite having equal potential ‘worth’.
Two main sets of policy related ideas are worth discussing here. First, the European Union, like nations such as the US and Japan, has in recent decades recognised the important health equity and industrially funded research challenges associated with developing better treatments for rare indications. Special incentives have been put in place to promote relevant activities. The EU has also encouraged Member States to develop rare disease strategies, which they have done at varying speeds and with differing degrees of excellence. But there is in the short term at least little point in bringing new treatments for rare disorders to market if they are not subsequently made available to those who could benefit from them.
Some commentators may take this to imply that in the European context more effort should now be made to facilitate the timely production of orphan medicine assessments produced to an agreed Union-wide standard and implemented in a manner consistent with the spirit of the existing European legislation on rare diseases. Whether or not this will be achievable is uncertain. But Advance-HTA clearly identifies this as a question that will demand increasing attention during 2016 and beyond.
Second, and following on from the above, present-day orphan drug supply issues can also be seen as indicative of the limitations of approaches to HTA that fail to take into realistic account industrial policy issues. These range from, for example, current and future EU-wide balance of trade and employment concerns to the impacts that major variations in the levels of volume demand for innovative products such as new medicines have on the economics of their development and their sustainable supply at what are perceived to be affordable unit prices.
Some experts may argue that such factors cannot properly be taken into account when calculating ICERs and should not be considered when interpreting their product pricing and purchasing implications. But if this is the case it severely limits the practical utility of HTA findings. There is a strong policy case for arguing that even if ‘cost per marginal QALY’ figures should continue to be calculated in much the same way as they are at present, affordability thresholds could and should be systematically adjusted to permit better quality future decision making. If this is agreed, then more effort needs to be put into the ‘science and art’ of determining affordability thresholds across Europe, and more widely.
Concerns like these can be linked back to the ‘utilitarianism versus Rawlsian social justice’ debate noted earlier. Benthamite thinking may, rightly or wrongly, taken to imply that minority interests should be sacrificed if greater overall welfare gains can be made by pursuing alternative social goals. But John Rawls’ theory of justice indicates that in some circumstances the (apparent) sacrifice of majority welfare optimisation interests in favour of meeting the needs of the least advantaged in a community can offer a more desirable path towards achieving what is generally accepted as being the fairest possible society-wide distribution of goods and the welfare derived from them.
Whether or not there is a genuinely irreconcilable divide at the heart of this apparent dilemma is disputable. But for the purposes of this analysis it can be said that the Advance-HTA programme’s outcomes point to the dangers of failing to understand the significant of such philosophical issues in the context of the ‘cost per QALY’ calculations currently central to the cost-effectiveness assessments undertaken by European HTA agencies. At worst, ‘simplistic’ utilitarianism could lead to an undermining of social solidarity and the intent of European policies aimed at ends like encouraging rare disease research.
4. Implications for global health and health care development
Advance-HTA also revealed marked variations in the numbers of qualified health economists and other HTA practitioners in emergent as opposed to more mature economies, and in the ways in which HTA based findings are used to determine policies and decision making. The world-wide picture is to a degree complicated by the position of the United States. There both industrial and allied research investment arguments, together with policy questions relating to the acceptability of using quality of life related calculations to determine care access, have had a significant influence on the evolution of HTA and attendant disciplines like comparative efficacy studies. In broad policy terms the emergence of health economics and within it the field HTA can be taken to be related to the processes of global demographic and epidemiological transition. These seem in their later stages to be normally attended by increased public (including compulsory insurance supported) spending on UHC provision. As health outlays increase, so do concerns for equity, efficiency and service excellence. So too do investments in aspects of quality management and value-for-money improvement. However, it should not be uncritically assumed that these are always appropriate or in themselves cost-effective. It is debateable, for instance, as to whether or not individual emerging economies should currently be seeking to increase their indigenous HTA capacity, as opposed to drawing on evaluative work conducted elsewhere and concentrating attention on further building their clinical workforces and generic health sector management capacities. Nevertheless, they will either way benefit from becoming more able to use health care resources to optimal effect. Advance-HTA work undertaken in the Americas identified an increasing tendency for local courts to require public health care systems to supply medicines to individuals seeking better health care. In other parts of the world judicial actions appear to have been aimed more at improving treatment access by limiting intellectual property rights. It would not be appropriate here to make judgments on the desirability or otherwise of such interventions. However, their existence underlines the importance of being able to price and supply products such as innovative medical devices and medicinal products in ways that are consistent with individual human rights and collective interests in achieving better care standards and ongoing global efforts to improve the effectiveness of treatments.
5. Overall
The Advance-HTA project’s findings offer European decision makers a wide range of useful insights into how in future cost effectiveness evaluations of not only medicines but products such as medical devices (and perhaps in time interventions such as surgical operations) could be better conducted. They also raise important questions as to the extent to which in contexts like promoting enhanced outcomes in areas of cancer care and the treatment of rare monogenic diseases the criteria used for evaluating the ‘worth’ of therapeutic innovations might be standardised across the European Union, and how regionally or globally consistent approaches to adjusting for factors like GDP and disease prevalence variations could be instituted.
Technically, Advance-HTA’s results raise questions about how the affordability thresholds used in assessments can best be set, and the degree to which the extended use of MCDA techniques would materially enhance the validity and acceptability of currently established evaluation methods. They also highlight as yet unresolved uncertainties about the extent to which national policy decisions or individual treatment choices could ever be determined on the basis of HTA findings alone.
Potential Impact:
Impact
In this section, the impact of Advance-HTA is discussed, by focusing on different dimensions of impact as it relates to the project’s objectives, notably (a) impact on methods, (b) impact on EC policies, (c) impact on EC competitiveness and (d) impact on the partners and the audiences the project has sought to influence. These are outlined below.
Impact on methods
Advance-HTA’s fundamental remit was to contribute to the advancement of methods and the development of conceptual frameworks in a number of areas where it was felt there was urgent need to do so. This has taken place in 4 very specific areas, whilst in 2 more, new evidence base has been created pointing to the need to re-think current practices and methods.
The three areas of methodological improvement are as follows: First, the Advance Value Framework using decision analysis (MCDA) principles to address value assessment, has already received significant attention both in Europe and internationally, beyond its publication. In addition to what is already included in the project materials, a number of case studies are in the pipeline with national HTA bodies seeking to understand how it works and under what circumstances they can use the framework in their respective practices. Case studies are currently under way with national insurers in Belgium, Poland, Lithuania and are being planned with France and Scotland. Second, the Retrospective Analysis Framework has been presented in numerous conferences, workshops and has also been published. The feedback received suggests that it provides an important analytical tool to determine how assessments take place in different settings and what trade-offs exist between scientific and social value judgments. As the evidence base that is included in the framework increases, so will its applicability and usefulness across settings. Third, the analytical framework that has been produced for medical devices links legislation at EU level with policy and practice, and, importantly, highlights the need for continuous work on the subject of medical devices given their differences to other technologies, the type of evidence available when coverage is sought, and the diversity of devices themselves, making a single approach to assess ‘value’ very difficult. And, finally, a toolbox has been compiled containing information, advice and recommendations for countries wishing to expand, strengthen and/or implement HTA in their jurisdictions. The applicability of the material in the toolbox is independent of country or setting and its use can be adapted to any local circumstances.
In terms of new evidence pointing at the need to re-think current practices and methods in HTA, this has been achieved in the following areas: First, in the area of patient preference elicitation, where it has been shown that generic tools may have limited capacity at taking into considerations dimensions that are relevant to patients (as opposed to non-patients, which is the case today), while, additionally, patient vs. non-patient preferences are different across a significant number of health states, resulting in different priorities in resource allocation. And, second, Advance-HTA has shown that through the use of discrete choice experiments, it is possible to derive (and ultimately take into consideration) utilities from decision-makers and patients that will be very useful in the context of decision-making.
Impact on EC policies
The results of Advance-HTA feed into an evolving landscape relating to the assessment of new medical technologies and the interface between regulation of new technologies and their incorporation in benefits catalogues in different settings. A number of changes in the regulatory framework and the requirements for marketing authorisation (adaptive licensing, conditional approval, authorisation under exceptional circumstances) imply have a concomitant knock-on effect on the way HTA is conducted, its robustness to inform decision-making and the extent to which additional sources of information are taken into account when making coverage recommendations. Whereas Advance-HTA has not been able to address all the issues that have arisen in recent years in technology assessment stemming from the increased complexity of medical technologies and their production, it clearly has created new ground in a number of fields which impact EU policy in the health sector: these fields relate to (a) a new methodological framework for value assessment and the breadth of criteria that (may need to) inform value and decision-making, (b) the importance of other considerations - as they relate to the disease or its treatment – in shaping the direction of a recommendation across therapeutic areas, including orphan treatments, (c) a methodological framework for the assessment of value in certain types of medical devices, (d) the issue of preference elicitation and the evidence that generic tools as they are currently used in decision-making processes are unlikely to capture patient experiences in their totality which may require us to look further into this to optimise resource allocation processes and (e) a conceptual framework that may be of assistance to countries or settings that do not have an HTA process or function but wish to establish one. All of the above are at the heart of health care decision-making, both at European, but also at national level. Importantly, they will all feed into the forthcoming process that will commence through Joint Action 3 in the context of EUnetHTA and will feed into its agenda. Through continuous efforts, the objective will be to provide fresh evidence on research and allied findings from work in the project’s workstreams.
Impact on EC competitiveness
Advance-HTA has dedicated a significant amount of time and resource to building links, networks as well as disseminating its results to geographical areas outside the EU and has laid the foundation for increased cooperation and knowledge transfer in the years to come. In itself, this is an indicator of Europe’s intellectual leadership in this field and contributes to its external competitiveness. Specifically, Eastern Europe – including the Russia and the former CIS – and Latin America and the Caribbean have been two geographical areas associated with the project, where networks of decision-makers have been built and extensive capacity-building and dissemination have taken place in 2014 and 2015. In addition, further dissemination activities have taken place in Southeast Asia and in China, which relate to some of the methodological frameworks developed by the project and constitute testaments to its rising reputation and legacy. The latter have taken place on an ad hoc basis and because of the project’s rising reputation in a number of fields (specifically value assessment, medical devices and preference elicitation). These activities will unavoidably lead to further contact in the years to come and form part of the overall project legacy. They also serve as indicators of the wider international leadership undertaken by Europe-based organisations.
Impact on the partners (products or services)
Several issues have emerged in this context of the likely impact on partners and the knowledge that has emerged. These are addressed below.
• First, with regards to the project outputs and what has come out of the project: the project has contributed scientific evidence, including a number of toolkits and methodological/conceptual frameworks, which are available to decision-makers and can be used in different settings and can serve the objectives of capacity-building as well as introducing more robust methods of assessment depending on context and type of technology. Importantly, all partners have contributed significantly to this process;
• Second, in what concerns the type of audience for the project outputs: the key recipients of the project outputs are health care decision-makers in EU countries, in Central and Eastern Europe, Latin America and countries in Southeast Asia and China that have already implemented or are planning to strengthen and/or implement and/or reform current practices in the HTA system they may have in place. Additional audiences include academia and the broader stakeholder community, specifically, clinicians, patients and industry.
• Third, with regards to the reason why the above audiences need the results of the project: it has become clear that new methods, tools and guidance are needed for decision-makers as well as other stakeholders to address gaps in particular themes related to HTA, for example, what criteria are used in value assessment and how do we treat social value judgments; how do we assess value in orphan drugs; how do we treat HTA in medical devices from a methodological and evidence perspective; and should patient-reported outcomes be used more intensely compared with generic preference elicitation tools and, if so, how.
• Fourth, in terms of how the interested parties learn about the results: Advance-HTA has put together a multi-faceted dissemination strategy comprising (a) academic publications; (b) policy briefs; (c) direct dissemination through face-to-face workshops; (d) online dissemination through the project website; (e) social media accounts to enable access to materials (LinkedIn and Twitter); and (f) inclusion of published or validated new knowledge into training curricula. This dissemination strategy will continue after the completion of the project. There is also willingness by the project leaders and some WP leaders to explore synergies with decision-makers in order to diffuse further the research results in the near future.
• Fifth, to the question of how appropriate the measures are for the dissemination of results: Advance-HTA has pursued a combination of measures aiming to maximise the widest possible diffusion of the research results. Clearly, the long-run credibility of the research results will emerge from their publication in high-impact peer review journals. This has been one of the top priorities of the consortium. The contribution of the HTA partners (NICE Int, TLV and AOTM), PAHO and EBC in this process has been invaluable because it has enabled the policy/stakeholder dimension to be included. A significant component has been the use of networks for face-to-face dissemination; this provides immediate benefits but also has longer-term implications for greater leveraging of such networks. Finally beyond peer review and face-to-face activities, electronic means of dissemination are available (website, social media), potentially making it feasible for larger audiences to benefit from the availability of results.
• Sixth, with regards to the management of intellectual property: there is considerable flexibility as the project has been structured around work packages where individual work package leaders and those who contribute meaningfully share the intellectual property. In a number of cases, it is expected that the rights holders will pursue applications (e.g. software or online tools) to their property.
• Finally, and based on the above, we feel that Advance-HTA has followed very closely the Commission’s work programme in terms of expected impact by linking academic rigour with policy-maker needs and sensitivities, whilst at the same time involving the wider stakeholder community both as a partner and as recipient of the knowledge created.
Dissemination
1. Overview
The primary objective of the dissemination activities is to ensure that results of the project’s research become known to and used by the HTA stakeholder community. Dissemination is meant to be continuous throughout the life of the project and is also expected to continue after the end of the project through publications and symposia (e.g. the LSE SUMMIT, taking place in March 2016), among others. The dissemination plan is meant to be a dynamic rather than static draft of planned and executed activities. The objective was split into 3 sub-components:
1. To effectively link policy makers, stakeholders and patient networks to the research evidence on HTA;
2. To define a dissemination strategy, taking into consideration the advice of the Scientific Advisory Board (SAB) as well as the recommendations of the three institutional partners (NICE International, TLV and AOTM);
3. To target groups for different results and the most adequate dissemination mode: publications, face-to-face and electronic dissemination.
ADVANCE-HTA used specific measures to communicate results in ways that would help disseminate findings and build capacity on HTA methods; these are grouped under 3 headings detailed in Section A (public) Tables: Publications and List of Dissemination Activities, which includes the relevant target groups:
a) Publications
b) Face-to-face dissemination (e.g. conferences, workshops)
c) Project website and electronic dissemination (Linkedin, Twitter, Blogs)
2. Publications
The project results have been and will continue to be disseminated through publications in print in a variety of ways, each targeting different audiences. The list of activities specific to each Work Package and partner responsible are listed in Appendix 2 – Publication plan, which includes planned publication means, timelines, authors and a short description of content.
First, publication sources have been used and will be used such as Health Policy, Health Economic Reviews, Health Economics Policy and Law, Globalisation and Health, Social Science and Medicine, Global Policy, Pharmacoeconomics, International Journal of Technology Assessment in Health Care, among others, where members of the consortium are involved in the editorial boards. Policy briefs and reports have been distributed through LSE Health working papers (on-line distribution) and the same can be with other members of the consortium. PAHO has assisted with the dissemination activities in Latin America. These outputs are intended for the broader stakeholder community. Tailor-made outputs (e.g. HTA for patient organisations) will also be created.
Second, it is the intention of the partners to pursue a special issue in a leading peer review journal in addition to any stand-alone peer review publications that emerge from the project (e.g. Health Policy). These activities target a more technical audience and a selection of articles is already under way for this purpose.
Third, one book project bringing together the conceptual/analytical framework on HTA strengthening and implementation will be published through the European Observatory. According to current plans, the book will be divided in two sections. First, details of the current state of plan will be presented to provide a mapping or framework of current HTA methodologies, emphasising possible ways forward and anticipated challenges, together with implications for policy-makers who would like to reform HTA. Second, findings will be presented detailing the broader HTA framework, with the core lessons and options for innovating HTA for its continued improvement in different settings. This output is intended for policy-makers and all HTA stakeholders.
Fourth, the needs of researchers will be met by publication of papers in academic journals in the health economics and comparative health policy fields. This is intended for a more technical audience.
Finally, other opportunities will be taken as and when they arise, such as articles in newspapers or other media, which will target the broader stakeholder community. In addition, the project will continue to leverage to the maximum extent possible the project website as well as social media (linkedin, twitter) to make these activities more widely known to relevant audiences. Further opportunities may arise through collaborative arrangements with other EU FP7 HTA-related projects or/and EUNetHTA. These will be explored during the course of the project life.
3. Face-to-face dissemination – Workshops
(a) Background
One of the main activities undertaken to achieve these goals were a number of capacity-building workshops targeting key decision-makers in Eastern European and Latin American countries. In both regions, interest in the application of HTA and in creating an HTA function in health care systems has increased significantly over the past decade. Consequently, the rationale for conducting these workshops in these two regions was, first, to provide updated information & training on the introduction and implementation of HTA in key settings and, second, to disseminate the research and policy-relevant findings of Advance-HTA into countries that have recently implemented HTA or were planning to do so. The objectives of these workshops were:
➢ Undertake HTA capacity-building in Latin America and Eastern Europe
➢ Present the HTA methodological advances developed by the Advance HTA Consortium
➢ Discuss the relevance of such methodological advances with local participants from Latin America and Eastern Europe
➢ Discuss future developments in HTA and their policy implications
➢ Foster international collaboration and create a network for exchange of expertise
(b) Concept
Four workshops were undertaken in M21 (September 2014), M23 (November 2014) and M33 (September 2015) to build capacity in HTA as well as disseminate the results of the project in Latin America and Eastern Europe. These capacity building workshops were in the form of “train-the-trainers” workshops, in order to give the participants the knowledge and tools to continue to build capacity in HTA in their own countries. Two workshops were conducted in each region. The first wave of workshops aimed to disseminate findings about the “state of the art”: where are we today, what are the methodological challenges and current and best practice examples for HTA. The second wave of workshops focused on disseminating findings from the empirical research conducted within Advance-HTA.
(c) First wave of workshops
The aim of the first wave of workshops, which were held in Warsaw (September 2014) and Mexico City (November 2014), was to give an overview of the results from what we know about the implementation of HTA to date and the rationale for conducting the research undertaken within the project. Both workshops had the same structure and were organised around 6 thematic areas, but content was adapted to reflect geographical differences and requests made by participants ex ante. The Warsaw and Mexico City workshops addressed the following thematic areas as summarised below (See table in final report).
First Workshop Eastern Europe – Warsaw, 2014
Session 1 - Introduction to health technology assessment: An introduction to the models, costs and benefits in HTA were first outlined, followed by existing processes and evidence requirements in different settings. This session finished with a presentation about the place of patients within these processes.
Session 2 - HTA experiences in South and Eastern European and lessons learned: This session started by showcasing results on existing HTA processes from a survey in South and Eastern Europe. The process of adoption and current application of HTA in Poland were then presented, followed by experiences from Hungary and Croatia.
Session 3 - Critical issues in appraisal of medical technology: Current ways to assess value for money were discussed, followed by a presentation on the application of multiple criteria decision analysis and its relevance to value assessment, based on Advance-HTA research results. The second part of this session discussed existing methods of preference elicitation, and willingness to pay thresholds.
Session 4 - HTA in context: Medical devices and orphan drugs: This next session discussed the particularities in assessing medical devices and orphan drugs and the possible ways forward. Based on Advance-HTA results, a new taxonomy for medical devices and a framework for analysing value in orphan drugs were also presented.
Session 5 - Empirical evidence on the use of HTA: Empirical evidence on the use of HTA were presented by NICE (England), as well as evidence on the application of HTA in different settings. An overview of results from the BURQOL-RD study was also presented, about the socio-economic burden and health-related quality of life in patients with rare diseases in Europe.
Session 6 - The way forward: This last session portrayed innovative models or tools to overcome some of the known limitations in the application of HTA. This included an overview of the EUnetHTA Core model, and an introduction to risk sharing agreements together with an empirical example of its use in Sweden.
First workshop Latin America, Mexico City, 2014
Session 1 - Introduction to health technology assessment: as per Warsaw 2014
Session 2 - HTA experiences in the Americas and Lessons learned: This session also portrayed results from the survey on HTA capacity in Latin American countries. Experiences from Colombia, Brazil, Mexico, and Costa Rica were presented.
Session 3 - Critical issues in appraisal of medical technology: as per Warsaw 2014
Session 4 - What could Latin American countries learn from other jurisdictions: During this session, case studies from European countries (England and Spain) were presented. Results from the survey on HTA capacity in Eastern European countries were also presented.
Session 5 - HTA in context: A number of applications of HTA were presented, namely the rapid assessments and how they support decision-making, the transferability of HTA results to other settings, as well as empirical evidence of the application of HTA in different settings. The session ended with a presentation on the role of risk sharing agreements and other mechanisms and how these approaches may help overcoming some of the limitations in the application of HTA.
Session 6 - Current patterns and future directions in HTA implementation in Latin American countries: The final session was in the form of a panel, where the current patterns and future directions in HTA implementation in Latin American countries were discussed amongst key faculty members and the audience.
Advance-HTA partners who formed the faculty in Warsaw included: LSHTM (Cairns); LSE (Kanavos, Nicod, Angelis), EBC (Mossmann), EASP (Espin), AOTM (Zawada), IER (Prevolnik-Rupel), TLV (Larrson), TUB (Busse), ISS (Taruscio), NICE Int (Ruiz), UCLM (Lopez Bastida), EunetHTA (Borlum Kristensen). A number of additional speakers were invited to share their experiences with some of the topics discussed, including representatives from the Hungarian HTA agency (GYSEMSZI) and the Croatian HTA agency (AAZ). In the Mexico City workshop, the same Advance HTA institutional partners participated, supplemented by: PAHO (Lemgruber), NICE Int (Freiburg). Policy reactions were made by representatives from the following institutions: Colombian HTA agency (IETS), Brazilian HTA agency (CONITEC), Costa Rica Ministry of health, National Institute for Cardiology in Brazil.
(d) Second wave of workshops
The second workshops were structured in a similar manner, but contained two key differences: first, they were far more interactive and included a significant amount of breakout sessions with group-work by participants followed by feedback in plenary; second, each session also included policy reactions from participants on new key findings and how these may be relevant to their own context. The six sessions that were included in the agenda in Santiago (9-10 September 2015) and Warsaw, (24-25 September 2015) respectively, are summarised below (See table in final report).. Both workshops were very similar with each other and differed only in the adaptation of the issues to local audiences.
Second Workshop Eastern Europe – Warsaw, 2015
Session 1 - Assessing Value for Money: The first part of the session touched upon the different ways of assessing value for money, while the second part of the session outlined results from the research conducted that aimed to describe the differences in drug reimbursement decisions on cancer drugs in ten European countries. The interactive case study was about discussing the most recent tools developed to assessing the value of cancer drugs, namely the Cancer Drug Fund prioritization tool in England, the European Society for Medical Oncology tool for assessing clinical benefit, and the American Society for Clinical Oncology framework for comparing the relative clinical benefit, toxicity and costs.
Session 2 - Using alternative methods to assess the value of new medical technologies: The first part of this session provided insights into the use of Multiple Criteria Decision Analysis (MCDA) in the context of HTA, while the second part presented its empirical application to a particular oncology indication. The group break out session related to a number of therapeutic options within a particular indication. Each group was given a list of criteria structured in the form of a value tree, spanning all the critical dimensions of value and were invited to choose among them.
Session 3 - HTA and rare diseases: assessing the societal value of orphan drugs: The first part of the session outlined the particularities of rare diseases and orphan drugs across the clinical development pipeline and implications for HTA. The second part of the session presented results from the research conducted within the project that aimed to identify the similarities and differences in how countries are appraising orphan drugs and how they are dealing with the common issues related to their rarity. During the breakout sessions, groups was asked to discuss the issues they are facing and how they are dealt with in their setting in order to ensure financial sustainability.
Session 4 - HTA and quality of life measurement: The first part of the session provided insights about the theoretical foundations and empirical investigation about which preferences should be accounted for (general population or patient preferences) in health care. Results from their empirical investigation were presented, in terms of the methods used to arrive to both value sets, their comparison and in case of differences, on the meaning of the differences in health policy. During the break out session, the question about which value sets should be used were discussed amongst participants.
Session 5 - Strengthening and implementing HTA in emerging settings: This session first highlighted the importance of HTA in emerging settings, while the second part described results from the mapping exercise conducted and the tool kit developed aimed at helping countries implement HTA. During the breakout sessions, the main challenges in implementing HTA were discussed. Participants were then asked to discuss having an official threshold in each country.
Session 6 - HTA and medical devices: This session, first, discussed the particularities of assessing value of medical devices and how it differs from pharmaceuticals or other technologies, presenting current practices across European HTA agencies. Second, it outlined results from the research conducted about a taxonomy developed for medical devices and approaching the evaluation based on this model. Different topics were discussed during the breakout sessions, namely prioritization, existing HTA tools and guidelines, and identification of existing challenges and particularities in assessing medical devices in different contexts.
Second workshop Latin America, Santiago, 2015
Session 1 - Assessing Value for Money: as per Warsaw, 2015
Session 2 - Value Assessments in Health Technology: The first part of the session provided insights into the current systems of value assessment and their features, including clinical benefit assessment, economic evaluation and value based assessment, and their combination. The second part of the session outlined the findings from the research conducted within the project on building a value proposition from an HTA perspective. The group breakout sessions evaluated evidence relating to the performance of a new drug versus its comparator(s) across a range of dimensions, indicated for the treatment of an advanced cancer. Participants were asked to write up their own value proposition for these treatments.
Session 3 - HTA and rare diseases: assessing the societal value of orphan drugs: as per Warsaw, 2015
Session 4 - Using alternative methods to assess the value of new medical technologies: The first part of this session provided insights into the use of Multiple Criteria Decision Analysis (MCDA) in the context of HTA, while the second part presented its empirical application to a particular oncology indication. The group break out session related to a number of therapeutic options within a particular indication. Each group was given a list of criteria structured in the form of a value tree, spanning all the critical dimensions of value and were invited to choose among them.
Session 5 - Strengthening and implementing HTA in emerging settings: This session first highlighted the importance of HTA in emerging settings, while the second part described results from the mapping exercise conducted and the tool kit developed aimed at helping countries implement HTA. During the breakout sessions, the main challenges in implementing HTA were discussed. Participants were then asked to discuss having an official threshold in each country.
Session 6 - HTA and medical devices: This session, first, discussed the particularities of assessing value of medical devices and how it differs from pharmaceuticals or other technologies, presenting current practices across European HTA agencies. Second, it outlined results from the research conducted about a taxonomy developed for medical devices and approaching the evaluation based on this model. Different topics were discussed during the breakout sessions, namely prioritization, existing HTA tools and guidelines, and identification of existing challenges and particularities in assessing medical devices in different contexts.
Advance-HTA partners who attended included: LSHTM (Cairns); LSE (Kanavos, Nicod, Angelis), EBC (Mossmann), EASP (Espin), AOTM (Zawada), IER (Orgorevc), TUB (Pantelli, Fuchs), NICE (Cutler). A number of additional speakers were invited to share their experiences with some of the topics discussed, including representatives from: Faculty of Public Health (Bulgaria), Ministry of Health (Ukraine), National Health Fund (Greece), Ministry of Health (Latvia), and the Transparency Council (Poland). The same Advance-HTA partners attended the second workshop in Santiago de Chile, with the addition of PAHO (Lemgruber). Policy reactions were made by representatives from the following institutions: Ministry of Health (Chile), University of York, ISP (Chile), Ministry of Health (Ecuador), IETS (Colombia), IETSI (Peru), Ministry of Health (Uruguay), UFMG (Brazil), INESSS (Canada), INVIMA (Colombia), CONITEC (Brazil), Ministry of Health (Costa Rica).
(e) Workshop participation
The 1st Advance-HTA Capacity Building Workshop (Warsaw, 25-26 September 2014) was attended by 64 participants. Attendees were mainly policy makers working in public institutions such as Ministries of Health, Health Insurance Funds, National Health Services and HTA Agencies. In total, 13 countries were represented: Bulgaria, Croatia, Cyprus, Estonia, Greece, Hungary, Latvia, Poland, Serbia, Slovakia, Russia, Turkey and Ukraine. The 2nd Advance-HTA Capacity Building Workshop (Warsaw, 24-25 September 2015) was attended by 57 participants. Importantly a significant overlap existed between the attendees of the 1st and the 2nd workshop ensuring a degree of continuity in the dissemination process. In total, 14 countries were represented: Bulgaria, Czech Republic, Denmark, Estonia, Greece, Hungary, Latvia, Lithuania, Moldova, Poland, Russia, Slovenia, the Ukraine and the UK. Our World Health Organisation contacts in Europe kindly provided travel funding for a small number of participants from Eastern Europe to attend.
From a Latin American perspective, the 1st Advance-HTA Capacity Building Workshop (Mexico City, 6-7 November 2014), was organized by Advance-HTA with additional support from CENETEC-SALUD, the National Center for Excellence in Health Technology of Mexico and PAHO and was attended by 59 participants, mainly policy makers working in public institutions such as Ministries of Health, Health Insurance Funds, Social Security Institutes and HTA Agencies. In total, 14 countries were represented: Argentina, Bolivia, Brazil, Chile, Colombia, Costa Rica, Cuba, Ecuador, El Salvador, Mexico, Paraguay, Peru, Trinidad and Tobago, and Uruguay. The 2nd Advance-HTA Capacity Building Workshop (Santiago, 9-10 September 2015), was organised by Advance-HTA and was supported by PAHO, the MoH-Chile and the Instituto de Salud Pública de Chile. It was attended by 64 participants from Argentina, Bolivia, Brazil, Canada, Chile, Colombia, Costa Rica, Cuba, Ecuador, Mexico, Paraguay, Peru and Uruguay. We were able to broaden our participation invitations by building our workshops around regional meetings in Latin America, such as the regional HTAi – Latin America (2014 meeting) or the ISPOR-Latin America (2015 meeting), bringing together HTA agencies and members of the REDETSA network (managed by PAHO) from across the Americas providing a forum to discuss a wide range of issues, challenges and opportunities with the various HTA agency models.
4. Project website and electronic dissemination
All four workshops held were based on a mix of presentations, case studies, groupwork and workshops, whereby participants could interact with each other while discussing a case study. Materials and presentations are publicly available to download from the project website, weblink provided below.
https://arquivo.pt/wayback/20170603122218/http://www.advance-hta.eu/dissemination.php
A Twitter group (@AdvanceHTA) has been created, where all novelties, including new publications, meetings, workshops, opening of registration to workshop, or other activities, will be tweeted with a link to the Advance-HTA website. https://twitter.com/ADVANCEHTA . We will also aim to have our partners or relations “re-tweet” our tweets in order to have a greater impact.
An Advance-HTA Linkedin group has been established, to enable the project a platform linking Advance-HTA participants to the broader HTA stakeholder community. The platform has been used to disseminate information about the project, and as a discussion forum. The Linkedin group is linked to the Advance-HTA website, as well as the Advance-HTA twitter account. http://www.linkedin.com/groups/AdvanceHTA-5051411
Exploitation of results
The partners and the research require involvement by key decision makers and stakeholders. Face to face dissemination is encouraged in order to receive feedback from a broad range of stakeholders. There have been several abstract submissions for poster and oral presentations at conferences, panel submissions for conferences, a final conference. All these media have enhanced the audience, decision makers and forum by integrating in a sustainable way and use them actively in systems and practices at local, regional, national and European levels.
1. Toolkits and evidence
A number of toolkits have emerged from the project, as follows: (a) The Taxonomy, enabling an understanding of country- and system-specific factors that may influence coverage recommendations; (b) the Retrospective Analysis Framework, scrutinizing coverage recommendations based on the evidence submitted, its interpretation and a significant number of other considerations, which may be disease- or therapy-specific and which may have predictive power once populated with a larger number of observations; (c) The Advance Value Framework, which is a value assessment framework based on MCDA principles and which has undergone testing across two indications already; (d) The patient database, which has enabled face-to-face interviews and surveys to be conducted across the patient community in order to establish patient-related QoL preferences and utility values; (e) a methodological taxonomy enabling an analysis of HTA across medical devices; and (f) a toolbox that can be used in emerging HTA settings in order to strengthen and/or implement an HTA function based on a number of parameters and a specific conceptual framework. It is envisaged that these toolkits will be disseminated further; in some cases, further research is envisaged – for example, through additional applications for funding in areas that still merit basic research or its application; in other cases, the knowledge generated will be leveraged through the development of online or software applications; finally, in some cases a combination of further research with software development is envisaged.
2. National and International Scientific Conferences
Results have been communicated at conferences and events that individuals attend, including some key international gatherings such as, Health Technology Assessment international (HTAi); International Health Economics Association (iHEA); European Health Economics Association (ECHE); AcademyHealth in the USA and different chapters of the International Society for Pharmacoeconomics Outcomes Research (ISPOR). To date, project results have generated a steady stream of abstract submissions to the above conferences and will continue to do so throughout 2016 and, possibly, 2017.
3. Advocacy, seminars, workshops, capacity-building
Advocacy has played a vital role in disseminating evidence to multiple stakeholders. The strategy has made effective use of both upstream and downstream advocacy to reach out to different players in the arena of access to medicine. Consultations, workshops, apart from seminars, and conferences have been an effective tool to disseminate findings. Seminars and workshops have been organised by the various members who hold in-house capabilities and competencies for doing so (such as AOTM for the Warsaw Workshops and PAHO for the Latin American workshops). Two conferences deserve particular merit in this context, which were also part of the project’s dissemination strategy.
(a) Haute Autorité de Santé (HAS) – Paris, 30 October 2015
For its 10th anniversary, the Haute Autorité de Santé (HAS) in collaboration with the European Commission organised a one-day conference dedicated to HTA and the benefits of European cooperation in this field. Key questions around European cooperation on HTA, stakeholder involvement and early dialogue, current challenges in applying HTA, and HTA in special contexts (e.g. complex interventions, hospital setting) were discussed. Together with EUnetHTA, all four FP7 HTA projects funded by the European Commission were invited to present their findings and outline their policy implications for (HTA) stakeholders. Some of the research streams from Advance-HTA were presented within Session 2 - Towards innovative methodologies & practices. The session focused on ways forward to overcome some of the methodological challenges in current practices of HTA for medicines. Current practices of value assessment have been subject to criticism about whether these measures capture all the important dimensions of value, including living with the disease and receiving treatment in real world settings, while adequately recognising and valuing innovation. This one-day conference took place on October 30th in Paris, at the Ministry of Social Affairs, Health and Women's Rights. A total of 332 participants attended the conference. Each session included a presentation of the empirical evidence and the recommendations, followed by comments from two invited discussants on the results, representing different HTA stakeholders. At the end of each session, about 30 minutes were allocated for a general discussion with all participants. LSE was in charge of the coordination of and logistics for Session 2. Materials and recordings are publicly available – click here
(b) Final Advance HTA Dissemination Conference – London, 19 November 2015
The final dissemination conference of the Advance-HTA project successfully took place on Thursday 19th November 2015. Over 120 participants attended the conference where the results of each work package were presented by the partners of the project and commented by key stakeholders that acted as discussants. The opening keynote address was delivered by the Director General of the Dental and Pharmaceutical Benefits Board (TLV), the Swedish HTA agency. Topics presented included “Assessing value for money”, “New tools and approaches in the value assessment of medical technologies”, “Preference elicitation and quality of life” and “Strengthening and implementing HTA in emerging settings”, followed by stakeholder reflections on future HTA direction. Besides the partners of the project consortium, discussants involved a range of experts including the Head of the European Observatory on Health Systems and Policies, the Programme Manager for Health Technologies and Pharmaceuticals from the World Health Organisation - EURO, the Chairman of the Executive Committee of the European Network for Health Technology Assessment and the President of the European Brain Council, among other leading international academics and researchers. Research findings highlighted differences in the recommendations of HTA agencies across European countries while explaining the underlying causes of decision heterogeneity; presented the development and application of a Multiple Criteria Decision Analysis methodological approach for assessing the value of new drugs; applied the use of a new taxonomical framework for grouping and evaluating new medical devices; highlighted the differences in perspective and preferences between patients and other stakeholders; illustrated the use of a discrete choice experiment approach to elicit the preferences of decision makers and patients for assessing the value of orphan drugs; and showcased the importance of HTA in emerging countries including a mapping exercise and the development of a methodological toolbox. Please see https://arquivo.pt/wayback/20170603123252/http://www.advance-hta.eu/conferences.php for details on the agenda, overview, attendees and presentations.
4. Publications
A significant number of publications have emerged during the lifetime of the project in peer review journals and, more widely in the literature. A list of these publications and those that are currently under review or in the preparation stage are provided in the appropriate section of this report.
Communication beyond the project
Dissemination of the findings generated during the project will target entities and individuals beyond the partners of ADVANCE-HTA. HTA networks in Europe, the networks established from the 4 workshops in Eastern Europe and Latin America will provide the first port of call for new activities and materials emerging from the project. The website is accessible to the public and will be used post-project to communicate the new knowledge generated from the project; the European Observatory platform will also be used to disseminate ADVANCE-HTA findings such as books and reports. Findings are already being disseminated through peer-reviewed publications and will continue to beyond the project end-date. Social Science and Medicine, Medical Decision Making, International journal of Technology Assessment in Health, Health Policy, Pharmacoeconomics and the International Journal for Technology Assessment in Health Care are few of the key journals and publications that are featuring or will feature key pieces from the project. The consortium is keen to showcase the specific areas explored by the project as well as the key messages, findings and recommendations as a whole to share with policy makers, stakeholders and HTA groups on a national and international level. The consortium has an agreement regarding authorship and intellectual property that enables its members to publish their results in the literature in a collaborative and inclusive way. Any publication or communication regarding the project will specify that it reflects the authors’ views and that the EC is not liable for any use of the information communicated.
List of Websites:
https://arquivo.pt/wayback/20170603121424/http://www.advance-hta.eu/
ADVANCE-HTA aims to contribute to advances in the methods and practices for HTA in European and other settings by involving the wider stakeholder community in areas actively and heavily debated given their implications for decision-making and resource allocation. ADVANCE-HTA aims to make a number of contributions in six distinct areas, which carry significant policy implications for resource allocation (see Figures 1 and 2 in attached report). These are:
• First, the issue around value for money and the different approaches surrounding current thresholds for resource allocation, where ADVANCE-HTA will systematically explore alternative means of assessing value for money and trace the implications for the conduct of HTA and the use of cost-effectiveness data to inform decision-making.
• Second, the concept of value assessment, and the factors that need to be considered and incorporated beyond cost effectiveness, such as burden of disease, disease severity, quality of the data and evidence produced and the implications these are having on the continuous assessment of new health technologies and relative effectiveness. ADVANCE-HTA will aim to explore new tools and methodologies in this domain, for example Multi-Criteria Decision Analysis, and investigate their adoption and implementation.
• Third, to improve the quality of the evidence required for and the methods associated with the assessment of rare diseases by relying on new data providing a more realistic understanding of the socio-economic benefits of orphan drugs. In this context, ADVANCE-HTA it will develop and validate a framework to support decision-making relating to orphan drugs for rare diseases, by means of a Multi-Criteria-Decision framework.
• Fourth, to improve the robustness of the evidence on the elicitation of preferences by deriving these in more realistic settings, by drawing on the wider EU citizenship and from within the patient community. ADVANCE-HTA will create new data that will incorporate patient-relevant values into widely used tools of quality of life measurement, such as the EQ-5D.
• Fifth, to advance the debate on the suitability of current HTA tools across different categories of medical devices, including diagnostics, ADVANCE-HTA will consolidate the current methods for assessing HTA in medical devices in different settings, address their suitability to appraise different types of medical devices, including diagnostics and propose how current tools can be modified or adapted in order to arrive at more robust methods of assessment.
• Sixth, to improve the implementation and capacity building of HTA, also incorporating improvements as outlined above in settings outside Europe, where HTA is beginning to be considered explicitly in decision-making. ADVANCE-HTA will create a framework for HTAs at different levels (national, hospital [mini-HTAs]) by benchmarking with evidence from countries that have developed such frameworks.
Additional activities within ADVANCE-HTA contribute to furthering the debate on future developments in HTA by bringing together the research being conducted across the Consortium. Further, extensive dissemination of the results has been carried out by effectively linking policy makers, stakeholders and patient networks to the research evidence on HTA.
ADVANCE-HTA impacts a range of stakeholders and activities. The methodological advances in HTA are likely to influence developments in areas such as value-based pricing of medical technologies, or other areas of HTA (e.g. HTA in medical devices). Capacity building activities in Latin America and Eastern Europe have transferred the accumulated expertise to countries that are new to HTA, while involving experts from other settings on the debate surrounding HTA and the health care resource allocation debate.
Overall, ADVANCE-HTA aims to broaden the spectrum, complement and address areas of intense methodological debate in the application, use and implementation of HTA. It also aims to improve HTA methods, which can be taken further by competent authorities nationally whilst supplementing the work of supra-national bodies (e.g. EUnetHTA) towards a common understanding of choices in health care decision-making.
Project Context and Objectives:
The Advance-HTA Project was based on a collaboration between 13 partner agencies and funded by the European Commission's Research Framework Programme (FP7) during the three year period 2013-15. Advance-HTA explored the different processes and varying criteria used for assessing clinical cost-effectiveness across Europe, and how frameworks such as Multi-Criteria Decision Analysis (MCDA) might be used to further enhance the value and acceptability to all stakeholders of current ‘mainstream’ approaches to cost-effectiveness evaluation as the central element in modern Health Technology Assessment. The Project, in addition, sought to determine the extent to which disease rarity impacts on assessments of the incremental cost per QALY ratios (the ICERs) for orphan drugs, and the ways in which preferences relating to health related quality-of-life differ between the general population and patients who have actual experience of given conditions. Two further work streams were designed to cast light on the degree to which, and ways in which, HTA is used in appraising the value of medical devices, and the establishment of both governmental and private sector HTA capacity in ‘emerging’ economic settings like those of Eastern Europe and the Americas, excluding the US. Finally, this LSE-led initiative sought to identify and promote public consideration of the wider policy implications of the research findings generated via the above sets of research activity.
Work Package 1: Value for money (WP Lead: LSHTM)
Most European countries have Health Technology Assessment (HTA) procedures for informing drug reimbursement decisions. Depending on the drug and the country assessing it, the decision can be Favourable, Favourable with restrictions or Non-Favourable. The main objective of this research was to determine the factors that might lead to different drug reimbursement decisions in ten European countries, on Cancer, Rheumatoid Arthritis and Multiple Sclerosis drugs (2002-2015). More specifically, the main areas of focus for this work package were:
• To expand knowledge and understanding of the advantages and disadvantages of the different approaches to assess cost-effectiveness and the main factors defining HTA decision-making and to construct a database of HTA decisions of different European countries;
• To consolidate the lessons from existing research, and to identify what new research would be of greatest value to improve HTA decision-making processes; and
• To advance the methodology in HTA by systematically exploring alternative means of assessing value for money and tracing the implications for the conduct of HTA and the use of cost-effectiveness data to inform decision-making regarding which health technologies to adopt, by empirically identifying the main factors driving HTA decision-making across countries
Work Package 2: Value Assessment in HTA (WP Lead: LSE)
This work package relates to the methodological and empirical aspects of value assessment for new medical technologies as well as advancing our thinking on methods and their implementation for the same. The need for a WP that re-visits the issue of value has become clear in light of the limitations of cost-effectiveness analysis to adequately account for other dimensions of value (which are often considered on an ad hoc basis), as well as the wide variability in coverage recommendations by different HTA bodies for the same technology – indication pair, indicating that there are other factors at play beyond the strict dimensions of economic evaluation. The WP’s specific objectives were:
1. To understand the parameters of value in HTA appraisals from an international perspective;
2. To explore how factors such as disease severity, burden of disease, distinguishing between levels of innovation, and the quality of the available evidence can be incorporated more explicitly – and in a quantifiable way, in the HTA process;
3. To explore how alternative analytical frameworks, such as Decision Analysis, can be used to elicit value;
4. To conduct case studies in specific disease areas by using alternative analytical tools and by explicitly incorporating all identified parameters of value.
Work Package 3: HTA and Rare Diseases - Assessing the Societal Value of Orphan Drugs (WP Lead: ISS)
There were 4 key areas that WP4 focussed on.
• First, to understand whether rarity has an impact on the assessment of the incremental cost per QALY of orphan drugs compared with drugs for more common diseases and whether rarity impacts mainly on the incremental costs or on the incremental QALYs;
• Second, given the difficulties of conducting randomized clinical trials and the progressive nature of many rare diseases, to identify alternative robust sources of data to accumulate knowledge on the effectiveness and societal value of orphan treatments, which can be used for policy decisions;
• Third, to determine in what way(s) do current processes for assessing and appraising drugs need to be adapted to make them suitable for orphan drugs, and whether all elements of societal value can be adequately reflected in existing decision-making procedures.
• Fourth, to develop and validate a framework to support decision-making relating to orphan drugs for rare diseases, by means of a Multi-Criteria-Decision framework.
Work Package 4: HTA and Quality of Life Measurement (WP Lead: IER)
The key objectives of this work package were:
• To determine whether the preferences towards health related quality of life differ between the general population and defined patient groups, including those benefitting from personalised treatments;
• To examine the causes of the established differences between general population and patient group preferences;
• To define advantages and disadvantages of the value sets drawn from the general population and patient populations respectively of cost-effectiveness.
Work Package 5: HTA and Medical devices (WP Lead: TUB)
The work carried out in WP5 aimed to structure existing knowledge on and improve understanding of the particular challenges characterizing the assessment of MDs. Furthermore, it intended to set the groundwork for bridging knowledge and implementation gaps and providing best practice recommendations for HTA institutions and policy-makers alike. Specifically, the key objectives were:
• Develop a Taxonomy of Medical Devices (MDs) based on existing classifications and nomenclatures and test its plausibility and usefulness;
• Identify and compare current HTA methodologies, processes and practices across EU Member States;
• Clarify and supplement earlier findings, and to trace methodological and procedural challenges and trends.
Work Package 6: HTA in Emerging Settings (WP Leads: EASP & PAHO)
WP6 consisted of 3 parts:
• To identify the use and capacity for using HTA in emerging settings (Eastern Europe, Region of the Americas) including not only decision making bodies, but also some other institutions (universities, private companies, etc.). This includes the identification of HTA mechanisms and techniques apply in different emerging countries/regions;
• To encourage the use HTA for decision making through the development of formal guidelines for HTA methodologies in the context of Emerging Countries (South-) Eastern Europe, Region of the Americas, etc.) that make more transparent the use of these tools; and;
• To enable and facilitate an effective exchange of economic evaluations and evidence, reducing duplication of effort, between countries in Eastern Europe/Latin America and EU countries that have strong HTA tradition.
Work Package 7: Potential Implications - Lessons learnt, recommendations and implementation (WP Lead(s): UCLM & LSE)
This final deliverable of Advance-HTA (D7.1) aims to contribute to the debate on future developments in Health Technology Assessment, the advances in a number of aspects of HTA implementation, the policy implications these are having and their contribution to the debate of efficiency in resource allocation in health care systems and the related health system sustainability. The specific objectives of this WO are threefold:
• To provide a concrete analysis of the policy implications of the research results for HTA in the domains researched upon (part I of the report);
• To identify how the research results can be incorporated in the decision-making process and implemented at national, supra-national and international level and what implications are there for specific stakeholders; and
• In synthesizing the results of this project, part II of this report explores the wider conceptual/theoretical, economic, social, political, R&D, and policy implications for HTA and the relevance of the findings of Advance-HTA in this context.
Work Package 8: Dissemination of Project Objectives and Results (WP Lead: LSE)
The main objective of this work package was to effectively link policy makers, stakeholders and patient networks to the research evidence on HTA. The dissemination strategy determined the target groups for different results and the most adequate dissemination mode: publications, face-to-face and electronic dissemination. Dissemination was continuous throughout the life of the project and is also expected to continue after the end of the project through publications in the peer review literature and symposia among others. A number of concrete dissemination activities have been undertaken during the lifetime of the project which are discussed in section 4.1.4.
Overall, Advance-HTA’s findings indicate that further investment needs to be made in developing Health Technology Assessment tools for use in areas outside the narrowly defined pharmaceutical sector, in order to further increase the productivity of health care as a whole. They may also need to be adapted to accommodate the reality that drugs do not normally act alone to achieve the best possible outcomes, albeit the discovery of optimum combinations and administration strategies can – as experience in spheres such as the treatment of child leukaemias has shown – be a long drawn-out process.
However, from a practical perspective it is presently the case that HTA is centrally concerned with a relatively narrow form of medicinal product evaluation, pricing, purchasing and use. This paper reflects this fact. It initially offers an outline of the origins of health economic evaluation in Europe and the United States, and highlights some basic economic aspects of pharmaceutical research, development, manufacturing and supply. It then explores from a public policy perspective a spectrum of topics arising from the Advance-HTA research. These centre on issues like the value of further involving patients and their representatives in determining whether or not treatments are affordable, and the need for well-balanced national and international policies aimed at incentivising public interest focused private investment in pharmaceutical research and development.
Project Results:
Work Package 1 (WP Lead: LSHTM, UK)
Even if European countries have common objectives for HTA systems, the procedure is not homogenous. The operative processes and the organisations work differently across countries. Moreover, evidence shows that there are differences in the final decisions across countries, despite the clinical evidence reviewed being largely the same, and the countries - while not of equal wealth - are of broadly comparable levels of economic development. The main objective of WP1 was to determine the factors that might lead to different drug reimbursement decisions in ten European countries, on Cancer, Rheumatoid Arthritis and Multiple Sclerosis drugs. This goal was achieved through a specially designed taxonomy and an econometric analysis. In a preliminary analysis (Maynou and Cairns, 2015 ), a number of hypotheses were tested that could explain the differences in cancer drug reimbursement decisions across ten European countries. While the results showed that the HTA system characteristics, the drug particularities and the socioeconomic situation can explain some of the differences between countries, a fuller explanation requires a model which determines cancer drug reimbursement decisions incorporating a wide range of health system characteristics and specific characteristics of the individual drugs. The WP proceeds in building a taxonomy, then a database, performing econometric analysis before presenting results and policy implications.
1. Taxonomy
A detailed analysis of the drug reimbursement systems in ten European countries was conducted. Research focused on: Belgium, France, Germany, the Netherlands, Poland, Portugal, Spain, Sweden and the United Kingdom (England and Scotland analysed separately ). These countries were selected because they each have a well-defined HTA process and publicly available information on their drug reimbursement decisions. This first analysis involved: (1) a review of policy documents and relevant literature and examination and querying of the study country decision-making bodies’ websites and (2) a discussion with experts knowledgeable regarding the process in each of the countries. In this second process, we drew heavily on Advance-HTA Consortium members representing some of the studied countries.
This analysis was essential to identify potentially important system-wide factors. This taxonomy (see Table 1 in the appendix) describes the main characteristics of the drug reimbursement system in each country and the main features of each drug-indication pair. Moreover, time dependent variables are also included in the classification to capture the socioeconomic situation of each country. This taxonomy can be classified into three groups, the system-wide variables (organisational, process and method), the product-specific variables (general characteristics and country-specific) and the time dependent variables. Table 1 defines the taxonomy variables. This taxonomy is used to populate the econometric model with explanatory variables.
2. Database
The sample included the reimbursement decisions for Cancer, Multiple Sclerosis and Rheumatoid Arthritis drugs from January 2002 to April 2015 appraised in the ten selected countries (Belgium, France, Germany, Netherlands, Poland, Portugal, Spain, Sweden, England and Scotland ). The dataset contained the outcome of the decision, the date and all the variables of the taxonomy. The decision outcome described the final decision regarding the adoption of the technology: Non-Favourable, Favourable with restrictions and Favourable. In order to create the database, the reimbursement decisions were taken, when possible, from national websites. However, for some countries, these decisions were not publicly available or some details were missing. These limitations were overcome with the collaboration of some partners, namely, UPEC (France), AHTAPoL (Poland), EASP and UCLM (Spain) and TLV (Sweden). The database comprised 1077 decisions, of which 62% were Favourable, 13% were Non-Favourable, and 26% were Favourable with Restrictions (see Table 2 in the appendix). Of these decisions, 79% related to Cancer drugs, 9% related to Multiple Sclerosis and 12% to Rheumatoid Arthritis.
3. Econometric analysis
The objective of the econometric analysis was to determine empirically which of the taxonomy variables were associated with a higher or lower probability of reimbursement (Favourable decision). The dataset was designed as a particular case of a hierarchical model within the family of multi-level models. It considered the decisions for each drug-indication in ten European countries from January 2002 to April 2015. The primary interest was to observe the effect of the explanatory variables (defined in the taxonomy section) on the probability of reimbursement. Although this variable was unobserved, it could be approximated through a categorical variable corresponding to the final decision: 0 = Non-Favourable; 1 = Favourable with restrictions; and 2 = Favourable. In order to take into account different time periods, the spatial distribution and the heterogeneity of the model, a Hierarchical, Random-Effects Ordered Probit was considered to be the best model specification and was used in the analysis. The inference was made through the Bayesian framework, the Integrated Nested Laplace Approximation (INLA).
4. Results
The results of the econometric model pointed to a number of policy relevant findings. In terms of the system-wide variables, the model showed that less stakeholder involvement in the HTA process was associated with a higher probability of reimbursement. However, the requirement of economic evaluation for all drugs, an external review of the evidence, the manufacturer being the initiator of the process, price based on a(n) (external) reference pricing and the budget impact requirement were related to a lower probability of reimbursement.
According to the results of the model for product-specific variables, a drug-indication with a NICE-Favourable decision was linked to a higher probability of this drug-indication being accepted in another country. A drug considered cost-effective by NICE/SMC was associated with a higher probability of reimbursement. The other product-specific variables included in the model were not statistically significant. One of the specific objectives was to observe if the therapeutic area was an additional determinant of the probability of reimbursement. However, the relevant variable was not statistically significant.
Finally, the results also showed the effects of the time-dependent variables included in the model. In particular, the rate of female mortality and percentage of population older than 65 were positively and significantly associated with the probability of reimbursement. However, higher proportion of population below 14 years old was associated with a lower probability of reimbursement. The crisis dummy showing the time effect from 2008 onwards was not statistically significant. However, the interaction between the crisis dummy and the Managed Entry Agreement (MEA) variable shows that a MEA in the crisis period was associated with a higher probability of reimbursement. The remaining time-dependent variables included in the model did not have a statistically significant association with the probability of reimbursement.
A further finding was that according to previous literature and our data collection, differences existed in the final reimbursement decision across the ten countries analysed. As a result, a key finding of our study documented the differences in decisions across the ten European countries. The database showed that for the same drug-indication pair there were different reimbursement decisions for the selected countries. However, the specifications of the econometric model were able to control for the differences across the HTA systems and the socioeconomic conditions of each country, as the random effect used to allow for country variation was not significant.
Moreover, in order to overcome one of the main limitations (i.e. meaning of decision outcome), we carried out two further econometric analyses. In the first, France and Germany were excluded because both countries’ drug assessment is used to inform the pricing decision procedure and not to accept or reject the new technology for coverage purposes. As this procedure differs from the remainder of the sample of countries, which take a decision on whether to accept or not the drug in their benefits catalogue, a decision was made to run the model without taking into account the decisions of France and Germany. Comparing the original model (all countries) with the new model, the main finding was that the new model was more efficient in terms of the DIC and the CPO (both, goodness of fit of the Bayesian approach). The main differences of this new model were that the initiator and the time dependent variables were not statistically significant.
The second analysis adopted a different approach in an attempt to solve the previous limitation. Instead of directly removing both countries, this approach tried to capture this difference by controlling for the pricing system in each case. In this model, the pricing decision variable was removed, but some interactions between the main explanatory variables and the pricing systems were included. The aim of this analysis was to observe different effects depending on the pricing system used in each decision. The main finding was that there was a statistically significant effect when the price-setting is undertaken by the manufacturer (such as in England, Scotland, Sweden, Netherlands and Poland), compared to the baseline category, i.e. price-setting based on negotiation (France, drug non-adding value, and Germany, drug adding value). In other words, the pricing system of each country has an effect on the decision outcome.
5. Policy implications
A number of policy implications have emerged from this work package. First, the research conducted implied a better understanding of the main determinants of drug reimbursement decisions. As a result, policy-makers can be aware of the factors that may drive the HTA decisions. In a new or reformed HTA process, these analyses could help defining the HTA procedure. Second, based on this analysis, differences in drug reimbursement decisions across countries can be better explained to the HTA community. In particular, patient groups and health care professionals can benefit from this information and better understand why some drugs are approved while others do not. Third, from the data collection procedure, it can be determined which of the study countries have a more transparent and accessible drug reimbursement system. The policy implication is that differences in transparency and accessibility of decisions should be addressed, if there is to be greater understanding of the differences in access to these medicines across countries. In other words, if policy-makers are interested in why decisions are different across countries, non-transparency is an important barrier. Finally, during the data collection process, experience was acquired in finding the HTA documentation on the websites and/or internal databases. A pathway can be created for each country, which can be very useful for the HTA community and would be of interest not only to policy-makers, HTA agencies or Departments of Health, but patient groups, health professionals and researchers.
Work Package 2 (WP Lead: LSE, UK)
This WP has researched the domain of value assessment of medical technologies and provides insights on different modalities and frameworks. The WP is associated with the development of 2 methodological frameworks, one that explains a significant part of the unexplained heterogeneity in coverage recommendations across HTA settings and another - using decision analysis principles - that incorporates a wider selection of value dimensions and stakeholder preferences to provide a more holistic perspective on value assessment. Both frameworks have been piloted and tested and evidence is provided on their performance.
1. Value assessment drivers and criteria
A systematic literature review was conducted taking into account evidence from England, Scotland, Sweden, Poland, France, Germany, Italy and Spain showed that, overall, all countries assess similar types of evidence, however the specific endpoints used, their level of provision and requirement (i.e. extent of guidelines provided and any flexibility around them), the way they are incorporated (e.g. explicitly vs. implicitly) and their relative importance varies across countries. The main evidence assessed could be divided into four clusters of evidence: (a) burden of disease; (b) therapeutic and safety impact; (c) innovation level; and (d) socioeconomic impact. In addition, efficiency parameters such as cost-effectiveness and budget impact are considered, but these are essentially derived through the initial assessment of the main evidence and thus could be regarded as different analysis approaches. Furthermore, ethical value characteristics such as equity are also taken into account in many countries, but they usually act as a type of ‘fundamental principles’ on which the complete evaluation, including the committees’ judgements, should be based on. The relative importance of the different criteria, or their overall and relative importance on the overall decision is in the majority of cases, unclear (e.g. the relative importance of overall survival versus the relative importance of quality of life). The analysis has shown that the value of new medical technologies is multi-dimensional and not limited to clinical benefit and cost dimensions.
A number of the value dimensions mentioned above such as the burden of disease and its overall socioeconomic implications are not adequately reflected in the assessment process. These value dimensions are not always considered, and, if they are, this is done in an implicit and non-systematic manner or an ad-hoc basis. Consequently, the methodological framework of these value assessment approaches remains inadequate and at best partial mainly because the criteria used to assess evidence and determine value are valid but incomplete or inadequate. The gap between the assessment of evidence and the appraisal of technologies seems to give rise to a fragmented decision-making process. These results highlight the limitations of current value assessment approaches mainly with regards to the incomplete and implicit set of evaluation criteria considered.
2. Criteria driving HTA decisions: A Retrospective Analysis Framework
Beyond the state of the art in value assessment, this WP has aimed to understand the criteria driving HTA decisions based on a retrospective analysis of these in four countries (England, France, Scotland, Sweden). A methodological framework was developed and subsequently piloted and tested, that enables to systematically identify and compare the decision-making criteria across countries and therapy areas (see Figure 1 in the appendix). This allowed for an enhanced understanding of these decision processes in different settings. Each step undertaken to set up and test this framework was described in detail to ensure its transparency and also transferability to third parties, by means of a detailed coding manual and case study template. Its application to analysing a sample of drugs in several countries facilitated the identification of taxonomy of criteria that may have contributed to the decision process, as well as of similarities and differences across countries. This was then used to raise awareness about how value assessments are conducted in different settings and identify those cases where differences are a consequence of a particular process captured by this framework, or by limitations in applying HTA. A sequential exploratory mixed-methods qualitative design was used to develop the framework, dividing the research process into three distinct phases. First, case study analyses were piloted in order to identify the range of criteria accounted for and the structure of the decision-processes, such that they were comparable across countries. Second, the structure of analysis was used to establish the coding manual and case study template, which formed the basis of the methodological framework, used to inform the qualitatively thematic coding of all drugs and countries. This enabled for coding to be homogeneous, systematic and flexible given the iterative process adopted. Third, qualitative data was quantitatively analysed in order to understand and measure the level of agreement at each stage of the decision-process and reasons for differences, as well as identify agency-specific preferences (risk and value preferences). Data sources consisted in the publicly available documents containing the specific HTA recommendation, which will be referred to as the “HTA report”. Based on the assumption of transparency, the information reported in the HTA reports was considered to reflect the key reasons around the final decision.
The different phases of the decision-making process included in the methodological framework comprised (a) the clinical and economic evidence submitted by the manufacturer or produced by the HTA body, (b) the assessment of the evidence including the uncertainties raised, stakeholder input and “other considerations” influencing the decision, also as part of the deliberative process, and (c) to what extent these actually influenced the final HTA recommendation usually in the form of a positive or restricted recommendation or a negative recommendation for coverage.
The main originalities of this research are: (a) the applicability of a new methodological framework to other countries and disease areas given the iterative and flexible approach adopted; and (b) its innovative design through the integration of both a vertical component (decision-making criteria) and a horizontal component (reasons for differences across countries). The vertical component entails coding each individual drug-indication assessment processes in a systematic, homogeneous and comprehensive manner (based on an existing and flexible coding manual) in order to quantitatively devise - through correspondence analysis - agency-specific preferences in the type of evidence, concerns (e.g. risk preferences), and “other considerations” (e.g. value preferences) identified. The horizontal component of applying the framework enables to capture how these processes compare across countries, whether the criteria had a positive or negative influence on the decision, where the criteria put forward came from (e.g. stakeholder input), and whether the criteria were one of the main reasons for the final recommendation. Quantitative analysis (through Cohen’s kappa scores) of these allowed to measure the degree of agreement across countries in the criteria accounted for.
3. Application of the Retrospective Analysis Framework: Evidence from Oncology and Orphan Drugs
The Retrospective Analysis Framework was applied to a sample of cancer and central nervous system treatments. Twenty-one drug-indication pairs were systematically analysed and compared (13 cancer, and 8 central nervous system (CNS) across four study countries – England, Scotland, France and Sweden resulting in 74 drug-indication pair observations across the 4 countries [N=74]). 14/21 drug-indication pairs received diverging recommendations: 11 out of 13 in cancer indications, and 3 out 8 in CNS indications received diverging recommendations.
When comparing the evidence, the same primary trials were appraised for the cancer drugs (and therefore the clinical assessments made were considered comparable), and a high level of heterogeneity in the evidence appraised was found in the CNS sample, particularly for four of the drugs (Alemtuzumab, Fingolimod, Retigabine and Natalizumab). In terms of the economic assessments (for NICE, SMC and TLV), they were deemed partially comparable when the same types of models and comparators were appraised. The primary clinical trials considered were mainly phase III trials, with the exception of two cases that included phase II trials (2/8 primary trials in the CNS sample) and one case that included indirect comparisons. The comparators used were either another treatment, standard of care, placebo, or no other treatment. Important differences were seen in the level of reporting these same primary trials, which partially explained some of the differences across countries (e.g. Eltrombopag, Mifamurtide, Trabectedin, Erlotinib, Rituximab, Pazopanib and dimethyl fumarate).
Focusing on the interpretation of the evidence presented, a number of value judgments were made about this evidence. One of the main contributions of this research was that it was possible to capture these additional (softer) dimensions of decision-making and their influence on the decision. This was facilitated through the double coding conducted for all uncertainties identified with: (a) whether the evidence that was considered was accounted for by the other agencies, (b) whether this concern was addressed or not (and through what means), and (c) whether it was one of the main reasons put forth for the decision. On this basis, it was possible to measure how different HTA agencies dealt with evidence considered by all by comparing the types of issues raised. For example, the similarities in the concerns identified were even lower, with 2.5% of the concerns raised for the cancer drugs common across all agencies (n=162 cumulative number of concerns) and 2.7% for CNS drugs (n=11 cumulative number of concerns), highlighting a lack of common pattern in evaluating the evidence. Further, uncertainties were addressed through various means: stakeholder input was used to confirm the plausibility of a (n)(uncertain) clinical claim; or the uncertainties were raised but nevertheless considered acceptable by Appraisal Committees. Raising awareness about how these issues were dealt with across settings may provide insights on alternative ways to address these through cross-country learning. Social value judgments were captured by coding all the emerging “other considerations”, together with their influence on the decision and whether they were provided by the different stakeholders (through double coding). The main contribution of this research from applying the framework is that it enables to differentiate elicited social value judgments (and preference explicitly given) from those that were not elicited but were nevertheless raised by the decision-makers.
The last step was to examine how the evidence and its interpretation influenced the decision. Findings show that there is generally not a single main reason for having different HTA recommendations, but a number of reasons and circumstances that can be identified at each step of the process. The reasons for a certain decision or for different decisions across countries are, therefore, a combination of these three components rather than one criterion further emphasising the complexity of HTA processes and the multiplicity of criteria beyond clinical-cost-effectiveness that feed into decision-making processes as identified by this research.
4. Using MCDA principles to measure overall value in the context of HTA: The Advance Value Framework
Given the variety of criteria that feed into decision-making processes and the fact the clinical & cost-effectiveness analysis may account for only part of a new intervention’s overall value, this component of the WP departed from these principles and developed a model based on decision analysis, more specifically, multiple criteria decision analysis (MCDA). This was undertaken in an attempt to determine whether and how additional dimensions of value and stakeholder preferences could be incorporated in the decision-making process explicitly rather than implicitly.
In the context of HTA the MCDA process developed could be divided into five distinct phases (see Figure 2 in the appendix), involving (a) problem structuring, (b) model building, (c) model assessment, (d) model appraisal, and (e) development of action plans. Problem structuring involves an understanding of the problem to be addressed (i.e. defining the aim of the analysis, as for example ranking a set of drugs for a particular indication based on their overall value). This includes key concerns, envisaged goals, relevant stakeholders that may participate in or contribute to decisions, and identification of uncertainties in terms of a new technology’s clinical evidence and its quality. In the phase of model building criteria selection takes place, while being consistent with a set of assumptions, followed by the phases of model assessment and model appraisal, which involve the construction of decision makers’ value judgements across the criteria, aiming to help decision makers identify the most preferred solution(s) as well as the intensity of individual preferences. Finally, given that the outcome of the analysis needs to inform decision-making, action plans need to be shaped involving a clear pathway for result implementation. In the case of HTA this could involve the policy-making actions relating to coverage that take place following the evaluation of a new technology.
The criteria selection and their hierarchical organisation in the form of a tree, known as value tree, were completed in parallel during the model-building phase following a four-stage process aiming to limit subjectivity to a minimum through an iterative process of evidence identification, supplementation, revision, and validation. This involved (a) a systematic review of the value assessment literature in the context of HTA; (b) a second more targeted examination of the methodological literature by focusing on known initiatives in the wider health care decision-making context while also capturing some grey literature, including the benefit-risk assessment of new drugs from a licensing perspective, pricing and coverage decisions from a payer perspective and patient access of orphan medicines from a social responsibility perspective; (c) a consultation with HTA experts that acted as Partners and Scientific Advisory Board members of the Advance-HTA project; and (d) a number of dissemination activities through the Advance-HTA project. Criteria were structured in the form of a tree to reflect the values under consideration, offering an organised overview of the various concerns at hand. Ultimately, the resulting value tree (Advance Value Tree©), consisted of criteria clusters relating to five key groups of parameters that can be explicitly quantified and assessed (see Figure 3 in the appendix): (a) burden of illness, (b) therapeutic impact, (c) safety profile (d) innovation level, and (e) socioeconomic impact.
The Advance Value Framework© is a new value framework that has been developed specifically based on MCDA principles and adapted to the needs of HTA. It addresses the following methodological aspects: (a) with regards to the Decision Context, it adopts a societal perspective; however, the choice of perspective can be adapted to local circumstances and choices by the decision maker. (b) With regards to Model Building, it assesses the value of a new medical technology through the set of criteria outlined as part of the Advance Value Tree©; specifically, there is incorporation of social value judgements relating to burden of disease, innovation level and socioeconomic impact, in addition to therapeutic impact and safety, all of which can be captured explicitly; (c) in terms of model assessment it implements a combination of (i) a value measurement method relating to multi-attribute value theory (MAVT), and, more precisely, an indirect technique for the elicitation of preferences in the form of value functions (MACBETH), (ii) a swing-weighting technique, and (iii) an additive linear aggregation approach. This combination of techniques is used because of their comprehensiveness and methodological robustness, but also their ability of reducing ambiguity and motivational bias.
5. Application of the Advance Value Framework: Case Studies on Cancer
The framework was applied and the value tree was adapted in the context of two case studies. In both case studies the decision support software M-MACEBTH, based on the MACBETH approach, was used to build the value tree, elicit the value functions for the different criteria in order to score the options, weigh the criteria through a swing weighting approach, aggregate the scores and weights using an additive aggregation, and conduct sensitivity analysis.
An experimental case study was conducted as a first proof of concept, applying the methodological framework in practice, for assessing the value of alternative options for the treatment of metastatic colorectal cancer (mCRC) following first line chemotherapy. For this purpose, a decision conference workshop was organised with the participation of a wide range of stakeholders. Metastatic colorectal cancer was chosen because of its severity, the availability of several expensive alternative treatment options, and the fact that it has been the topic of appraisals by several HTA agencies, including the National Institute for Health and Care Excellence (NICE). The framework was used to rank three therapies (Cetuximab, Panitumumab, and Aflibercept with FOLFIRI).
In a value scale of 0 to 100, where 0 corresponded to a lower reference level reflecting “satisfactory performance” and 100 to a higher reference level reflecting “best performance”, Cetuximab scored the highest overall value score of 51.8 followed by Panitumumab with an overall value score of 45.2 and Aflibercept in combination with FOLFIRI with an overall value score of 17.4. These numbers represent a value index score reflecting the performance of the alternative therapies across all the criteria of interest while being adjusted for their relative importance. In terms of total weights across criteria clusters (out of 100), therapeutic impact cluster attributes totalled overall a relative weight of 46.5 (three attributes), the safety profile cluster a relative weight of 23.3 (single attribute), the innovation level cluster attributes had a relative weight of 18.6 (four attributes), and the socioeconomic impact cluster had a relative weight of 11.6 (single attribute). By incorporating the total purchasing costs of the different drugs (including their administration costs), their overall value scores versus total costs plot was produced. By using rounded up total cost figures and dividing them with overall value scores, the costs per MCDA value unit were calculated to be £348, £598, and £1,693 for Cetuximab, Panitumumab and Aflibercept in combination with FOLFIRI, respectively. Therefore, in terms of value-for-money Aflibercept in combination with FOLFIRI is shown to be dominated by Panitumumab, both of which are shown to be dominated by Cetuximab, which is associated with the highest overall value score and the lowest cost.
A second proof of concept case study was conducted, applying the methodological framework for assessing the value of treatment options used in the management of metastatic prostate cancer (mPC) following first/second line chemotherapy. In contrast to the first case study, which elicited preferences from a wide range of stakeholders, this case study utilised the value judgements of an actual HTA agency. For this purpose, a decision conference workshop was organised with the participation of assessors from the Swedish Dental and Pharmaceutical Benefits Agency (Tandvårds- & Läkemedelsförmånsverket, TLV). Metastatic prostate cancer was chosen because of its severity, the availability of several and expensive alternative treatment options, and the fact that it has been the topic of appraisals by several HTA agencies, including TLV and NICE. The treatments under consideration were Enzalutamide, Abiraterone and Cabazitaxel.
Enzalutamide scored the highest overall value score of 58.8 followed by Abiraterone with an overall value score of 18.9. Cabazitaxel scored an overall value score just above zero (0.3) mainly because of its “negative” performance in the treatment discontinuation attribute. In terms of total weights across criteria clusters (out of 100), the therapeutic impact cluster had a relative weight of 44.5 (3 attributes), the safety profile cluster had a relative weight of 33.3 (2 attributes), the socioeconomic impact cluster had a relative weight of 14.8 (single attribute), and the innovation level cluster had a relative weight of 7.4 (two attributes). By incorporating the total purchasing costs of the different drugs (including their administration costs), their overall value scores versus total costs plot was produced. By using rounded up total cost figures, the costs per unit of MCDA value were calculated to be £419, £1,159, and £89,348 for Enzalutamide, Abiraterone and Cabazitaxel, respectively. In terms of value-for-money Cabazitaxel was shown to be dominated by Abiraterone, while also being very close to be dominated by Enzalutamide (£500 difference). Enzalutamide on the other hand was associated with a higher cost (£2,500 difference) and a higher value score (39.86 difference) compared to Abiraterone.
6. Recommendations based on primary and secondary evidence
The Retrospective Analysis Framework has shown that both the similarities and differences in HTA recommendations across settings are not random, but can be explained through the parameters developed in the framework. Important policy implications have also arisen. First, the framework captures the full range of criteria accounted for during decision processes, systematically and in a comparable manner. Second, it raises awareness about cross-country differences and potential methodological improvements; this is relevant in light of on-going efforts to harmonize HTA processes at European level. Third, a better understanding of the criteria driving these processes and how value is assessed in different settings can be useful for all relevant stakeholders involved in the process. For example, it enables the identification of the criteria across all drugs, therapy areas, indications, countries, which could be useful for other applications such as within multiple criteria decision analysis. It allows an understanding of how scientific and social value judgments (e.g. elicited or not) are applied and how the types of agency-specific preferences are factored in. The application of “other considerations” can contribute towards greater accountability for reasonableness and help define social values. Finally, it sheds light on the type of input needed from stakeholders (e.g. patient or clinical experts) and how stakeholders may influence the process.
In the case of the Advance Value Framework and compared to economic evaluation, the resulting aggregate metric of value emerging from the MCDA process is more encompassing in nature as multiple evaluation criteria are informing its composition. By adopting this value index metric for each option as the benefit component and incorporating the cost of purchasing the different options, the incremental cost per incremental value ratio(s) (ICVR) could act as the basis for resource allocation in a way comparable to that of an incremental cost effectiveness ratio (ICER): options with lower ICVRs would be interpreted as more valuable and efficient and would as a result be prioritised.
The resulting value index and its value scores would be context-specific reflecting stakeholder preferences: first, the value index incorporates value judgements and preferences for a set of options based on a group of criteria, all of which are informed through stakeholder input. Second, the MCDA process, as proposed, respects stakeholder preferences in individual settings, whilst reducing decision heterogeneity by introducing clarity, objectivity and greater transparency about the criteria based on which decisions can be shaped. Third, MCDA has the potential to be used in assessing the value of health care interventions in the context of HTA, mainly because of its robustness in terms of the multiplicity of criteria that can be used to assess value, its flexibility in terms of differential weightings that can be applied to the identified criteria, and its encompassing nature in terms of extensive stakeholder engagement.
Given that MCDA is a departure from conventional HTA approaches, a roadmap would be needed on how MCDA could be factored in current HTA practices. First, any efforts of MCDA implementation should start by building a multi-disciplinary research team with the appropriate technical expertise as part of an education phase, spanning the fields of decision analysis, medical and life sciences, health economics, and statistics. Second, a number of hypothetical pilot studies could be carried out in a testing phase, which would act as testing exercises, in order for the HTA agencies to gain a first-hand experience on the technical aspects of the MCDA process and exploring a variety of MCDA methods and techniques in the first instance. Third, actual case studies could subsequently be conducted as part of a transition phase, using evidence from past health technology appraisals. Value judgements and preferences could be elicited from actual members of an agency’s appraisal committees while acting as participants in decision conferences taking the form of facilitated workshops. The results could be used to highlight any differences in recommendations with past appraisals, realising the benefits and insight of the methods used as part of real practice. Finally, the MCDA approach could become fully operational as part of an execution phase, initially running in parallel with any existing formal appraisals.
Work Package 3 (WP Lead: ISS, Italy)
• WP3 sought to understand whether rarity has an impact on the assessment of the incremental cost per QALY of orphan drugs compared with drugs for more common diseases and whether it impacts mainly on the incremental costs or on the incremental QALYs. Additionally, it seeks to develop and validate a framework to support decision-making relating to orphan drugs for rare diseases, by means of a Multi-Criteria-Decision framework, more specifically using a discrete choice experiment (DCE) and, through that, canvassing views of decision-makers and patients.
1. HTA of Orphan Drugs: How are Countries Dealing with Rarity?
Through the application of a methodological framework (Retrospective Analysis Framework) developed within WP2, the systematic comparison of 35 drug-indication pair appraisals enabled a better understanding of how different HTA bodies conduct value assessments for orphan drugs, and whether orphan drugs have a special status. The methodological framework adopted a mixed methods research design comprising two stages: (1) qualitative in-depth analysis of the decision-making processes; and (2) quantitative identification of agency-specific risk and value preferences through correspondence analysis, and agreement levels across countries through Cohen’s kappa scores. The framework was developed on the basis of in-depth case study analyses and the creation of a coding manual, and was applied to the same sample of orphan drugs in four countries (England, Scotland, Sweden and France (N=35)). Orphan drugs were selected because they generally undergo the same process as drugs to treat non-rare diseases, but are by definition not cost-effective because of the challenges in producing robust evidence due to small patient population and their heterogeneous nature.
Analysis of the study drugs on the basis of the analytical framework resulted in the following: (a) identification of reasons for differences at each stage of the process across countries, and (b) identification of criteria driving HTA decisions for this sample of drugs. First, levels of differences across countries were quantified using Cohen’s kappa scores, and a difference and difference matrix was created to highlight those cases where differences at each stage of the process were associated with differing HTA outcomes. Second, agency-specific risk and value preferences were quantitatively derived using correspondence analysis and were defined as the criteria driving HTA decisions in each country.
The application of this model on orphan drug decision processes enabled to understand the factors, considerations and criteria (whether these are HTA setting-specific, evidence-specific, or related to further considerations) that contribute to decisions about coverage in different policy settings for orphan drugs. Specifically, the evidence accounted for in the decision was compared and instances where different levels of evidence were associated with different HTA recommendations across countries were identified. The issues about the evidence presented and raised by the HTA bodies were also coded and categorized per type of concern (Nuncertainty=124). Agreement in the concerns raised (whether or not they raised the same concern) was poor to less than expected by chance (-0.3< κ <0.08); and in cases when the same concern was highlighted, agreement was moderate to less than expected by chance (-0.50< κ < 1.0) depending on the agency in question. Agency-specific risk preferences were also identified through correspondence analysis showing whether one country/agency was relatively more likely to raise a concern than another, also explaining some of the differences in HTA recommendations seen. This contributed to the identification of the types of concerns raised and how they were addressed across countries, facilitating cross-country learning particularly when the issues dealt with relate to limitations in HTA methodological approaches.
The scientific and social value judgments made that influenced these processes were also identified (Njudgments=125), and pertained to characteristics related to living with a disease or having a treatment. A classification framework was developed based on the literature and findings from this study, differentiating for (non-elicited) social value judgments and (non-quantified) scientific value judgments. Results also highlighted whether and how these value judgments influenced the decisions, and if they were based on stakeholder input. This contributed to a better definition of the determinants of social value, improving the lack of accountability for reasonableness and their transparency and consistency across decisions, and furthering the debate as to whether orphan drugs have a special status.
HTA bodies were subsequently interviewed to validate the findings and collect further insights about a number of methodological issues identified during the comparison. An interview topic guide was developed based on previous findings with open-ended questions. Qualitative thematic data analysis was applied to the interview transcripts using the Framework Approach. Eight HTA body representatives were interviewed between March and June 2015. Evidentiary requirements and approaches to dealing with uncertainty were discussed around: (a) trial design, (b) population and duration, (c) comparators, (d) relevant endpoints and (e) economic modelling. There seemed to be agreement that decisions regarding orphan drugs are made in a context of greater uncertainty. The threshold of acceptable uncertainty varied by country and was generally not related to the risk of not marketing the drugs. The acceptability of surrogate endpoints was not consistent across countries nor were the validation requirements. Different mechanisms were used to modulate the ICER in cases of uncertainty (e.g. Patient Access Schemes). Some countries require higher evidentiary standards for greater clinical claims, which may be more challenging for orphan diseases. The most common social value judgments identified related to innovation, disease severity and unmet need. Trivial differences were seen in the way these concepts were defined and accounted for across countries. Although agreement was seen in evidentiary requirements or preferences, there were subtle differences in the circumstances where uncertain evidence may be considered acceptable, possibly explaining differences in HTA recommendations.
This component of the WP showed substantial differences across countries in their ways of assessing value and in the different means to deal with uncertainty (e.g. stakeholder input, other considerations). The nature of the evidence presented was typical of the type of evidence generated for orphan conditions, namely relatively small trial populations, surrogate and subgroup data, lack of comparators or non-phase III trials. Although this lower level of evidence is specific to – but not limited to – rare conditions, this component of the WP showcased the different ways seen in dealing with it. In some instances, more flexibility was given because of the situation of the disease (e.g. recognised difficulties in recruiting sufficient patient numbers, orphan status) or through other means (e.g. managed entry agreements). There were also other circumstances, where this had a negative effect on the decision. Mixed scenarios were seen across and within countries, emphasising to a greater extent the complexity of these processes, where it was not possible to conclude whether orphan drugs deserve special status in practice. Despite this, findings did highlight various ways identified in dealing with these issues related, but not limited, to rare diseases.
2. A DCE to understand decision-makers’ perceptions
In order to understand whether rarity has an impact in assessing the cost effectiveness of orphan drugs compared with drugs for more common conditions and whether rarity impacts mainly on the incremental costs or on the incremental QALYs, we used the evidence collected on costs and effectiveness/quality of life across a set of orphan conditions in order to quantify the socio-economic costs and Health Related Quality of Life (HRQOL) of patients affected by rare diseases. A methodological analysis has been carried out in order to understand how various factors might alter relevant cost-effectiveness thresholds. A systematic literature review was performed to identify the attributes to conduct a discrete choice experiment (DCE) survey across several countries and on decision makers and patients with rare diseases to identify the criteria leading HTA decisions related to rare diseases. This study revealed that attributes such as improvement in health, cost of treatment, side effects, waiting time, severity of illness, availability of other treatments and value for money should be considered to better capture societal preferences and their implications for HTA. Taking into account this information, a DCE survey was designed and applied to a representative sample of 199 decision makers, using an online questionnaire, across five EU Member States (England, France, Germany, Italy and Spain). The data obtained from the DCE survey was used to estimate the weights associated with each of the criteria by means of regression models. Logic and probit model coefficients have been tested. Choice data were used to consider the relative importance of changes across attribute levels, and to model utility scores and relative probabilities for the full set of combinations of attributes and levels. The five country models show preferences for some attributes over others. “Cost of the treatment” (Spain, France, England and Italy), “improvement in health” (Germany, France, England and Italy), “value for money” (England, Germany and France), “availability of other treatment” (England and France), “waiting times” (Germany, Spain and Italy), “side effects” (Italy) and “beginning of life” (Germany) are the attributes receiving the highest attention, while “importance of the disease” (France, England and Germany), “value for money” (Spain), “availability of other treatment” (Italy and Spain), “waiting times” (France and England) “beginning of life” (Italy) and “side effects” (France and Spain) were found to be less important. Further analysis could be made in terms of finding classes of respondents with similar patterns using latent class models when sample sizes allow this.
3. A DCE to understand patient perceptions
A similar DCE survey was undertaken amongst patients by furthering the knowledge and testing empirically the effectiveness and societal value of orphan drugs in patients with two rare diseases: cystic fibrosis and haemophilia. Patient registers can help in the assessment of effectiveness and societal value. A specific DCE questionnaire available online was elaborated to collect information from 54 patients with cystic fibrosis and haemophilia in Italy about the societal values for orphan drugs with a view to further eliciting preferences across a number of rare conditions. The findings provide evidence about how patients with cystic fibrosis and haemophilia think that decision should be made in Italy when considering which health technology scenarios are more appropriate to receive funding. “Improvements in health”, “the cost of treatment” and “value for money” were the attributes receiving greatest attention from patients with rare diseases, while “importance of the diseases” and available of other treatment” were less important for patients with rare diseases. The findings presented provide valuable evidence for future designs following similar approaches. The DCEs conducted in this study provided valuable insights regarding benefit valuation and are therefore useful as additional information to complement QALYs when assessing health care interventions.
Work Package 4 (WP Lead: IER, Slovenia)
Recent literature does not provide an irrefutable answer on whether differences between patient and general public preferences exist. There are some empirical studies demonstrating no significant differences between patient and general public sets, while others have shown that although preferences were dependent on whether they were derived from the general population or from patients, QALY gains were similar between groups. The evidence of the differences between the sets has however been growing. In this work package hypothetical values of the general public to hypothetical values of the patients were compared. This was a departure from debating experience-based vs. hypothetical health state valuation. Specifically, the WP generated new datasets from: (a) face-to-face interviews with Spanish patients with a diagnosis of breast cancer or rheumatoid arthritis using the EQ-5D-5L, and (b) surveyed patients on quality of life across countries and diagnoses.
1. Preferences drawn from general and patient populations
The advantages and disadvantages of the preferences drawn from the general population and patient populations have been defined based on literature review. According to the groups arguing in favour of preferences being drawn from the general public, their major advantage is the fact that they better represent taxpayers and potential patients. Besides, they are supposed to be behind the “veil of ignorance”, meaning that they are blind to their own self-interest, therefore establishing principles of justice which would ensure fair distribution of resources. The review, however, revealed that this argument is questionable and does not sustain theoretical justification: a thick veil is in conflict with the requirement that members of the community must be at least partially informed about the health status in order to be able to attach utilities to it. The next argument connects to the issue of adaptation - utilities elicited from patients are supposed to be higher than those from general population due to adaptation to disease. Adaptation has many sources, two of them being cognitive denial and suppressed recognition. They both are seen as negative aspects of the process of adaptation as they relate to difficulty of admitting how poor a patient’s health really is. The other sources of adaptation are skill enhancement, activity adjustment and substantive goal adjustment, which are seen as positive for patients. In case adaptation is comprised primarily of cognitive denial or suppressed recognition of full health, it becomes an inappropriate influence on preferences and health care allocation priorities. The other arguments to support general population preference elicitation seem to be more practical: certain patient groups are unable to attach utility to health states, there seem to be not enough patients in certain diseases as well as patient preferences cannot be used for comparisons across interventions due to their differences. On the other hand, the most obvious and most commonly mentioned argument which supports the use of patients’ preferences is that the general public lacks the experience with impaired health that forms a valid basis for valuation. This problem and its effects on ratings can never be entirely overcome. Furthermore, it is supposed to be easier and far less expensive to survey a convenience sample of patients than it is to survey a probability sample of an entire nation’s population.
2. EQ-5D-5L administered to breast cancer and rheumatoid arthritis patients
Based on HRQoL data, acquired from face-to-face interviews of 611 breast cancer and rheumatoid arthritis patients in Spain, following the Euroqol (EQ-5D-5L) protocol and achieved with the direct assistance and involvement of Euroqol, a database was created. The analysis of the data revealed that patients, regardless of the disease, have similar preferences towards hypothetical health states, yet different to the general population. Comparison of health states ranked by severity (from best to worst) shows that eight better health states are valued higher by the patient population, while the last eleven states hold lower average values compared to the average values in the general population. While the difference between the utilities did not differ a lot for health states where the patients’ valuations were higher, the difference was bigger for those health states, where the patients’ valuations were lower (worse health states). The average absolute difference between general and patient population is 0.232 while the average absolute difference between RA and BC patient group is 0.073.
The acquired data also enabled the examination of the causes of the established differences between general population and patient group preferences. We found statistically significant differences between patients’ valuations and general public valuations on three health dimensions: mobility, pain or discomfort, and anxiety or depression. Patients consider problems related to mobility less problematic, while problems related to pain/anxiety are seen as more problematic. The greatest disagreement is on the dimension of anxiety or depression. The results suggest that not all health dimensions defined by EQ-5D are equally well understood by the general public. A possible explanation for the differences in the valuation that is suggested by the regression analysis is that problems with mobility, self-care, and usual activities are easier to imagine and understand by the general population. The easiness to understand and visualize problems on these three dimensions might explain why differences do not exist or are not as large as those with pain/discomfort and anxiety/depression dimensions. The problems on last two dimensions, with their non-tangible nature, are more difficult to imagine to general population behind a veil of ignorance.
In order to define the advantages and disadvantages of patient and general population preferences, we examined the agreement of arguments given in the literature with our results. Our results show quite the opposite picture to the findings from the literature reviews. We found no significant difference in preferences between the groups of patients with different diseases. Therefore, the most commonly cited advantages of general population preferences seem to be irrelevant: the small number of patients in certain patient groups, patient being unable to answer and absence of comparison possibilities across patient groups are no longer a valid argument. The results have also shown that adaptation in the case of HRQoL no longer possess a normative problem, as the effects of adaptation on hypothetical health states valuation are trivial. In our opinion and suggested by the results of the analysis the differences exist not because of adaptation but because patients have experience with the impaired health and can more accurately imagine hypothetical health states. For that reason, preferences based on patient population will not exhibit higher utilities and hence lower priority with respect to treatment, but on the contrary, with the exception of mobility, receive more priority.
Regarding the argument stating that the general population is the preferred source of preferences because they are behind a ‘‘veil of ignorance’’ and blind to their own self-interest, one would expect to observe the highest differences on a health dimension with largest prevalence discrepancy (usual activities). However, no statistically significant differences were observed on that dimension between general population and patients. It seems that a “veil of ignorance” is too thick and fails the requirement that community must be at least partially informed about the health status of individuals as the general public lacks the experience with impaired health that forms a valid basis for valuation. The argument of easier and less expensive survey of convenience samples of patients does not hold in our research as we studied hypothetical health state utilities elicitation by patients and not their own health state valuations. In order to elicit utilities for hypothetical health states, sample size is not smaller and the elicitation process seems to be even more expensive due to special training required for interviewers eliciting preferences from patients.
As far as it is safe to claim that preferences based on patient and population values differ, further data would be needed to determine differences in health state utilities for all health states. The results of our analysis suggest that the differences in preferences stem from patients being more able to accurately imagine “non-tangible” dimensions of health states (anxiety or depression and pain or discomfort) than the general population with less experience in various health states. Consequently it is recommended to use patient based value sets for resource allocation. A departure from general population value sets to patient-based value sets would result in a change of priorities. According to the patients, treatments of conditions/health states that mostly affect mobility should receive less funding, while treatments for diseases, where the dimensions of pain/discomfort and anxiety/depression are more relevant, should receive higher funding compared to the current decision-making processes where preferences of the general public are used to set priorities.
In order to further extend and test the results of our study the whole value set based on patient population preferences should be calculated and tested in a pre-selected HTA model for diseases where health states have various stress on dimensions: either on mobility or on pain/discomfort and anxiety/depression. The differences in impact of using both preference sets should be closely observed and differences determined. As the elicitation of preferences from patients is much more demanding and difficult as the elicitation of preferences from general population, possible cross mapping could be considered in order to deter the need for patient preference elicitation. A question remains open of comparability of patient preferences across countries. Further research is needed to determine whether patterns and relations to general population sets across countries are the same as in Spain.
3. Main findings from the online QoL survey
767 web-survey responses, across 38 countries and 122 disease areas demonstrated that patients had an average utility loss of 24% compared to the general population, although disease-specific health state utility (HSU) variations were also highlighted. The survey results regarding average utility losses mirrored the results reported earlier in the context of face-to-face interviews among patients. Blood cancer and asthma patients had an average utility loss of 12%, in contrast to Ehlers Danlos Syndrome (EDS) and Rheumatoid Arthritis (RA) patients for whom utility losses amounted up to 53% and 27% respectively. The highest utility loss was exhibited by EDS patients, shaping a recommendation for health-care professionals to take a greater interest in understanding the widespread manifestations of this rare disease.
QoL deterioration was largely attributed to the domains of Pain/Discomfort and Usual Activities, where problems of all severity levels comprised 78% (16% severe and/or extreme) and 71% (15% severe and/or extreme) of all responses for each domain respectively. On the contrary, 76% of individuals were problem-free in the dimension of Self-Care. Discrepancies on the extent to which each EQ-5D domain impacted upon QoL outcomes across diseases, suggested that similar EQ-5D utilities (i.e. HSU of 0.7 for both breast cancer (BC) and Myelodysplastic Syndrome (MDS) patients) reflected different severities within domains depending on the disease group under investigation (i.e. anxiety was the most and least problematic domain among BC and MDS patients, respectively).
Country-specific variation on QoL preferences was also observed, such as a statistically significant (p<0.05) health utility difference between France (0.53) and Denmark (0.78). Inherent cultural differences, country-specific health expenditure and health behaviour likely impacted upon a population’s QoL, and should be considered when developing strategies to address health inequalities both within and between countries.
It was also demonstrated that EQ-5D-5L missed important health aspects in approximately 51% of a general, chronically ill population, even though this percentage fluctuated between therapeutic areas. The representativeness of EQ-5D-5L was largely questioned by EDS (60%) and cancer (52%) patients, whereas lower percentages were observed when asthma and RA patient groups reported on important QoL aspects not captured by EQ-5D-5L (45% and 42% respectively). 17 additional QoL dimensions were identified, the most common being fatigue (19.5%) and medication side effects (9%). Finally, 57% of patients reported that aspects of their illness that had a big impact on their health changed over the course of their disease, and of these aspects fatigue was also the most commonly reported (17% of responses). Fatigue can be related to both disease and treatment and has been associated with multidimensional consequences ranging from functional impairment, interference with relationships and performance of daily activities. However, its importance as a bolt-on item to the existing EQ-5D has been controversial, underscoring the need to understand better why patients report it as being highly important to their wellbeing. Overall, it was shown that EQ-5D-5L in HRQoL measurement raises inconsistencies in capturing QoL attributes in disease-specific patient populations, however further research is needed to clarify how patients’ disease-specific utilities map to generic tools and ensure that these capture aspects important to patients.
Work Package 5 (WP Lead: TUB, Germany)
The work carried out in WP5 has provided important insights into the suitability of existing HTA assessment tools for medical devices (MDs), taking into account similarities as well as differences across different device categories. Moreover, it has clearly identified areas where knowledge and tools are lacking or can be expanded.
1. Current European HTA practices with regard to MD assessment
A systematic identification of HTA institutions across Europe involved in HTA of MDs as well as a systematic review along with a website search for specific information was carried out. The systematic search encompassed the collection of information on (a) institutional methodologies, (b) processes and (c) practices, including a search for and analysis of methodological documents.
From the identified 84 institutions involved in HTA in Europe, 47 institutions are actively involved in the commissioning or performing of HTA reports on MDs (assessment and/or appraisal). Sufficient information was not publicly available for all institutions. For example, details on how technologies are prioritized could be obtained from less than half of the institutions. There still exists variability in the understanding of what the term “MD” entails, which is also reflected in different structural, procedural and methodological elements among institutions. Regarding the institutions with publicly available HTA reports on MDs, our results show that the number varies from 1 to more than 200 assessment reports per institution surveyed.
Although a large number of methodological documents (n=52) were identified, only five institutions developed specific documents (n=9) for the assessment of MDs. Three of the nine identified MD-specific documents are intended for other stakeholders rather than for the institution’s own assessments. Interestingly, five out of nine separate documents focus on diagnostic technologies (including tests). Similarities between documents for internal use are mainly related to the type of preferred evidence and outcome parameters to be considered, appraisal of evidence quality and stakeholder involvement. Differences identified refer mainly to the assessment base and the comparator used, and thus largely reflect the different types of devices addressed (diagnostic vs. therapeutics). Institutions such as NICE and HAS address additional parameters, for example learning curves or utilization setting, as crucial elements that also need to be considered. Only few institutions made separate provisions for the assessment of MDs in their general methodological documents (e.g. HIQA). This reflects that certain methodological evaluation steps, which are described in a general methodological paper, apply to all types of technologies including MDs. Of those, some have separate sections in their documents dedicated to MDs (e.g. IQWiG) concentrating primarily on diagnostic technologies.
Despite growing recognition of their importance and related international initiatives, such as the EUnetHTA Core Model, specific tools for the assessment of MDs are rarely developed and implemented at national level. Separate additional signposts incorporated in existing general methods guides may be sufficient for the evaluation of MDs.
Although HTA methodology already considers different approaches depending on the therapeutic or diagnostic nature of technologies, other elements such as device-operator interaction, duration of patient-device contact, level of device activity or combined diagnostic/therapeutic functionality require further methodological discussion.
2. Developed taxonomy: testing and usefulness
To form a comprehensive taxonomic model the classifications included in the EU licensing process for medical devices (Directives 90/385/EEC, 93/42/EEC, 98/79/EC), the international classification for medical devices used by the Global Medical Device Nomenclature (GMDN) as well as the categorization employed in the OECD Classification of Health Care Functions and elaborated on by the EUCOMED project were analyzed in combination. A matrix in table format (see Figure 4 in the appendix) was created based on relevant aspects from the existing classification schemes, incorporating the elements of risk and role/functionality of device types. The matrix further incorporates an additional distinction between the diagnostic or therapeutic nature of devices, which can be crucial for HTA purposes.
1006 reports of different types and length addressing 1234 technologies produced by 32 European institutions from 17 countries were collected to test the plausibility of the normative taxonomic model. Overall, the distribution of identified reports on the matrix generally confirms that the taxonomy is plausible. Only one report was identified for one grey field where no HTAs were expected. The majority of reports in the sample addressed technologies from the green fields, considered of high relevance. Relatively few reports were available for the red fields considered of low relevance. Some correlation to risk level and type of device is apparent, at least for devices that are implanted or serve to assist medical professionals.
Within a survey including 16 European HTA institutions, interviewees were asked about the usefulness of the taxonomy, especially for their institutions’ own work. Many of the interviewees stated that the taxonomy could be useful. However, only four found that it could be used directly by their institutions, while seven stated that it would not be directly applicable for them. Four interviewees generally saw the taxonomy as a helpful tool but did not specify for whom. Finally, two interviewees assessed the taxonomy as not useful or were unsure about its added value. Suggestions for refinement and further development of the model were also given.
Based on the described plausibility testing in conjunction with the opinions from HTA institutions illustrated above, we believe that the taxonomy developed in this WP can be useful for HTA institutions and decision makers (e.g. MoH, insurers) alike. It can serve as a support tool to (a) select topics for assessment and (b) identify certain aspects/particularities that require tailored (methodological) approaches. Work on detailed recommendations on these aspects continues beyond the end of the project.
3. Methodological considerations with regard to the developed taxonomy
In total, fifty-five HTA reports from the large HTA report pool mentioned above, covering 6 case studies (e.g. PET/CT) were extracted and analysed with regard to specific methodological aspects to explore similarities and differences across taxonomic positions. These reports were published between 2010 and 2014 and produced by a total of 15 institutions.
With regards to dimensions of evaluation and using the EUnetHTA Core Model domains as reference, we identified that for MDs of lower risk categories safety and economic aspects seemed to be of lesser importance. For implants (e.g. cochlear implants, artificial joints), social, ethical and legal aspects seemed to be of higher importance. Organizational aspects (e.g. learning curves) appeared to be of greater relevance for taxonomic positions, which entail a MD used within a procedure.
Regarding the level of evidence a lack of high quality evidence in most cases was reported. Reports assessing implants more frequently identified high quality studies. Almost all reports, irrespective of the case study, were based primarily on direct evidence and on independent research rather than manufacturer submissions. Most institutions included both primary and secondary evidence. The majority of assessments considered an active comparator, while placebo or sham interventions were used mainly in reports of low risk devices.
With regards to the endpoints, all case study reports considered mortality, morbidity, QoL and safety. In the evaluation of ‘Risk class I’ MDs mortality seemed to play a minor role, in contrast to reports regarding ICDs. Most institutions used existing checklists/tools to appraise the evidence (e.g. QUADAS 2) but also a lack of fitting checklists, e.g. for surgical RCTs, was mentioned.
Based on the information included in the reports, the majority of institutions included stakeholders in the various stages of evaluation, except evaluations of ‘Risk class I’ MD. The types of stakeholders involved varied by agency. The type of approach for economic evaluation was agency-specific (e.g. simple mention of costs, literature review or own economic evaluation) and depended on the agency remit and potentially also financing streams for MDs.
This analysis of HTA report showed that there were some differences across taxonomic positions with regard to aspects such as acceptable level of evidence or chosen comparators in current HTA practice. Furthermore, input from HTA institutions provided in the interviews supported that a common understanding on methodological requirements and specific tools as well as further research (e.g. GRADE for prognostic studies) was vital and needed, including both the regulatory and the HTA-doers’ side.
We formulated ‘pointers’ addressing certain methodological aspects that could be applicable to specific device types (and, therefore, taxonomic cells) and could/should be considered along with the regular methodological approach adopted by each institution. Recognizing that larger samples and additional, more in-depth analysis would be required to reach unequivocal conclusions, we consider these recommendations work in progress. Three examples of such ‘pointers’ are presented below.
The first example relates to the EUnetHTA Core Model elements. The EUnetHTA Core Model™ for Diagnostic Technologies was frequently mentioned by interviewees as a methodological basis. It can serve as a basic understanding on how to assess diagnostics. The Core Model™ iteration specific to Medical and Surgical Interventions and Screening Technologies can also be used as the basis for assessing related technologies. Based on our findings, the following aspects could be taken into account for specific taxonomic positions: First, for MDs of lower risk categories: safety and/or economic aspects might not be fully touched upon; however, closer inspections of these elements maybe of importance, for example, for low cost MDs broadly used in the healthcare system; second, for certain implanted devices and devices that require user skills, social, ethical and legal aspects have to be considered in greater detail; third, taxonomic positions, which entail a MD used within a procedure require that organisational aspects have to be explicitly considered (e.g. by using OSTEBA’s checklist).
The second example relates to the selection of an active or inactive (placebo/sham intervention) comparator to be considered on a case-by-case basis . In principle, the use of an active comparator is generally recommended. Pragmatically, however, there are cases where placebo/sham interventions may be acceptable for comparison. For example, in the assessment of MDs of lower risk classes, placebo or sham interventions can be chosen without major ethical concerns or limitation in the resulting evidence. Similarly, in circumstances where an active comparator is not ethical and/or feasible, a placebo control arm might also be appropriate. In contrast, high-risk MDs should have active comparators to demonstrate comparative effectiveness and safety.
The third example relates to the quality assessment of studies. We identified certain quality appraisal checklists and tools for specific study designs that are used quite often in current HTA practice. These differ mainly between therapeutic and diagnostic technologies. Based on our results, QUADAS 2 is indicated for diagnostic studies and also recommended by EUnetHTA . For therapeutic studies, tools vary depending on study type and domain addressed (e.g. effectiveness). Non-randomized controlled studies (NRS) of lower level of evidence were often accepted in existing HTA reports. While it is acceptable to take these into account for some types of technologies (see above), we strongly recommend the use of a quality appraisal tool to evaluate NRS studies before they are included in the body of evidence. EUnetHTA recommends the instrument ‘ACROBAT-NRSI’ by the Cochrane Collaboration as the most suitable for this purpose, a recommendation confirmed by the interviews conducted in this work.
4. Regulatory recommendations based on primary data collection
Although WP5 mainly focused on the methodological implications of MD assessment, relevant potential regulatory changes also merit attention. While a broad range of institutions with different levels of experience participated in the survey carried out within this work, a uniform emerging statement was that regulatory changes needed to be made in order to facilitate and strengthen their work. Interviewees wished to have stricter regulation for MDs, which includes evidence-based requirements on effectiveness and safety for receiving marketing authorization (CE mark). More specifically, these changes should entail:
• A singular system for marketing authorization across EU countries as it already exists for drugs;
• A requirement for data including high quality studies such as RCTs but also studies addressing organizational aspects (e.g. learning curves);
• A transparent and publicly available system/database that provides an overview of all licensed MDs at the European level including marketing authorization documents, prices, instructions and indications for use as well as other information;
• Mandatory trial registration and publication: Obligation for studies on MDs to be documented in a central register and to be published, once results are available, regardless the nature of these results.
In addition, interviewees emphasized that more attention and focus on MDs on behalf of decision-makers, including a more structured and systematic planning for their introduction to the market was necessary. The need for greater interaction of all parties including manufacturers and patients but also between regulatory and reimbursement bodies was also raised. Finally, initiating a discussion about what needs to be assessed before reimbursement decisions for MDs are taken was seen as crucial. Given that European regulation is currently being revised, there is an opportunity for insights from our work, especially the opinions of 16 HTA institutions, to contribute to the discussion and hopefully help ameliorate the current situation.
Work Package 6 (WP Leads: EASP, Spain & PAHO, United States)
This WP had two key objectives, notably to map and understand the HTA activities in two geographical regions (Latin America and Eastern Europe), and to create a toolbox that brings the collective wisdom on current and likely future HTA policies and practices to settings where HTA is currently not used.
1. Mapping HTA policies and practices in Eastern Europe and Latin America
This exercise included a mapping of HTA activities in Central, Eastern and South-Eastern Europe (CESEE) and Latin America and the Caribbean (LAC) countries. This exercise encompassed the design of two surveys: (i) diagnosing capacity in HTA and (ii) HTA decision-making processes.
In relation to capacity to perform HTA, in CESEE a total of 41 responses were obtained (response rate 19.8%) and 31 from LAC (response rate 67.8%). The main activities of respondents were health policy (65.5%) and HTA (62.1%), followed by organization and management of health care (51.7%) and research (48.3%) (non-mutually exclusive options). Significant heterogeneity was observed in the characteristics of respondents (e.g. affiliation, responsibility, HTA experience). Similarities in the barriers faced by the institutions to which respondents were affiliated were observed and the most common barriers faced by those performing HTA were skills training and sources of funding in CESEE, while skills training and institutional support were the most cited in LAC. In CESEE countries, the majority of respondents reported having partnerships with domestic and international institutions. Requests to perform HTA were generally received from government (62.1%) and private companies (31%). Some respondents indicated that, at that moment, they did not perform HTA, because of a lack of political interest and funding. In LAC countries, 22% of respondents indicated that their institutions undertake HTA, while 15% reported coordinating HTA activities, and being involved in pricing and coverage/reimbursement decisions. However, a majority of respondents (30%) indicated that they participated in other activities, including policy development, regulation, HTA training, research on the impact of sanitary measures, public health research, and epidemiological surveillance.
In relation to the HTA decision-making process, data was obtained in a number of key dimensions as follows: first, technology selection and type of technology assessed for decision-making process: For both CESEE and LAC countries, the most cited criteria for selecting a technology for assessment were the frequency of the clinical condition (prevalence, incidence) and/or disease burden (mortality, morbidity and quality of life). For CESEE countries, budget impact was the third most common criteria, while this position was occupied by ethical, legal or social implications (equity criteria included) in LAC. The second dimension was potential conflicts of interest (CoI) addressed when undertaking HTA; conflict of interest may significantly affect the outcome of the HTA process. In many settings staff involved in HTA have to confirm in writing that they do not have CoI, whereas in other settings, respondents indicated that there were no established procedures to manage CoI, or that they did not know the procedure. The third dimension was how HTAs are ultimately linked to decision-making; countries with legislation establishing that HTA must be considered in the decision-making process as mandatory are Slovakia, Latvia, Greece, Poland, Czech Republic, Estonia, Slovenia, Bermuda and Dominican Republic (although in some cases respondents highlighted that there is no link and the decisions are not informed by HTA). The legislation in Estonia, Poland and Surinam also considers HTA to support coverage decisions as recommendation. Countries such as Croatia, Lithuania, Russia, Hungary, Belize, Costa Rica, El Salvador, Jamaica, Panama and Saint Lucia did not have specific legislation at the time of the survey stating that coverage decisions should be informed by HTA reports, but HTA reports have been used to support policymaking. In Serbia, Honduras, Costa Rica, Dominican Republic, Saint Maarten and Trinidad and Tobago there is no link, and the decisions are not informed by HTA. The fourth dimension of interest concerned stakeholder involvement in the decision-making process; Latvia, Poland, Czech Republic, Lithuania, Russia, Estonia, El Salvador, Suriname, Saint Lucia stated that HTA entities involve stakeholders in the HTA process, and submissions of evidence are required from stakeholders. In Poland, stakeholders are involved in the HTA assessment and appraisal stages, but not in decision-making. Only the HTA organization from the Czech Republic allows stakeholders to comment on HTA at the draft stage. The fifth dimension examined was the type of HTA sources that could de distinguished according to the institution that drafted the report; three such sources were identified, notably, internal reports produced by the Ministry of Health, Social Security, or HTA agency (which are always or often taken into consideration in decision-making in Barbados, Bermuda, St. Maarten, Slovakia, Latvia, Poland, Czech Republic, Croatia, Lithuania, Estonia, Bulgaria and Russian Federation; by contrast, in Slovenia, Honduras and Trinidad & Tobago, decision-makers never take any input from internal reports), reports commissioned externally by the Government (St. Maarten, Panama, Lithuania and Bulgaria take into consideration these reports regularly) and information presented by the industry requesting the technology being publicly subsidized or incorporated in a benefits package (Panama, Latvia, Czech Republic and Croatia considered this type of report for the decision-making process. Finally, a number of other dimensions were examined, notably: requirements for economic evaluations (required in El Salvador, Latvia, Poland, Czech Republic, Rumania, Estonia, Bulgaria, Slovakia and Hungary); and the timelines for HTA completion (ranged from 90 to 365 days).
2. Toolbox on Best Practices in HTA
The other key objective of this WP concerned the development of a toolbox outlining best practices and including recommendations for HTA, based on the information gathered in earlier phases of the research. The Toolbox was divided into five sections:
I. Healthcare system and HTA: Offers a general view of what HTA is and its use. It provides information about the functioning of HTA systems in specific settings, illustrating diverse HTA practices.
II. Building the HTA function: Offers examples of types of HTA agencies and their performance. In order to assist in the harmonization of methods and information sharing, a selection of HTA networks is shown. Recommendations, examples, and tools aiding the analysis of social or ethical values are compiled.
III. HTA products: Discusses types of evidence to inform decision-making, and the structure and contents of a typical HTA report. It offers the key elements to read and understand the different typologies of HTA reports, with hyperlinks to relevant repositories.
IV. Beyond HTA: Focuses on multiple Criteria Decision Analysis (MCDA) as a tool to facilitate priority-setting and decision-making.
V. Decision implementation: Offers tools and examples of the adaptation of guidelines, disinvestment & reinvestment decisions with country examples, and monitoring and evaluation of HTA impact.
Work Package: WP7 (WP Leads: UCLM, Spain and LSE, UK)
Work package 7 synthesized the results of the previous six work packages in horizontal and vertical terms. While the horizontal aspect outlined the results of each of the previous work packages, the vertical aspect underscored a number of dimensions related to HTA, the resulting policy implications and the broader socio-economic context in which Advance-HTA operated, notably: (a) the theoretical and conceptual considerations addressed by Advance-HTA; (b) the importance of appropriate service user involvement in HTA evaluations; (c) the orphan medicine treatments and the value of HTA in industrial policy formation; and (d) the implications for global health and health care development. These are briefly discussed below.
1. Theoretical and conceptual considerations addressed by Advance-HTA
Some of Advance-HTA’s results in general show that differences in the evaluation criteria underlying how products such as innovative medicines are assessed can (over and above issues such as variations in national per capita GDPs) consistently and significantly influence whether or not a treatment is judged cost-effective in one country as opposed to another. This does not mean that the work undertaken by health economists and others involved in HTA leads to arbitrary decisions. But it does mean that if more or less value is ascribed to, say, the scientific originality of a therapeutic innovation (and so, implicitly, the unknowable but potentially important long term returns to communities from publicly or privately funded biomedical research investment) relative to the immediately demonstrable benefits generated for individual patients, then differing purchasing and treatment supply decisions will result. Similar points apply to the use or non-use of ‘adjusting’ factors such as disease severity and rarity or ‘end of life’ care applications.
From a pan-European policy perspective this raises social equity concerns and a number of wider economic and industrial policy questions, alongside more immediate health sector resource allocation issues. One important challenge for the future relates to ensuring a better balance between achieving short-term equity in health care areas where there is already evidence available relating to the promotion of allocative efficiency and the inevitably more uncertain task of defending public interests in long-term innovation, and the eventual achievement of fundamentally improved technologies for preventing and treating conditions such as cancers and neurodegenerative disorders. These are different goals, which in some circumstances conflict. Similar questions and possible consequences arise in fields ranging from how affordability criteria are set in relation to ‘cost per QALY’ and allied utility measures in different nations and/or contrasting spheres of social and economic activity in a single country, through to – not least in the rare disease and cancer care contexts – whether or not rationing choices should be made on the basis of Benthamite utilitarianism as opposed to Rawlsian concepts of social justice. The latter are more likely to ascribe premium values to treatments that benefit minority populations than the former.
Other important Advance-HTA findings range from new observations showing that patients tend to ascribe higher relative values to physical pain and mental distress reduction than observers who have not experienced relevant disease states directly, through to revealing the limited extent to which HTA techniques have to date been applied to the evaluation, pricing and purchasing of medical devices as compared to that of medicines and vaccines. Its existence stems from the relative lack of centrally accessible data on medical devices use and the outcomes attributable to them, and the fact that their employment is often intimately associated with other forms of hospital spending. The special focus of HTA on drug evaluations is also linked to the (in some ways questionable) Western pharmacological tradition of evaluating medicines as discrete molecular entities, as opposed to items that act in combination with not only additional drug treatments but many other medical, surgical, nursing, psychological and wider social inputs.
Advance-HTA’s findings indicate that further investment needs to be made in developing Health Technology Assessment tools for use in areas outside the narrowly defined pharmaceutical sector, in order to further increase the productivity of health care as a whole. This stretches beyond conventional technologies and includes diverse health technologies such as combination therapies and companion diagnostics, among others. They may also need to be adapted to accommodate the reality that drugs do not normally act alone to achieve the best possible outcomes, albeit the discovery of optimum combinations and administration strategies can – as experience in spheres such as the treatment of child leukaemias has shown – be a long drawn-out process.
2. The importance of appropriate service user involvement in HTA evaluations
From a public policy perspective some of the strongest findings of the Advance-HTA research discussed here stem from observations made in relation to preference elicitation, and in the rare disease treatment context. These include the discovery that people who have experienced conditions such as physical pain or depression (and perhaps other psychiatric states such as, for example, severe anxiety disorders) are likely to value their alleviation more highly than those without such direct experiences. At the same time reductions made in the degree of QoL lost recorded via instruments like the EQ-5D-5L in the context of problems that can more readily be accommodated, such as being unable to conduct self-care tasks autonomously, are likely to be less highly valued by patients with relevant experiences than they are by ‘naïve’ observers.
Although the likely impact of such variations on quality of life estimations should again not be over-stated, such variations are potentially significant with regard to the ways that HTA research might best be conducted. This is particularly so when they are combined with findings that demonstrate that in areas such as rare disease research it is often the case that patients and their family members have deeper and more accurate insight into their conditions than many clinicians, let alone academic or directly State employed economists.
Given this, the key policy related conclusions drawn here are not only that there are opportunities for technical improvements in the ways in which the quality of life related impacts of health care interventions are measured, but also that closer patient involvement in the governance and direction of HTA and other forms of health care research and evaluation remains an important priority. At worst, patient involvement can be little more than a form of ‘box ticking’. But done well it is likely to add significantly to the quality of HTA findings, and the ways in which they are understood and implemented.
3. Orphan medicinal treatments and the value of HTA in industrial policy formation
Advance-HTA’s findings with regard to rare disease treatment and the affordability and access to orphan medicines highlight the fact that if evaluations include criteria which relate to their low prevalence and variables such as the severity of distress and other exceptional burdens that inheritable monogenic disorders can impose on not only individuals but entire families and, occasionally, on specific ethnic/racial minorities, then they are more likely to be judged affordable than if more general assessment methodologies or processes are used.
In linked areas like some areas of cancer care (and probably in future some forms of dementia treatment) patient access is also likely to be affected by the existence or otherwise of additional HTA evaluation modifying factors. These could relate to not only the value of demonstrating social solidarity in situations where only relatively small numbers of people are involved, but also the social fact that at given points in history some disease may be popularly seen as deserving greater investment than others. Such prioritisation judgement may be driven by subjectively perceived fears or objectively based beliefs that some sorts of innovation are more likely to prove viable than others, despite having equal potential ‘worth’.
Two main sets of policy related ideas are worth discussing here. First, the European Union, like nations such as the US and Japan, has in recent decades recognised the important health equity and industrially funded research challenges associated with developing better treatments for rare indications. Special incentives have been put in place to promote relevant activities. The EU has also encouraged Member States to develop rare disease strategies, which they have done at varying speeds and with differing degrees of excellence. But there is in the short term at least little point in bringing new treatments for rare disorders to market if they are not subsequently made available to those who could benefit from them.
Some commentators may take this to imply that in the European context more effort should now be made to facilitate the timely production of orphan medicine assessments produced to an agreed Union-wide standard and implemented in a manner consistent with the spirit of the existing European legislation on rare diseases. Whether or not this will be achievable is uncertain. But Advance-HTA clearly identifies this as a question that will demand increasing attention during 2016 and beyond.
Second, and following on from the above, present-day orphan drug supply issues can also be seen as indicative of the limitations of approaches to HTA that fail to take into realistic account industrial policy issues. These range from, for example, current and future EU-wide balance of trade and employment concerns to the impacts that major variations in the levels of volume demand for innovative products such as new medicines have on the economics of their development and their sustainable supply at what are perceived to be affordable unit prices.
Some experts may argue that such factors cannot properly be taken into account when calculating ICERs and should not be considered when interpreting their product pricing and purchasing implications. But if this is the case it severely limits the practical utility of HTA findings. There is a strong policy case for arguing that even if ‘cost per marginal QALY’ figures should continue to be calculated in much the same way as they are at present, affordability thresholds could and should be systematically adjusted to permit better quality future decision making. If this is agreed, then more effort needs to be put into the ‘science and art’ of determining affordability thresholds across Europe, and more widely.
Concerns like these can be linked back to the ‘utilitarianism versus Rawlsian social justice’ debate noted earlier. Benthamite thinking may, rightly or wrongly, taken to imply that minority interests should be sacrificed if greater overall welfare gains can be made by pursuing alternative social goals. But John Rawls’ theory of justice indicates that in some circumstances the (apparent) sacrifice of majority welfare optimisation interests in favour of meeting the needs of the least advantaged in a community can offer a more desirable path towards achieving what is generally accepted as being the fairest possible society-wide distribution of goods and the welfare derived from them.
Whether or not there is a genuinely irreconcilable divide at the heart of this apparent dilemma is disputable. But for the purposes of this analysis it can be said that the Advance-HTA programme’s outcomes point to the dangers of failing to understand the significant of such philosophical issues in the context of the ‘cost per QALY’ calculations currently central to the cost-effectiveness assessments undertaken by European HTA agencies. At worst, ‘simplistic’ utilitarianism could lead to an undermining of social solidarity and the intent of European policies aimed at ends like encouraging rare disease research.
4. Implications for global health and health care development
Advance-HTA also revealed marked variations in the numbers of qualified health economists and other HTA practitioners in emergent as opposed to more mature economies, and in the ways in which HTA based findings are used to determine policies and decision making. The world-wide picture is to a degree complicated by the position of the United States. There both industrial and allied research investment arguments, together with policy questions relating to the acceptability of using quality of life related calculations to determine care access, have had a significant influence on the evolution of HTA and attendant disciplines like comparative efficacy studies. In broad policy terms the emergence of health economics and within it the field HTA can be taken to be related to the processes of global demographic and epidemiological transition. These seem in their later stages to be normally attended by increased public (including compulsory insurance supported) spending on UHC provision. As health outlays increase, so do concerns for equity, efficiency and service excellence. So too do investments in aspects of quality management and value-for-money improvement. However, it should not be uncritically assumed that these are always appropriate or in themselves cost-effective. It is debateable, for instance, as to whether or not individual emerging economies should currently be seeking to increase their indigenous HTA capacity, as opposed to drawing on evaluative work conducted elsewhere and concentrating attention on further building their clinical workforces and generic health sector management capacities. Nevertheless, they will either way benefit from becoming more able to use health care resources to optimal effect. Advance-HTA work undertaken in the Americas identified an increasing tendency for local courts to require public health care systems to supply medicines to individuals seeking better health care. In other parts of the world judicial actions appear to have been aimed more at improving treatment access by limiting intellectual property rights. It would not be appropriate here to make judgments on the desirability or otherwise of such interventions. However, their existence underlines the importance of being able to price and supply products such as innovative medical devices and medicinal products in ways that are consistent with individual human rights and collective interests in achieving better care standards and ongoing global efforts to improve the effectiveness of treatments.
5. Overall
The Advance-HTA project’s findings offer European decision makers a wide range of useful insights into how in future cost effectiveness evaluations of not only medicines but products such as medical devices (and perhaps in time interventions such as surgical operations) could be better conducted. They also raise important questions as to the extent to which in contexts like promoting enhanced outcomes in areas of cancer care and the treatment of rare monogenic diseases the criteria used for evaluating the ‘worth’ of therapeutic innovations might be standardised across the European Union, and how regionally or globally consistent approaches to adjusting for factors like GDP and disease prevalence variations could be instituted.
Technically, Advance-HTA’s results raise questions about how the affordability thresholds used in assessments can best be set, and the degree to which the extended use of MCDA techniques would materially enhance the validity and acceptability of currently established evaluation methods. They also highlight as yet unresolved uncertainties about the extent to which national policy decisions or individual treatment choices could ever be determined on the basis of HTA findings alone.
Potential Impact:
Impact
In this section, the impact of Advance-HTA is discussed, by focusing on different dimensions of impact as it relates to the project’s objectives, notably (a) impact on methods, (b) impact on EC policies, (c) impact on EC competitiveness and (d) impact on the partners and the audiences the project has sought to influence. These are outlined below.
Impact on methods
Advance-HTA’s fundamental remit was to contribute to the advancement of methods and the development of conceptual frameworks in a number of areas where it was felt there was urgent need to do so. This has taken place in 4 very specific areas, whilst in 2 more, new evidence base has been created pointing to the need to re-think current practices and methods.
The three areas of methodological improvement are as follows: First, the Advance Value Framework using decision analysis (MCDA) principles to address value assessment, has already received significant attention both in Europe and internationally, beyond its publication. In addition to what is already included in the project materials, a number of case studies are in the pipeline with national HTA bodies seeking to understand how it works and under what circumstances they can use the framework in their respective practices. Case studies are currently under way with national insurers in Belgium, Poland, Lithuania and are being planned with France and Scotland. Second, the Retrospective Analysis Framework has been presented in numerous conferences, workshops and has also been published. The feedback received suggests that it provides an important analytical tool to determine how assessments take place in different settings and what trade-offs exist between scientific and social value judgments. As the evidence base that is included in the framework increases, so will its applicability and usefulness across settings. Third, the analytical framework that has been produced for medical devices links legislation at EU level with policy and practice, and, importantly, highlights the need for continuous work on the subject of medical devices given their differences to other technologies, the type of evidence available when coverage is sought, and the diversity of devices themselves, making a single approach to assess ‘value’ very difficult. And, finally, a toolbox has been compiled containing information, advice and recommendations for countries wishing to expand, strengthen and/or implement HTA in their jurisdictions. The applicability of the material in the toolbox is independent of country or setting and its use can be adapted to any local circumstances.
In terms of new evidence pointing at the need to re-think current practices and methods in HTA, this has been achieved in the following areas: First, in the area of patient preference elicitation, where it has been shown that generic tools may have limited capacity at taking into considerations dimensions that are relevant to patients (as opposed to non-patients, which is the case today), while, additionally, patient vs. non-patient preferences are different across a significant number of health states, resulting in different priorities in resource allocation. And, second, Advance-HTA has shown that through the use of discrete choice experiments, it is possible to derive (and ultimately take into consideration) utilities from decision-makers and patients that will be very useful in the context of decision-making.
Impact on EC policies
The results of Advance-HTA feed into an evolving landscape relating to the assessment of new medical technologies and the interface between regulation of new technologies and their incorporation in benefits catalogues in different settings. A number of changes in the regulatory framework and the requirements for marketing authorisation (adaptive licensing, conditional approval, authorisation under exceptional circumstances) imply have a concomitant knock-on effect on the way HTA is conducted, its robustness to inform decision-making and the extent to which additional sources of information are taken into account when making coverage recommendations. Whereas Advance-HTA has not been able to address all the issues that have arisen in recent years in technology assessment stemming from the increased complexity of medical technologies and their production, it clearly has created new ground in a number of fields which impact EU policy in the health sector: these fields relate to (a) a new methodological framework for value assessment and the breadth of criteria that (may need to) inform value and decision-making, (b) the importance of other considerations - as they relate to the disease or its treatment – in shaping the direction of a recommendation across therapeutic areas, including orphan treatments, (c) a methodological framework for the assessment of value in certain types of medical devices, (d) the issue of preference elicitation and the evidence that generic tools as they are currently used in decision-making processes are unlikely to capture patient experiences in their totality which may require us to look further into this to optimise resource allocation processes and (e) a conceptual framework that may be of assistance to countries or settings that do not have an HTA process or function but wish to establish one. All of the above are at the heart of health care decision-making, both at European, but also at national level. Importantly, they will all feed into the forthcoming process that will commence through Joint Action 3 in the context of EUnetHTA and will feed into its agenda. Through continuous efforts, the objective will be to provide fresh evidence on research and allied findings from work in the project’s workstreams.
Impact on EC competitiveness
Advance-HTA has dedicated a significant amount of time and resource to building links, networks as well as disseminating its results to geographical areas outside the EU and has laid the foundation for increased cooperation and knowledge transfer in the years to come. In itself, this is an indicator of Europe’s intellectual leadership in this field and contributes to its external competitiveness. Specifically, Eastern Europe – including the Russia and the former CIS – and Latin America and the Caribbean have been two geographical areas associated with the project, where networks of decision-makers have been built and extensive capacity-building and dissemination have taken place in 2014 and 2015. In addition, further dissemination activities have taken place in Southeast Asia and in China, which relate to some of the methodological frameworks developed by the project and constitute testaments to its rising reputation and legacy. The latter have taken place on an ad hoc basis and because of the project’s rising reputation in a number of fields (specifically value assessment, medical devices and preference elicitation). These activities will unavoidably lead to further contact in the years to come and form part of the overall project legacy. They also serve as indicators of the wider international leadership undertaken by Europe-based organisations.
Impact on the partners (products or services)
Several issues have emerged in this context of the likely impact on partners and the knowledge that has emerged. These are addressed below.
• First, with regards to the project outputs and what has come out of the project: the project has contributed scientific evidence, including a number of toolkits and methodological/conceptual frameworks, which are available to decision-makers and can be used in different settings and can serve the objectives of capacity-building as well as introducing more robust methods of assessment depending on context and type of technology. Importantly, all partners have contributed significantly to this process;
• Second, in what concerns the type of audience for the project outputs: the key recipients of the project outputs are health care decision-makers in EU countries, in Central and Eastern Europe, Latin America and countries in Southeast Asia and China that have already implemented or are planning to strengthen and/or implement and/or reform current practices in the HTA system they may have in place. Additional audiences include academia and the broader stakeholder community, specifically, clinicians, patients and industry.
• Third, with regards to the reason why the above audiences need the results of the project: it has become clear that new methods, tools and guidance are needed for decision-makers as well as other stakeholders to address gaps in particular themes related to HTA, for example, what criteria are used in value assessment and how do we treat social value judgments; how do we assess value in orphan drugs; how do we treat HTA in medical devices from a methodological and evidence perspective; and should patient-reported outcomes be used more intensely compared with generic preference elicitation tools and, if so, how.
• Fourth, in terms of how the interested parties learn about the results: Advance-HTA has put together a multi-faceted dissemination strategy comprising (a) academic publications; (b) policy briefs; (c) direct dissemination through face-to-face workshops; (d) online dissemination through the project website; (e) social media accounts to enable access to materials (LinkedIn and Twitter); and (f) inclusion of published or validated new knowledge into training curricula. This dissemination strategy will continue after the completion of the project. There is also willingness by the project leaders and some WP leaders to explore synergies with decision-makers in order to diffuse further the research results in the near future.
• Fifth, to the question of how appropriate the measures are for the dissemination of results: Advance-HTA has pursued a combination of measures aiming to maximise the widest possible diffusion of the research results. Clearly, the long-run credibility of the research results will emerge from their publication in high-impact peer review journals. This has been one of the top priorities of the consortium. The contribution of the HTA partners (NICE Int, TLV and AOTM), PAHO and EBC in this process has been invaluable because it has enabled the policy/stakeholder dimension to be included. A significant component has been the use of networks for face-to-face dissemination; this provides immediate benefits but also has longer-term implications for greater leveraging of such networks. Finally beyond peer review and face-to-face activities, electronic means of dissemination are available (website, social media), potentially making it feasible for larger audiences to benefit from the availability of results.
• Sixth, with regards to the management of intellectual property: there is considerable flexibility as the project has been structured around work packages where individual work package leaders and those who contribute meaningfully share the intellectual property. In a number of cases, it is expected that the rights holders will pursue applications (e.g. software or online tools) to their property.
• Finally, and based on the above, we feel that Advance-HTA has followed very closely the Commission’s work programme in terms of expected impact by linking academic rigour with policy-maker needs and sensitivities, whilst at the same time involving the wider stakeholder community both as a partner and as recipient of the knowledge created.
Dissemination
1. Overview
The primary objective of the dissemination activities is to ensure that results of the project’s research become known to and used by the HTA stakeholder community. Dissemination is meant to be continuous throughout the life of the project and is also expected to continue after the end of the project through publications and symposia (e.g. the LSE SUMMIT, taking place in March 2016), among others. The dissemination plan is meant to be a dynamic rather than static draft of planned and executed activities. The objective was split into 3 sub-components:
1. To effectively link policy makers, stakeholders and patient networks to the research evidence on HTA;
2. To define a dissemination strategy, taking into consideration the advice of the Scientific Advisory Board (SAB) as well as the recommendations of the three institutional partners (NICE International, TLV and AOTM);
3. To target groups for different results and the most adequate dissemination mode: publications, face-to-face and electronic dissemination.
ADVANCE-HTA used specific measures to communicate results in ways that would help disseminate findings and build capacity on HTA methods; these are grouped under 3 headings detailed in Section A (public) Tables: Publications and List of Dissemination Activities, which includes the relevant target groups:
a) Publications
b) Face-to-face dissemination (e.g. conferences, workshops)
c) Project website and electronic dissemination (Linkedin, Twitter, Blogs)
2. Publications
The project results have been and will continue to be disseminated through publications in print in a variety of ways, each targeting different audiences. The list of activities specific to each Work Package and partner responsible are listed in Appendix 2 – Publication plan, which includes planned publication means, timelines, authors and a short description of content.
First, publication sources have been used and will be used such as Health Policy, Health Economic Reviews, Health Economics Policy and Law, Globalisation and Health, Social Science and Medicine, Global Policy, Pharmacoeconomics, International Journal of Technology Assessment in Health Care, among others, where members of the consortium are involved in the editorial boards. Policy briefs and reports have been distributed through LSE Health working papers (on-line distribution) and the same can be with other members of the consortium. PAHO has assisted with the dissemination activities in Latin America. These outputs are intended for the broader stakeholder community. Tailor-made outputs (e.g. HTA for patient organisations) will also be created.
Second, it is the intention of the partners to pursue a special issue in a leading peer review journal in addition to any stand-alone peer review publications that emerge from the project (e.g. Health Policy). These activities target a more technical audience and a selection of articles is already under way for this purpose.
Third, one book project bringing together the conceptual/analytical framework on HTA strengthening and implementation will be published through the European Observatory. According to current plans, the book will be divided in two sections. First, details of the current state of plan will be presented to provide a mapping or framework of current HTA methodologies, emphasising possible ways forward and anticipated challenges, together with implications for policy-makers who would like to reform HTA. Second, findings will be presented detailing the broader HTA framework, with the core lessons and options for innovating HTA for its continued improvement in different settings. This output is intended for policy-makers and all HTA stakeholders.
Fourth, the needs of researchers will be met by publication of papers in academic journals in the health economics and comparative health policy fields. This is intended for a more technical audience.
Finally, other opportunities will be taken as and when they arise, such as articles in newspapers or other media, which will target the broader stakeholder community. In addition, the project will continue to leverage to the maximum extent possible the project website as well as social media (linkedin, twitter) to make these activities more widely known to relevant audiences. Further opportunities may arise through collaborative arrangements with other EU FP7 HTA-related projects or/and EUNetHTA. These will be explored during the course of the project life.
3. Face-to-face dissemination – Workshops
(a) Background
One of the main activities undertaken to achieve these goals were a number of capacity-building workshops targeting key decision-makers in Eastern European and Latin American countries. In both regions, interest in the application of HTA and in creating an HTA function in health care systems has increased significantly over the past decade. Consequently, the rationale for conducting these workshops in these two regions was, first, to provide updated information & training on the introduction and implementation of HTA in key settings and, second, to disseminate the research and policy-relevant findings of Advance-HTA into countries that have recently implemented HTA or were planning to do so. The objectives of these workshops were:
➢ Undertake HTA capacity-building in Latin America and Eastern Europe
➢ Present the HTA methodological advances developed by the Advance HTA Consortium
➢ Discuss the relevance of such methodological advances with local participants from Latin America and Eastern Europe
➢ Discuss future developments in HTA and their policy implications
➢ Foster international collaboration and create a network for exchange of expertise
(b) Concept
Four workshops were undertaken in M21 (September 2014), M23 (November 2014) and M33 (September 2015) to build capacity in HTA as well as disseminate the results of the project in Latin America and Eastern Europe. These capacity building workshops were in the form of “train-the-trainers” workshops, in order to give the participants the knowledge and tools to continue to build capacity in HTA in their own countries. Two workshops were conducted in each region. The first wave of workshops aimed to disseminate findings about the “state of the art”: where are we today, what are the methodological challenges and current and best practice examples for HTA. The second wave of workshops focused on disseminating findings from the empirical research conducted within Advance-HTA.
(c) First wave of workshops
The aim of the first wave of workshops, which were held in Warsaw (September 2014) and Mexico City (November 2014), was to give an overview of the results from what we know about the implementation of HTA to date and the rationale for conducting the research undertaken within the project. Both workshops had the same structure and were organised around 6 thematic areas, but content was adapted to reflect geographical differences and requests made by participants ex ante. The Warsaw and Mexico City workshops addressed the following thematic areas as summarised below (See table in final report).
First Workshop Eastern Europe – Warsaw, 2014
Session 1 - Introduction to health technology assessment: An introduction to the models, costs and benefits in HTA were first outlined, followed by existing processes and evidence requirements in different settings. This session finished with a presentation about the place of patients within these processes.
Session 2 - HTA experiences in South and Eastern European and lessons learned: This session started by showcasing results on existing HTA processes from a survey in South and Eastern Europe. The process of adoption and current application of HTA in Poland were then presented, followed by experiences from Hungary and Croatia.
Session 3 - Critical issues in appraisal of medical technology: Current ways to assess value for money were discussed, followed by a presentation on the application of multiple criteria decision analysis and its relevance to value assessment, based on Advance-HTA research results. The second part of this session discussed existing methods of preference elicitation, and willingness to pay thresholds.
Session 4 - HTA in context: Medical devices and orphan drugs: This next session discussed the particularities in assessing medical devices and orphan drugs and the possible ways forward. Based on Advance-HTA results, a new taxonomy for medical devices and a framework for analysing value in orphan drugs were also presented.
Session 5 - Empirical evidence on the use of HTA: Empirical evidence on the use of HTA were presented by NICE (England), as well as evidence on the application of HTA in different settings. An overview of results from the BURQOL-RD study was also presented, about the socio-economic burden and health-related quality of life in patients with rare diseases in Europe.
Session 6 - The way forward: This last session portrayed innovative models or tools to overcome some of the known limitations in the application of HTA. This included an overview of the EUnetHTA Core model, and an introduction to risk sharing agreements together with an empirical example of its use in Sweden.
First workshop Latin America, Mexico City, 2014
Session 1 - Introduction to health technology assessment: as per Warsaw 2014
Session 2 - HTA experiences in the Americas and Lessons learned: This session also portrayed results from the survey on HTA capacity in Latin American countries. Experiences from Colombia, Brazil, Mexico, and Costa Rica were presented.
Session 3 - Critical issues in appraisal of medical technology: as per Warsaw 2014
Session 4 - What could Latin American countries learn from other jurisdictions: During this session, case studies from European countries (England and Spain) were presented. Results from the survey on HTA capacity in Eastern European countries were also presented.
Session 5 - HTA in context: A number of applications of HTA were presented, namely the rapid assessments and how they support decision-making, the transferability of HTA results to other settings, as well as empirical evidence of the application of HTA in different settings. The session ended with a presentation on the role of risk sharing agreements and other mechanisms and how these approaches may help overcoming some of the limitations in the application of HTA.
Session 6 - Current patterns and future directions in HTA implementation in Latin American countries: The final session was in the form of a panel, where the current patterns and future directions in HTA implementation in Latin American countries were discussed amongst key faculty members and the audience.
Advance-HTA partners who formed the faculty in Warsaw included: LSHTM (Cairns); LSE (Kanavos, Nicod, Angelis), EBC (Mossmann), EASP (Espin), AOTM (Zawada), IER (Prevolnik-Rupel), TLV (Larrson), TUB (Busse), ISS (Taruscio), NICE Int (Ruiz), UCLM (Lopez Bastida), EunetHTA (Borlum Kristensen). A number of additional speakers were invited to share their experiences with some of the topics discussed, including representatives from the Hungarian HTA agency (GYSEMSZI) and the Croatian HTA agency (AAZ). In the Mexico City workshop, the same Advance HTA institutional partners participated, supplemented by: PAHO (Lemgruber), NICE Int (Freiburg). Policy reactions were made by representatives from the following institutions: Colombian HTA agency (IETS), Brazilian HTA agency (CONITEC), Costa Rica Ministry of health, National Institute for Cardiology in Brazil.
(d) Second wave of workshops
The second workshops were structured in a similar manner, but contained two key differences: first, they were far more interactive and included a significant amount of breakout sessions with group-work by participants followed by feedback in plenary; second, each session also included policy reactions from participants on new key findings and how these may be relevant to their own context. The six sessions that were included in the agenda in Santiago (9-10 September 2015) and Warsaw, (24-25 September 2015) respectively, are summarised below (See table in final report).. Both workshops were very similar with each other and differed only in the adaptation of the issues to local audiences.
Second Workshop Eastern Europe – Warsaw, 2015
Session 1 - Assessing Value for Money: The first part of the session touched upon the different ways of assessing value for money, while the second part of the session outlined results from the research conducted that aimed to describe the differences in drug reimbursement decisions on cancer drugs in ten European countries. The interactive case study was about discussing the most recent tools developed to assessing the value of cancer drugs, namely the Cancer Drug Fund prioritization tool in England, the European Society for Medical Oncology tool for assessing clinical benefit, and the American Society for Clinical Oncology framework for comparing the relative clinical benefit, toxicity and costs.
Session 2 - Using alternative methods to assess the value of new medical technologies: The first part of this session provided insights into the use of Multiple Criteria Decision Analysis (MCDA) in the context of HTA, while the second part presented its empirical application to a particular oncology indication. The group break out session related to a number of therapeutic options within a particular indication. Each group was given a list of criteria structured in the form of a value tree, spanning all the critical dimensions of value and were invited to choose among them.
Session 3 - HTA and rare diseases: assessing the societal value of orphan drugs: The first part of the session outlined the particularities of rare diseases and orphan drugs across the clinical development pipeline and implications for HTA. The second part of the session presented results from the research conducted within the project that aimed to identify the similarities and differences in how countries are appraising orphan drugs and how they are dealing with the common issues related to their rarity. During the breakout sessions, groups was asked to discuss the issues they are facing and how they are dealt with in their setting in order to ensure financial sustainability.
Session 4 - HTA and quality of life measurement: The first part of the session provided insights about the theoretical foundations and empirical investigation about which preferences should be accounted for (general population or patient preferences) in health care. Results from their empirical investigation were presented, in terms of the methods used to arrive to both value sets, their comparison and in case of differences, on the meaning of the differences in health policy. During the break out session, the question about which value sets should be used were discussed amongst participants.
Session 5 - Strengthening and implementing HTA in emerging settings: This session first highlighted the importance of HTA in emerging settings, while the second part described results from the mapping exercise conducted and the tool kit developed aimed at helping countries implement HTA. During the breakout sessions, the main challenges in implementing HTA were discussed. Participants were then asked to discuss having an official threshold in each country.
Session 6 - HTA and medical devices: This session, first, discussed the particularities of assessing value of medical devices and how it differs from pharmaceuticals or other technologies, presenting current practices across European HTA agencies. Second, it outlined results from the research conducted about a taxonomy developed for medical devices and approaching the evaluation based on this model. Different topics were discussed during the breakout sessions, namely prioritization, existing HTA tools and guidelines, and identification of existing challenges and particularities in assessing medical devices in different contexts.
Second workshop Latin America, Santiago, 2015
Session 1 - Assessing Value for Money: as per Warsaw, 2015
Session 2 - Value Assessments in Health Technology: The first part of the session provided insights into the current systems of value assessment and their features, including clinical benefit assessment, economic evaluation and value based assessment, and their combination. The second part of the session outlined the findings from the research conducted within the project on building a value proposition from an HTA perspective. The group breakout sessions evaluated evidence relating to the performance of a new drug versus its comparator(s) across a range of dimensions, indicated for the treatment of an advanced cancer. Participants were asked to write up their own value proposition for these treatments.
Session 3 - HTA and rare diseases: assessing the societal value of orphan drugs: as per Warsaw, 2015
Session 4 - Using alternative methods to assess the value of new medical technologies: The first part of this session provided insights into the use of Multiple Criteria Decision Analysis (MCDA) in the context of HTA, while the second part presented its empirical application to a particular oncology indication. The group break out session related to a number of therapeutic options within a particular indication. Each group was given a list of criteria structured in the form of a value tree, spanning all the critical dimensions of value and were invited to choose among them.
Session 5 - Strengthening and implementing HTA in emerging settings: This session first highlighted the importance of HTA in emerging settings, while the second part described results from the mapping exercise conducted and the tool kit developed aimed at helping countries implement HTA. During the breakout sessions, the main challenges in implementing HTA were discussed. Participants were then asked to discuss having an official threshold in each country.
Session 6 - HTA and medical devices: This session, first, discussed the particularities of assessing value of medical devices and how it differs from pharmaceuticals or other technologies, presenting current practices across European HTA agencies. Second, it outlined results from the research conducted about a taxonomy developed for medical devices and approaching the evaluation based on this model. Different topics were discussed during the breakout sessions, namely prioritization, existing HTA tools and guidelines, and identification of existing challenges and particularities in assessing medical devices in different contexts.
Advance-HTA partners who attended included: LSHTM (Cairns); LSE (Kanavos, Nicod, Angelis), EBC (Mossmann), EASP (Espin), AOTM (Zawada), IER (Orgorevc), TUB (Pantelli, Fuchs), NICE (Cutler). A number of additional speakers were invited to share their experiences with some of the topics discussed, including representatives from: Faculty of Public Health (Bulgaria), Ministry of Health (Ukraine), National Health Fund (Greece), Ministry of Health (Latvia), and the Transparency Council (Poland). The same Advance-HTA partners attended the second workshop in Santiago de Chile, with the addition of PAHO (Lemgruber). Policy reactions were made by representatives from the following institutions: Ministry of Health (Chile), University of York, ISP (Chile), Ministry of Health (Ecuador), IETS (Colombia), IETSI (Peru), Ministry of Health (Uruguay), UFMG (Brazil), INESSS (Canada), INVIMA (Colombia), CONITEC (Brazil), Ministry of Health (Costa Rica).
(e) Workshop participation
The 1st Advance-HTA Capacity Building Workshop (Warsaw, 25-26 September 2014) was attended by 64 participants. Attendees were mainly policy makers working in public institutions such as Ministries of Health, Health Insurance Funds, National Health Services and HTA Agencies. In total, 13 countries were represented: Bulgaria, Croatia, Cyprus, Estonia, Greece, Hungary, Latvia, Poland, Serbia, Slovakia, Russia, Turkey and Ukraine. The 2nd Advance-HTA Capacity Building Workshop (Warsaw, 24-25 September 2015) was attended by 57 participants. Importantly a significant overlap existed between the attendees of the 1st and the 2nd workshop ensuring a degree of continuity in the dissemination process. In total, 14 countries were represented: Bulgaria, Czech Republic, Denmark, Estonia, Greece, Hungary, Latvia, Lithuania, Moldova, Poland, Russia, Slovenia, the Ukraine and the UK. Our World Health Organisation contacts in Europe kindly provided travel funding for a small number of participants from Eastern Europe to attend.
From a Latin American perspective, the 1st Advance-HTA Capacity Building Workshop (Mexico City, 6-7 November 2014), was organized by Advance-HTA with additional support from CENETEC-SALUD, the National Center for Excellence in Health Technology of Mexico and PAHO and was attended by 59 participants, mainly policy makers working in public institutions such as Ministries of Health, Health Insurance Funds, Social Security Institutes and HTA Agencies. In total, 14 countries were represented: Argentina, Bolivia, Brazil, Chile, Colombia, Costa Rica, Cuba, Ecuador, El Salvador, Mexico, Paraguay, Peru, Trinidad and Tobago, and Uruguay. The 2nd Advance-HTA Capacity Building Workshop (Santiago, 9-10 September 2015), was organised by Advance-HTA and was supported by PAHO, the MoH-Chile and the Instituto de Salud Pública de Chile. It was attended by 64 participants from Argentina, Bolivia, Brazil, Canada, Chile, Colombia, Costa Rica, Cuba, Ecuador, Mexico, Paraguay, Peru and Uruguay. We were able to broaden our participation invitations by building our workshops around regional meetings in Latin America, such as the regional HTAi – Latin America (2014 meeting) or the ISPOR-Latin America (2015 meeting), bringing together HTA agencies and members of the REDETSA network (managed by PAHO) from across the Americas providing a forum to discuss a wide range of issues, challenges and opportunities with the various HTA agency models.
4. Project website and electronic dissemination
All four workshops held were based on a mix of presentations, case studies, groupwork and workshops, whereby participants could interact with each other while discussing a case study. Materials and presentations are publicly available to download from the project website, weblink provided below.
https://arquivo.pt/wayback/20170603122218/http://www.advance-hta.eu/dissemination.php
A Twitter group (@AdvanceHTA) has been created, where all novelties, including new publications, meetings, workshops, opening of registration to workshop, or other activities, will be tweeted with a link to the Advance-HTA website. https://twitter.com/ADVANCEHTA . We will also aim to have our partners or relations “re-tweet” our tweets in order to have a greater impact.
An Advance-HTA Linkedin group has been established, to enable the project a platform linking Advance-HTA participants to the broader HTA stakeholder community. The platform has been used to disseminate information about the project, and as a discussion forum. The Linkedin group is linked to the Advance-HTA website, as well as the Advance-HTA twitter account. http://www.linkedin.com/groups/AdvanceHTA-5051411
Exploitation of results
The partners and the research require involvement by key decision makers and stakeholders. Face to face dissemination is encouraged in order to receive feedback from a broad range of stakeholders. There have been several abstract submissions for poster and oral presentations at conferences, panel submissions for conferences, a final conference. All these media have enhanced the audience, decision makers and forum by integrating in a sustainable way and use them actively in systems and practices at local, regional, national and European levels.
1. Toolkits and evidence
A number of toolkits have emerged from the project, as follows: (a) The Taxonomy, enabling an understanding of country- and system-specific factors that may influence coverage recommendations; (b) the Retrospective Analysis Framework, scrutinizing coverage recommendations based on the evidence submitted, its interpretation and a significant number of other considerations, which may be disease- or therapy-specific and which may have predictive power once populated with a larger number of observations; (c) The Advance Value Framework, which is a value assessment framework based on MCDA principles and which has undergone testing across two indications already; (d) The patient database, which has enabled face-to-face interviews and surveys to be conducted across the patient community in order to establish patient-related QoL preferences and utility values; (e) a methodological taxonomy enabling an analysis of HTA across medical devices; and (f) a toolbox that can be used in emerging HTA settings in order to strengthen and/or implement an HTA function based on a number of parameters and a specific conceptual framework. It is envisaged that these toolkits will be disseminated further; in some cases, further research is envisaged – for example, through additional applications for funding in areas that still merit basic research or its application; in other cases, the knowledge generated will be leveraged through the development of online or software applications; finally, in some cases a combination of further research with software development is envisaged.
2. National and International Scientific Conferences
Results have been communicated at conferences and events that individuals attend, including some key international gatherings such as, Health Technology Assessment international (HTAi); International Health Economics Association (iHEA); European Health Economics Association (ECHE); AcademyHealth in the USA and different chapters of the International Society for Pharmacoeconomics Outcomes Research (ISPOR). To date, project results have generated a steady stream of abstract submissions to the above conferences and will continue to do so throughout 2016 and, possibly, 2017.
3. Advocacy, seminars, workshops, capacity-building
Advocacy has played a vital role in disseminating evidence to multiple stakeholders. The strategy has made effective use of both upstream and downstream advocacy to reach out to different players in the arena of access to medicine. Consultations, workshops, apart from seminars, and conferences have been an effective tool to disseminate findings. Seminars and workshops have been organised by the various members who hold in-house capabilities and competencies for doing so (such as AOTM for the Warsaw Workshops and PAHO for the Latin American workshops). Two conferences deserve particular merit in this context, which were also part of the project’s dissemination strategy.
(a) Haute Autorité de Santé (HAS) – Paris, 30 October 2015
For its 10th anniversary, the Haute Autorité de Santé (HAS) in collaboration with the European Commission organised a one-day conference dedicated to HTA and the benefits of European cooperation in this field. Key questions around European cooperation on HTA, stakeholder involvement and early dialogue, current challenges in applying HTA, and HTA in special contexts (e.g. complex interventions, hospital setting) were discussed. Together with EUnetHTA, all four FP7 HTA projects funded by the European Commission were invited to present their findings and outline their policy implications for (HTA) stakeholders. Some of the research streams from Advance-HTA were presented within Session 2 - Towards innovative methodologies & practices. The session focused on ways forward to overcome some of the methodological challenges in current practices of HTA for medicines. Current practices of value assessment have been subject to criticism about whether these measures capture all the important dimensions of value, including living with the disease and receiving treatment in real world settings, while adequately recognising and valuing innovation. This one-day conference took place on October 30th in Paris, at the Ministry of Social Affairs, Health and Women's Rights. A total of 332 participants attended the conference. Each session included a presentation of the empirical evidence and the recommendations, followed by comments from two invited discussants on the results, representing different HTA stakeholders. At the end of each session, about 30 minutes were allocated for a general discussion with all participants. LSE was in charge of the coordination of and logistics for Session 2. Materials and recordings are publicly available – click here
(b) Final Advance HTA Dissemination Conference – London, 19 November 2015
The final dissemination conference of the Advance-HTA project successfully took place on Thursday 19th November 2015. Over 120 participants attended the conference where the results of each work package were presented by the partners of the project and commented by key stakeholders that acted as discussants. The opening keynote address was delivered by the Director General of the Dental and Pharmaceutical Benefits Board (TLV), the Swedish HTA agency. Topics presented included “Assessing value for money”, “New tools and approaches in the value assessment of medical technologies”, “Preference elicitation and quality of life” and “Strengthening and implementing HTA in emerging settings”, followed by stakeholder reflections on future HTA direction. Besides the partners of the project consortium, discussants involved a range of experts including the Head of the European Observatory on Health Systems and Policies, the Programme Manager for Health Technologies and Pharmaceuticals from the World Health Organisation - EURO, the Chairman of the Executive Committee of the European Network for Health Technology Assessment and the President of the European Brain Council, among other leading international academics and researchers. Research findings highlighted differences in the recommendations of HTA agencies across European countries while explaining the underlying causes of decision heterogeneity; presented the development and application of a Multiple Criteria Decision Analysis methodological approach for assessing the value of new drugs; applied the use of a new taxonomical framework for grouping and evaluating new medical devices; highlighted the differences in perspective and preferences between patients and other stakeholders; illustrated the use of a discrete choice experiment approach to elicit the preferences of decision makers and patients for assessing the value of orphan drugs; and showcased the importance of HTA in emerging countries including a mapping exercise and the development of a methodological toolbox. Please see https://arquivo.pt/wayback/20170603123252/http://www.advance-hta.eu/conferences.php for details on the agenda, overview, attendees and presentations.
4. Publications
A significant number of publications have emerged during the lifetime of the project in peer review journals and, more widely in the literature. A list of these publications and those that are currently under review or in the preparation stage are provided in the appropriate section of this report.
Communication beyond the project
Dissemination of the findings generated during the project will target entities and individuals beyond the partners of ADVANCE-HTA. HTA networks in Europe, the networks established from the 4 workshops in Eastern Europe and Latin America will provide the first port of call for new activities and materials emerging from the project. The website is accessible to the public and will be used post-project to communicate the new knowledge generated from the project; the European Observatory platform will also be used to disseminate ADVANCE-HTA findings such as books and reports. Findings are already being disseminated through peer-reviewed publications and will continue to beyond the project end-date. Social Science and Medicine, Medical Decision Making, International journal of Technology Assessment in Health, Health Policy, Pharmacoeconomics and the International Journal for Technology Assessment in Health Care are few of the key journals and publications that are featuring or will feature key pieces from the project. The consortium is keen to showcase the specific areas explored by the project as well as the key messages, findings and recommendations as a whole to share with policy makers, stakeholders and HTA groups on a national and international level. The consortium has an agreement regarding authorship and intellectual property that enables its members to publish their results in the literature in a collaborative and inclusive way. Any publication or communication regarding the project will specify that it reflects the authors’ views and that the EC is not liable for any use of the information communicated.
List of Websites:
https://arquivo.pt/wayback/20170603121424/http://www.advance-hta.eu/
