Final Report Summary - INSPIRE (Innovative methodology for small populations research)
Executive Summary:
This project has brought together eight European partner organisations to address the statistical challenges of the design and analysis of clinical trials in small populations. Participants are international experts in innovative trial design methods from academia, industry and regulatory authorities. We have focussed on methods to optimise clinical trial design, improve analysis to use data from relevant sources both within and external to clinical trials and to consider levels of evidence appropriate for decision-making. Our work in four specific areas is described below.
Research in early phase dose-finding trials in small populations:
We have focussed on innovative designs for early phase clinical trials and for extrapolation from adult to paediatric studies, developing methods to enable use of pharmacokinetics and pharmacodynamics data to help determine which doses to use both in the study as it progresses and for further evaluation. Our methods enable better estimation of the dose-response curve parameters without increasing the number of patients in the study who receive sub-optimal doses. This work has led to a novel design for an ongoing clinical trial in prevention of seizures in neonates.
Research in decision-theoretic designs for clinical trials in small populations:
We have considered determination of appropriate decision-making methods for small population clinical trials. We have conducted a review of clinical trial designs currently in use in clinical trials in rare diseases as well as exploring novel methods. In particular we have explored the use of a Bayesian decision-theoretic framework to compare the costs of clinical trial evaluation with the potential benefits to current and future patients, assessing how the cost-benefit balance differs between large and small patient populations.
Research in confirmatory trials for small populations and personalised medicines:
We have developed frequentist analysis methods and decision theoretic optimal design approaches for trials that assess targeted treatments based on genetic features or other biomarkers are being developed. These trials enable the simultaneous identification of subgroups of patients for which the benefit risk balance of a treatment is positive and confirmation of treatment effect for these patients without compromising statistical or scientific integrity.
Research in use of evidence synthesis in the planning and interpretation of clinical trials in small populations:
We have developed improved robust meta-analysis methods for synthesis evidence from a small number of trials as would be common in a small population. These methods can be used to support the planning, analysis and interpretation of a trial as well as enabling extrapolation between patient groups. The novel approaches have been applied in a review of studies in sporadic Creutzfeld-Jakob disease.
Dissemination and stakeholder engagement:
Project results have been disseminated through open-access publications in high quality peer reviewed journals, with 21 papers currently published and a further others submitted, along with open-access software applications to facilitate the use of methods we have developed. Results have been presented at national and international conferences and we have organised three conferences to present our results and interact with key stakeholders, two of which have been joint meetings with the other projects, Asterix and IDeAl, funded under the same EU FP7 call. One of the latter meetings was hosted by the European Medicines Agency, facilitating engagement with regulatory authorities along with other key stakeholders including industry and patient representative groups.
Project Context and Objectives:
The HEALTH.2013.4.2-3 call identified a need for new or improved statistical methodology for clinical trials for the efficient assessment of safety and/or efficacy of treatment for small population groups.
Clinical trials in small populations present a number of novel statistical challenges associated with the need to draw inference from the necessarily small sample sizes. The EMA CHMP Guideline on Clinical Trials in Small Populations (CHMP, 2007) and the report of the US Institute of Medicine Committee on Strategies for Small-Number-Participant Clinical Research Trials (Evans and Ildstad, 2001) propose a number of potential statistical approaches. These include efficient trial designs, such as sequential and adaptive methods, and innovative methods that enable data from outside the trial to be used both at the design stage and assimilated with trial data in the final inferential process. Based on the recommendations of these guidelines and building on recent relevant methodological advances in this area, in this project we will develop novel statistical methodology for clinical trials in small populations.
We identified four specific areas where we believed novel methodology for clinical trial design was needed and achievable. These are (i) early phase dose-finding studies in small populations, (ii) decision-theoretic methods for clinical trials in small populations, (iii) confirmatory trials in small populations and personalised medicines, (iv) use of evidence synthesis in the planning and interpretation of clinical trials in small populations and rare diseases. These four areas formed the four main Work Packages, WP1, WP2, WP3 and WP4, of the project. These were augmented by two further work packages on dissemination (WP5) and project management (WP6). The objectives for each work package are given in the following paragraphs.
Work package WP1: Research in early phase dose-finding trials in small populations
Early phase dose-finding studies are the first trials of a new medicinal product in humans and aim to obtain reliable information on an appropriate dose for use in further clinical trials. Sample sizes for such studies are typically small, particularly in a small population context. Recent work has led to efficient designs for dose-finding based on the minimum number of participants. These methods have generally relied primarily on observed safety data, however, and there is a growing awareness of the need to better incorporate pharmacokinetic/pharmacodynamic (PK/PD) information in the dose-finding process. In this project we will develop novel design methods to use PK/PD data more fully. The use of such data will give a fuller picture of drug safety and efficacy during the dose-finding trial, enabling improved estimation of the best dose level for further evaluation whilst ensuring the sample size for early phase trials is kept to a minimum.
Overall objective
O.1 Propose innovative designs for early phase clinical trials taking into account safety, efficacy and pharmacokinetic/pharmacodynamic (PK/PD) measures in order to better estimate the dose level to be recommended based on limited sample sizes and subgroups.
Specific objectives
O.1.1 Develop efficient model-based designs for dose-finding studies using PK/PD information with different continuous and binary outcomes.
O.1.2 Evaluate the performance of the methods in terms of information gain, number of subjects, efficiency, and robustness.
Work package WP2: Research in decision-theoretic designs for clinical trials in small populations
Decision-theoretic methods explicitly enable evaluation of the level of evidence required from a clinical trial to best inform clinical practice. This in turn can lead to efficient and appropriate clinical trial design. In a small population setting, the small available sample size, the fact that recruitment to one clinical trial may limit recruitment to concurrent trials and the small size of the target population may mean that standard clinical trial designs, proposed in the setting of large patient populations, may be inappropriate on impossible to achieve. In this project we will develop new decision-theoretic and value-of-information methods for the design and sequential monitoring of clinical trials in small populations. Application of these methods will lead to smaller studies that are more able to lead to appropriate decision-making.
Overall objective
O.2 Develop methods for small population clinical trials based on a decision-theoretic framework.
Specific objectives
O.2.1 Obtain optimised clinical trial designs based on utility functions that account for the benefits of treatment and the population size.
O.2.2 Incorporate health economic aspects in design of trials in small populations using a value-of-information approach.
O.2.3 Determine appropriate levels of evidence for decision-making in small population clinical trials.
Work package WP3: Research in confirmatory trials for small populations and personalised medicines
Confirmatory trials are usually conducted in a large heterogeneous patient population. The development of stratified and personalised medicine means that drugs may be effective in a small population subgroup, so that such a strategy could be inefficient or lead to erroneous results. In such a setting, use of multiple testing and adaptive design strategies are promising approaches for identifying subgroups of particular interest and assessing the treatment effect in these subgroups. The project objective is to develop efficient study designs and analysis methods for the identification and confirmation of small target subpopulations as well as a clear understanding of their properties.
Overall objective
O.3 Develop frequentist and decision theoretic methods to predict patients’ responses to targeted treatments based on genetic features or other biomarkers such that subgroups of patients for which the benefit risk balance of a treatment is positive can be identified and confirmed.
Specific objectives
O.3.1 Develop frequentist methods for the identification and confirmation of subgroups where the benefit risk balance is positive.
O.3.2 Develop decision theoretic approaches for the identification and confirmation of subgroups.
O.3.3 Develop optimised adaptive enrichment designs and understanding of the potential improvement in efficiency achievable by adaptive designs compared to fixed sample designs.
Work package WP4: Research in use of evidence synthesis in the planning and interpretation of clinical trials in small populations and rare diseases
Ensuring that all available evidence is fully and appropriately synthesised and utilised is particularly important in small populations, where possible sample sizes for clinical trials may be very limited. The use of observational data from disease registries, PK/PD data from early trials and data from uncontrolled or non-randomised studies alongside randomised trial data can improve estimate precision and enable extrapolation, for example between patient subgroups. In this project we will develop novel methods for evidence synthesis, with an emphasis on methods particularly appropriate in the small population setting.
Overall objective
O.4 Develop evidence synthesis methods for small populations and rare diseases to support the planning, analysis and interpretation of a single randomized controlled trial.
Specific objectives
O.4.1 Assess feasibility and utility of the newly developed methods in small populations.
O.4.2 Apply generalized evidence synthesis approaches to paediatric studies and compounds developed for potentially multiple rare indications.
O.4.3 Provide software tools for design and analysis to facilitate application of methods developed.
Work package WP5: Research dissemination
The ultimate aim of all methodological development is the implementation of the novel methods, in this case improving the design, analysis and conduct of clinical trials in small populations. Whilst this can be achieved through the improvement of clinical trial guidelines to reflect the availability of efficient new methods, realistically this may not be achieved within the duration of this project. A key step on the path to guideline change is the peer-reviewed publication of methods and case studies in the statistical and medical literature. This disseminates the work and shows that the methods have been accepted by the scientific community. Our first objective is thus the publication of novel statistical methodology that we have developed for the design, conduct and statistical analysis and interpretation of clinical trials in small populations. This will start with publication of methodological papers in high quality international peer-reviewed statistical journals such as Statistics in Medicine or Biometrics.
We will also disseminate the results widely through relevant conferences and a project website. Involvement of clinical and regulatory experts, pharmaceutical industry statisticians and patient representatives will ensure the relevance of our work to key stakeholders. This will enable our second objective; the development and publishing of case-studies illustrating the application of methodology developed to clinical trials in small populations, which will again be published in high-quality peer-reviewed journals. This will initially be retrospective application to data from completed trials. We see this as an important step towards acceptance of the novel methodology, leading to prospective inclusion in clinical trial protocols. This will ensure that the methodology is disseminated and implemented as widely and rapidly as possible.
Overall objective
O.5 Disseminate and present innovative methodological developments from work packages.
Specific objectives
O.5.1 Build and maintain awareness of the project in interested stakeholder groups including relevant statisticians, clinicians and decision makers in academic, pharmaceutical industry and regulatory settings.
O.5.2 Organise and host a conference for dissemination and discussion of key project research results.
O.5.3 Publish project results in high-quality peer-reviewed journals, with publications made available on an open access basis when possible.
Work package WP6: Research management
In order to ensure the successful execution of the InSPiRe project, we will create and maintain appropriate and effective management structures and procedures, as described more fully in Section 2.1 below.
Overall objective
O.6 Create and maintain an organisational framework that facilitates the successful conduct of the project, guaranteeing that participants are fully integrated in the decision-making, management and delivery of the project and that financial resources are effectively managed.
Specific objectives
O.6.1 Ensure that all milestones are completed and all deliverables are delivered in time and that the consortium’s contractual duties are fulfilled.
O.6.1 Ensure effective communication between project participants and with third parties.
O.6.3 Deliver effective management of financial resources.
O.6.4 Comply with ethical requirements and collate documentation as required.
Project Results:
The aim of the InSPiRe (Innovative Methodology for Small Populations Research) project is to address the statistical challenges of the design and analysis of clinical trials in small populations. Research in these populations is challenging as the small number of patients makes conventional large clinical trials either infeasible or impossible.
A consortium of eight European partners as listed in Table 1 (attached), the InSPiRe project has addressed this challenge through four interrelated main Work Packages in three broad areas as illustrated in Figure 1 (attached).
The four main work packages are as follows:
WP1: Research in early phase dose-finding trials in small populations
WP2: Research in decision-theoretic designs for clinical trials in small populations
WP3: Research in confirmatory trials for small populations and personalised medicines
WP4: Research in use of evidence synthesis in the planning and interpretation of clinical trials in small populations
These have been supported by work packages WP5: Research Dissemination and WP6: Research Administration.
Work packages WP1, WP3 and WP4 address the challenge of small population research in two ways. Work packages WP1 and WP3 focus on improved and optimal clinical trial design. This ensures that the collection of data is as efficient as possible, enabling the most information to be obtained from the small number of patients that can be recruited into a clinical trial in a small population. The main focus of work package WP3 has been the design of clinical trials in personalised or stratified medicine, where an efficient design can both identify a subset of the population for which a new treatment gives a positive risk-benefit balance, and demonstrate the effectiveness of the treatment in that population in a scientifically and statistically rigorous way. Work packages WP1 and WP4 focus on the use of data additional to the primary endpoint data from a clinical trial that are often considered in isolation in the main trial analysis. In WP1, these data are collected from the patients in the trial. In particular, we have considered the use of data on pharmacokinetics and pharmacodynamics in early phase dose-finding studies. Such data are usually collected, but their analysis is not usually combined with that of clinical endpoints or used for ongoing design of the study. This work package has considered how these data can be used to improve study efficiency. In work package WP4, use of additional data from outside the trial is considered. These data may come from other randomised trials or from observation or registry data.
Work package WP2 takes a more general view of the design of clinical trials in small population groups, investigating how the design of such studies might reflect the size of the population under investigation. By considering the value of information obtained from a clinical trial in terms of the benefits for the population, this work package has considered how the level of evidence appropriate for decision-making may vary depending on the population size.
The results from each of these four work packages are described in detail in the remainder of this section.
The InSPiRe project team have worked closely with the two other projects funded under the same EU FP7 call; the Asterix and IDeAl projects, with some work being undertaken jointly by more than one project. A particularly strong collaboration between the IDeAl and InSPiRe projects associated with the work of InSPiRe WP3. A summary of the work planned in all three projects has been published in the Orphanet Journal of Rare Diseases (Hilgers et al., 2016).
Work Package 1: Research in early phase dose-finding trials in small populations
Work package 1 has focussed on the development of innovative design for early phase dose-finding trials in small population. Early phase dose-finding studies are the first trials of a new medicinal product or a combination procedure in humans and aim to obtain reliable information on an appropriate dose for use in further clinical trials. Sample sizes for such studies are typically small, and this is particularly true in a small population context. Recent work in this field has led to efficient designs for dose-finding based on the minimum number of participants. These methods have generally relied primarily on observed safety data, however, and there is a growing awareness of the need to better incorporate PK/PD information. Our methods can be used to better estimate the dose level to be recommended allowing for small population characteristics and prior knowledge of the drug together with innovative use of PK/PD and other external data.
Our objective in WP1 was to include all available information to better design the dose-finding trial and to better estimate the recommended dose level for further evaluation whilst ensuring the sample size for early phase trials is kept to a minimum. This involved (i) the incorporation of the pharmacokinetic/pharmacodynamic (PK/PD) information in the dose-allocation process, (ii) the addition of multiple constraints on cumulative toxicities, (iii) the development of an innovative design for the LEVENEONAT clinical trial for reducing neonatal seizures, (iv) the development of new methods for bridging studies from adults to children and (v) the proposal of a new way of eliciting expert information.
Task 1.1 Combining PK data for optimal dose-finding with categorical PD outcome
We began our work by performing a literature review on already existing methods which combine PK or PK/PD data with dose-finding. Since only few methods with both PK and PD were found, we decide to start our work by including PK data only in the dose allocation of early phase clinical trials.
The aim of phase I clinical trials is to obtain reliable information on safety, tolerability, PK, and mechanism of action of drugs with the objective of determining the maximum tolerated dose (MTD). In most phase I studies, dose-finding and PK analysis are analysed separately and no attempt is made to combine them during dose allocation. In cases such as rare diseases, paediatrics, and studies in a biomarker-defined subgroup of a larger population, the available population size will limit the number of possible clinical trials that can be conducted. Combining dose-finding and PK analyses to allow better estimation of the dose-toxicity curve should then be considered. In this work, we proposed, studied, and compared methods to incorporate PK measures in the dose allocation process during a phase I clinical trial. These methods approach this in different ways, such as using PK observations as a covariate, as the dependent variable or in a hierarchical model. We conducted a large simulation study that showed that adding PK measurements as a covariate only does not improve the efficiency of dose-finding trials either in terms of the number of observed dose limiting toxicities or the probability of correct dose selection. However, incorporating PK measures through a hierarchical model does allow better estimation of the dose-toxicity curve while maintaining the performance in terms of MTD selection compared to dose-finding designs that do not incorporate PK information. In conclusion, we found that using PK information in the dose allocation process enriches the knowledge of the dose-toxicity relationship, facilitating better dose recommendation for subsequent trials.
This work has been accepted for publication by Biometrical Journal and is currently available online ahead of print (Ursino et al., DOI: 10.1002/bimj.201600084).
An R package called “dfpk” was developed to provide a practical implementation of the methods presented in the previous work. The package provides, for each method, a function (nextDose) to estimate the probability of toxicity and to suggest the dose to give to the next cohort, and a function to run trial simulations before starting the trial (nsim). Functions to generate data-sets and to display the results graphically are also provided. The package dfpk can be downloaded freely from the CRAN repository (https://cran.r-project.org/web/packages/dfpk/index.html). A manuscript has been submitted for publication (Toumazi et al., 2017) and is currently under review.
During the first year, there was the possibility to set a collaboration with the Columbia University (NY) in order to develop a novel methodology on including grade 2 toxicity information in the dose-finding design. Even if PK was not introduced in this work, this was considered to be a good idea for the WP1, since we focused on including the maximum available information in the design. Therefore, we developed a dose-finding design where multiple constraints are added on cumulative toxicities. The toxicity profile of newer anticancer treatments such as targeted and immunotherapeutic agents differs from that of chemotherapy. While some of these newer agents cause dose limiting toxicities, others cause lower grade toxicities which may not occur within the first cycle of treatment. Thus, in the early development of these agents it is necessary to account for both lower grade toxicity, as well as, late onset and cumulative toxicity. This new design addressed the concern regarding late-onset dose-limiting toxicities (DLT), moderate toxicities below the threshold of a DLT, and cumulative toxicities that may lead to a DLT, which are either disregarded or handled in an ad-hoc manner in cancer trials. An extension of the Time-to-Event Continual Reassessment Method (TITE-CRM) which allows for multiple constraints, and can account for partial information on both DLT and moderate toxicities was proposed. The method was illustrated in the context of an Erlotinib dose-finding trial with low DLT rates, but a significant number moderate toxicities that led to treatment discontinuation in later cycles. Based on simulations, our method performed well at selecting the dose that satisfies both the DLT and moderate toxicity constraints when the true MTD for the individual constraints do not coincide, and performs better than the TITE-CRM when the true MTDs coincide, by reducing the probability of overdose while having similar probability of correct selection. This work has been submitted for publication (Lee et al., 2017) and is currently under review.
Task 1.2 Using continuous PD data
During the project, we had the opportunity to help designing a new clinical trial in a paediatric context, the LEVNEONAT (NCT 02229123). The aim of the trial is to find the optimal dose of Levetiracetam for reducing neonatal seizures. The LEVNEONAT clinical trial follows the NEMO study (NEonatal seizures with Medication Off-patent, NCT 01434225, FP7-HEALTH-2009-4.2-1 grant agreement 241479, The NEMO Project), which was stopped after that unexpected safety events were measured at long term. The LEVNEONAT was planned for the same indication but with another drug. A novel approach was needed to respond to requirements based on what was observed in NEMO study. Clinical trials in vulnerable populations, such as newborns, are extremely difficult to conduct and this was considered to be a good case-study for the approaches developed in the WP1 package. It gave us the opportunity to evaluate the impact of using a continuous regression for efficacy, at the end of the trial, instead of the usual dichotomised variable.
The LEVNEONAT is a phase I/II trial aiming at finding the recommended dose of Levetiracetam for treating neonatal seizures and it was planned with a maximum sample size of 50. In the trial, initially 4 doses were considered with 3 primary outcomes: efficacy and two types of toxicity that occur at the same time but measured earlier or later in time. In the case of failure, physicians could add a second agent as a rescue medication, which may differ from centre to centre. The primary outcomes were modelled via a logistic model for efficacy and a weighted likelihood with pseudo outcomes for the two toxicities taking into account the dependences under Bayesian inference. Pseudo toxicity outcomes were introduced to consider the possible effect of the second agent. Finally, this model permits sequential patient accrual. Dose escalation rules were based on an adaptive threshold for posterior probabilities, considering only early toxicity in the start-up phase. Simulations were conducted to assess the design properties and the correct dose was recommended more than 60% of the time on average. A manuscript on this work has been submitted for publication and is currently under review (Ursino et al, 2017a). The trial got the regulatory and ethical committee agreement and inclusion should start in September 2017.
In parallel, we extended the work on combining PK data in the dose allocation of early phase clinical trials started in Task 1.1. The extension deals with combining full PK/PD analysis in the dose escalation process in Phase I/II. We started to focus on trials where the toxicity outcome is defined as the area under the curve (AUC) above a specified threshold and the efficacy outcome is defined as the percentage of inhibition of a biomarker. The right recommended dose is defined as the one with the highest probability of efficacy without toxicity, under marginal toxicity and efficacy constraints. We continued to use a Bayesian design which allowed us to add prior information on parameters and to run estimation also in presence of very limited sample size at the beginning of the trial. We proposed a two stage design: in the first stage, the bivariate continual reassessment method is used to guide the dose escalation process, while in the second stage, the full PK/PD model is used to compute posterior probabilities to guide the dose allocation. We conducted a large simulation study that showed that the two stage model is less sensitive to prior misspecification; it is also more conservative, reducing overdosing while increasing the percentage of the right selected dose. We started to adapt the design for Phase I/II studies where the probability of toxicity and efficacy depends on the AUC and the percentage of inhibition, respectively. A paper is in preparation to report this work.
Task 1.3 Combining data from different sources
Task 1.3 focussed on bridging studies, in particular on (1) how to use previous information to set up prior distributions and better design PK, (2) and phase I/II studies for children when PK and toxicity data are available from adults; (3) and on building prior distributions using elicited expert opinion. Parts (1) and (2) were done in collaboration with IDEX (Paris Cité Sorbonne) project.
Looking at bridging studies, we started by focussing on how to design PK studies in paediatric population using adult data and then we extended the work to early phase dose-finding trials.
Regarding PK, the objectives of the study were to design a pharmacokinetic study by using information about adults and evaluate the robustness of the recommended design through a case study, the antimalarial drug mefloquine. Combining PK modelling, extrapolation, and design optimisation led to a design for children with five sampling times. PK parameters were well estimated by this design with small relative standard errors. Although the extrapolated model did not predict the observed mefloquine concentrations in children very accurately, the design optimised based on the extrapolated model allowed precise and unbiased estimates across various model assumptions, contrary to the empirical design. In conclusion, using information from adult studies combined with allometry and maturation can help provide robust designs for paediatric studies.
The resulting paper has been published in Antimicrobial Agents Chemotherapy and is available online as an open access publication (Petit et al., Statistical Methods in Medical Research, 2016a).
This first work shows that even imperfect priors can be used to improve the design of clinical studies in children. The next step is figuring how to account for these priors in dose-finding studies. We propose a unified approach for extrapolation and bridging adult information in early phase dose-finding studies in paediatrics. We investigated calibrating an existing dose-finding model for a paediatric population using adult observations, such as PK, toxicity and efficacy, in order to choose the parameters of the dose-finding model, i.e. the initial guess of toxicity for each dose (working model) and the prior distribution of parameters, and the dose-range given to the paediatric population in the study. We evaluated the resulting method on a drug used against cancer, erlotinib.
This work has been accepted for publication by Statistical Methods in Medical Research and is currently available online ahead of appearing in the printed journal (Petit et al., DOI: 10.1177/0962280216671348).
We have also performed a systematic review on dose-finding research in paediatrics from January 1st 1996 to October 30th 2016 in order to evaluate the use of extrapolation in paediatric designs and to propose related methodologies. We included (1) publications from January 1st 1996 to October 30th 2016 with (2) a paediatric population aged 0 to 12 years old and (3) a drug administration of at least two or more doses, associated with or without another drug treatment. We excluded adult population and unrelated studies. A total number of 167 publications were selected and reviewed. Among them, 119 publications reported an adaptive sequential dose-finding design while 48 studies used a randomisation procedure. Extrapolation was used for the choice of doses in 34% of the publications and in one study for choices related to the dose-finding design. Among them, the extrapolation rule is predominantly made according to an arbitrary rule. This systematic review suggested that the use of an extrapolation criteria is rare and model-based and/or simulations are rarely used for designing a paediatric dose-finding study. A draft of the paper is in preparation.
During the project, we were involved in the analysis of a 70-patients randomized trial to compare two treatments for idiopathic nephrotic syndrome, a rare disease in children.
We proposed a Bayesian methodology for constructing a parametric prior on two treatment effect parameters, based on graphical information elicited from a group of expert physicians. The methodology relies on histograms of the treatment parameters constructed manually by each physician, applying the method of Johnson, et al. (2010). For each physician, a marginal prior for each treatment parameter characterized by location and precision hyper-parameters is fit to the elicited histogram. A bivariate prior is obtained by averaging the marginals over a latent physician effect distribution. An overall prior is constructed as a mixture of the individual physicians’ priors. A simulation study evaluating several versions of the methodology is presented. A framework is given for performing a sensitivity analysis of posterior inferences to prior location and precision, and is illustrated based on the idiopathic nephrotic syndrome trial.
A manuscript on this work has been submitted for publication and is currently under minor revision in Statistical Methods in Medical Research (Thall et al, 2017).
Publications in print or in press in peer reviewed publications (all are available as open access publications):
Ursino, M., Zohar, S., Lentz, F., Alberti, C., Friede, T., Stallard, N. and Comets, E. (2017). Dose-finding methods for Phase I clinical trials using pharmacokinetics in small populations. Biometrical Journal. In press. DOI:10.1002/bimj.201600084
Petit, C., Jullien, V., Samson, A., Guedj, J., Kiechel, J. R., Zohar, S., and Comets, E. (2016a). Designing a Pediatric Study for an Antimalarial Drug by Using Information from Adults. Antimicrobial agents and chemotherapy, 60(3), 1481-1491.
Petit, C., Samson, A., Morita, S., Ursino, M., Guedj, J., Jullien, V., ... and Zohar, S. (2016b). Unified approach for extrapolation and bridging of adult information in early-phase dose-finding paediatric studies. Statistical Methods in Medical Research. In press. DOI: 10.1177/0962280216671348.
Manuscripts submitted for publication in peer reviewed journals currently under review:
Toumazi, A., Comets, E., Alberti, C., Friede, T., Lentz, F., Stallard, N., Zohar, S. and Ursino, M. (2017). An R-package for Bayesian Dose-Finding Design using Pharmacokinetics (PK) for Phase I Clinical Trials.
Shing, L.M. Ursino, M., Cheung, Y.K. and Zohar, S. (2017). Dose-finding designs for cumulative toxicities using multiple constraints.
Ursino, M., Yuan, Y., Alberti, C., Comets, E., Favrais, G., Friede, T., Lentz, F., Stallard, N., and Zohar, S. (2017a). A Dose-finding Design for Seizure Reduction in Neonates.
Thall, P.F. Ursino, M., Baudouin, V., Alberti, C. and Zohar, S. (2017). Bayesian Treatment
Comparison Using Parametric Mixture Priors Computed from Elicited Histograms.
Work Package 2: Research in decision-theoretic designs for clinical trials in small populations
Work package 2 has focussed on the development of decision-theoretic designs for clinical trials in small populations. These methods explicitly enable evaluation of the level of evidence required from a clinical trial to best inform clinical practice enabling efficient clinical trial design. We have also built on the health economic value of information approach, enabling an assessment of the appropriate level of information required for clinical decision-making in the small population setting.
Most clinical trials are designed with no reference to the size of the population in which the research is conducted. Whilst this may be reasonable in a large population, in rare diseases or other small populations it can lead to designs that are inappropriate. If the population under investigation is small, a large proportion of the patient group may be recruited to a clinical trial. Recruitment to one trial may thus have an impact on the conduct of other trials or even reduce significantly the size of the population receiving usual care. The value of one clinical trial must therefore be compared with that of other trials if research is to proceed efficiently. Such a comparison, and resulting appropriate trial design, is enabled through the decision-theoretic methods that we have developed.
Task 2.1 Review of relevant literature
In order to provide an overview of the current state of the art and to establish the context for future research work, the initial work of Work Package 2, Task 2.1 was completion of a review of literature on the use of decision-theoretic approaches in clinical trial design.
After removal of duplicates, a total of 8,326 potentially relevant articles were identified, with 10 further articles identified through reference screening. Screening of article titles, abstracts and, where considered necessary, the whole paper, 67 articles were found to propose decision-theoretic design methods relevant to small clinical trials. The review discusses these 67 articles in detail, classifying them according to the type of study design proposed (single-arm single-stage, single-arm multi- stage, two-arm single-stage, two-arm multi-stage, multi-arm, series of trials and enrichment design) and the type of gain function proposed (simple or more realistic utility based on patient, societal, commercial or unspecified perspective).
The review has been published in Statistical Methods in Medical Research and is available online as an open access publication (Hee et al., Statistical Methods in Medical Research, 2016).
Task 2.2a Development of decision theoretic framework
Task 2.2a focussed on the development of a decision-theoretic model for the clinical evaluation process that incorporates gains both to patients within a trial who receive effective treatment and to future patients receiving either a new treatment or the current standard treatment depending on the conclusion reached at the end of a clinical trial. Based on a model proposed by Cheng et al (Biometrika 2003), attention has focused in particular on how the optimal sample size for a clinical trial depends on the size of the patient population.
Taking a general viewpoint, we have shown that for a wide range of response distributions, including responses with normal, exponential, Bernoulli and Poisson distributions, and gain function forms, the optimal trial sample size is proportional to the square root of the population size, with constant of proportionality depending on the gain function form and prior distribution of the parameters of the distribution of the data. This has important consequences in terms of optimal design as it challenges the usual method of sample size determination based on frequentist error rates to show that in a small population setting a smaller trial than usual may be optimal.
This work has been accepted for publication by Biometrical Journal and is currently available online ahead of appearing in the printed journal (Stallard et al., DOI: 10.1002/bimj.201500228).
In order to determine how our theoretical work relates to current practice in small clinical trials, we have conducted an analysis of trials in rare diseases recorded in the ClinicalTrials.gov database. The aim was to use disease prevalence data from Orphadata to identify trials registered in ClinicalTrials.gov that studied a rare disease and to investigate how the sample sizes for these trials depended on the disease prevalence and on other factors including the gender and age of patients recruited to the trial, trial phase, trial design and trial date. Evaluation of all clinical studies registered on ClinicalTrials.gov up to 27 September 2015 indicated that of the 186,941 trials registered, 1,567 studied a single rare condition with prevalence information on Orphadata. Of these, there were 19 (1.2%) trials studying disease with prevalence <1/1,000,000, 126 (8.0%) trials with 1-9/1,000,000, 791 (50.5%) trials with 1-9/100,000 and 631 (40.3%) trials with 1-5/10,000. Of these, 1160 (74%) were phase 2 trials. Our study found that the sample sizes in phase 2 trials were similar across all prevalence classes after adjusting for other covariates; mean, 15.7 (95% CI, 8.7–28.1) 26.2 (16.1–42.6) 33.8 (22.1–51.7) and 35.6 (23.3–54.3) for prevalence <1/1,000,000, 1-9/1,000,000, 1-9/100,000 and 1-5/10,000, respectively. Estimated size of phase 3 trials of rarer diseases, <1/1,000,000 (19.2 6.9–53.2) and 1-9/1,000,000 (33.1 18.6–58.9) were similar to those in phase 2 but were statistically significant lower than the slightly less rare diseases, 1-9/100,000 (75.3 48.2–117.6) and 1-5/10,000 (77.7 49.6–121.8) trials. This work has been published in Orphanet Journal of Rare Diseases (Hee et al., 2017a) and is currently under review.
We have also developed decision-theoretic methods for the simultaneous design of a series of trials. This work has focussed on optimal design and decision-making in a setting in which a number of treatments are to be evaluated in a small fixed population. This extends the work described above through the use of a joint prior distribution for effects of different treatments that are correlated provides gains in efficiency. Use of the methodology has been illustrated through retrospective application in an example in orthopaedic surgery. A manuscript on this work has been accepted for publication in Biometrical Journal (Hee et al, 2017b).
Task 2.2b Construction of an approximate utility function
Building on the theoretical work of Task 2.2a we have explored the application of the methodology through the development of a number of specific case-studies of clinical trials in rare diseases.
The aim of the work is to explore the application of methods developed in a more general context in Task 2.1 and to compare this with other approaches for the design of clinical trials, particularly in the rare disease setting.
We consider three case studies; Lyell’s disease, adult-onset Still’s disease and cystic fibrosis. In each case the aim to plan the sample size for an upcoming study. Three alternative sample size approaches are described: traditional sample size calculation based on power to show a statistically significant effect, sample size calculation based on assurance and the decision theoretic approach based on the published work from Task 2.2a. The latter has the property that trial sample size depends on the size of the disease population which is interesting in the context of rare diseases. We outline in detail the reasonable choice of parameters for these approaches for each of the three case studies and calculate sample sizes. We stress that the influence of the input parameters need to be investigated in all approaches and recommend investigating different sample size approaches before deciding finally on the sample size.
A manuscript has been submitted for publication (Miller et al, 2017) and is currently under review.
Task 2.3 Development of value-of-information approaches
Building on the work of Task 2.2a we have developed methodology for the use of a decision-theoretic value of information method for a confirmatory phase III trial, particularly in the small population setting. In particular, we determine the optimal significance level that should be used if a frequentist hypothesis test is used for decision-making at the end of the trial, and investigate how this significance level changes with the size of the disease population. We show how decision-theoretic VOI analysis suggests a more flexible approach with both type I error rate and power (or equivalently trial sample size) depending on the size of the future population for whom the treatment under investigation is intended.
We focus on a primary endpoint assumed to be continuous and normally distributed with unknown mean with some normal prior distribution, the latter representing information on the anticipated effectiveness of the therapy available from sources external to the trial itself. We explicitly specify the gain in terms of improvement in primary outcome for patients treated with the new therapy and compared this with the costs, both financial and in terms of risk of potential harm, of treating patients, either in the trial or in the future if the therapy is approved.
In line with our previous work, we find that as the size of the population increases, the optimal sample size for the clinical trial also increases. For non-zero cost of treating future patients, either monetary or potential harmful effects, a more stringent significance level, that is stronger evidence, is required for approval as the population size increases, though this is not the case if the costs of treating future patients are ignored.
A manuscript describing the work and also a retrospective application in a case study, has been submitted to a peer reviewed publication and is currently under review.
Task 2.4. Consideration of levels of evidence appropriate for decision-making in small populations
Building on Tasks 2.2 and 2.3 we have extended the models developed there in three ways. First, we relax the assumption that the population size is known. Rather we assume that we can specify a prior distribution for the incidence of new disease cases, and that we learn about this prevalence through the rate at which patients enter the trial. As the incidence of the disease is considered unknown, the size of the trial is specified not in terms of the number of patients to be recruited, but the duration in time over which recruitment will take place, so that the optimal length of trial is obtained as a proportion of patient horizon specified in terms of the total time that the new treatment might be considered to be in use. The second extension is to consider a two-stage trial, in which a decision is made at an interim analysis as to whether to stop the trial, either to recommend the new treatment or to abandon development, or to continue with a second stage of the trial prior to making a final decision. If the trial continues, the size of the second stage can be chosen optimally based on the first stage data. A manuscript describing this work is in preparation and will be submitted for publication.
The third extension to the model developed in Tasks 2.2 and 2.3 is to extends the utility function formulation to the setting in which we consider two decision-makers; the “sponsor” responsible for decisions regarding the design and conduct of the trial and “society”, who can make decisions concerning reimbursement offered to the sponsor on the basis of potential benefit from an effective treatment. We explore the consequences of this formulation of the decision-making problem, both in settings in which the sponsor and society each seek to maximise their own benefit and when they can cooperate. Again, a manuscript to be submitted for publication is in preparation.
Publications in print or in press in peer reviewed journals:
Hee, S.W. Hamborg, T., Day, S., Madan, J., Miller, F., Posch, M., Zohar, S. and Stallard, N. (2016) Decision theoretic designs for small trials and pilot studies: a review. Statistical Methods in Medical Research, 25, 1022-1038.
Kunz CU, Stallard N, Parsons N, Todd S, Friede T (2016) Blinded versus unblinded estimation of a correlation coefficient to inform interim design adaptations. Biometrical Journal 2, 344–357.
Hee, S.W. Willis, A., Tudur-Smith, C., Day, S., Miller, F., Madan, J., Posch, M., Zohar, S. and Stallard, N. (2017a) Does the low prevalence rate affect the sample size of interventional clinical trials of rare diseases? An analysis of data from the Aggregate Analysis of ClinicalTrials.gov. Orphanet Journal of Rare Diseases, 12, 44.
Stallard, N., Miller, F., Day, S., Hee, S.W. Madan, J., Zohar, S. and Posch, M. Determination of the optimal sample size for a clinical trial accounting for the population size. Biometrical Journal. In press. DOI: 10.1002/bimj.201500228.
Hee, S.W. Parsons, N. and Stallard, N. (2017b) Designing a series of decision-theoretic phase II trials when treatment effects are correlated with the Sarmanov multivariate beta-binomial distribution. Biometrical Journal. In press.
Manuscripts submitted for publication in peer reviewed journals currently under review:
Miller, F., Day, S., Hee, S.W. Madan, J., Pearce, M., Posch, M., Stallard, N., Vågerö, M. and Zohar, S. (2017) Approaches to sample size calculation for clinical trials in small populations.
Pearce, M., Hee, S.W. Madan, J., Posch, M., Day, S., Miller, F., Zohar, S. and Stallard, N. Value of information methods to design a clinical trial in a small population to optimize a health economic utility function.
Work Package 3: Research in confirmatory trials for small populations and personalised medicines
In work package 3 frequentist and decision theoretic methods were developed in order to predict patients’ responses to targeted treatments based on genetic features or other biomarkers such that subgroups of patients for which the benefit risk balance of a treatment is positive can be identified and confirmed.
Task 3.1. Literature review
In a first step we conducted a literature review describing the current trial designs and statistical approaches for the development of targeted therapies. These methods allow for the identification and confirmation of subpopulations of patients where the treatment under investigation is effective and has a positive benefit-risk balance. The identified methods were classified based on characteristics of the proposed trial designs and analysis methods. In particular we distinguished exploratory and confirmatory subgroup analysis and frequentist, Bayesian and decision theoretic approaches. Furthermore we reported on fixed-sample, group-sequential, and adaptive designs and illustrated the available trial designs and analysis strategies with published case studies.
The literature search was based on PubMed and was restricted to methodological journals. We first conducted a keyword search to identify an initial set of articles and then screened the references listed in these papers. Furthermore, relevant literature identified by manual searches was added to the review. In total 86 papers were classified as relevant and included in the survey.
The review has been published in the Journal of Biopharmaceutical Statistics and is available online as open access publication (Ondra et al., Journal of Biopharmaceutical Statistics, 2016). The publication is listed among the top 3 most read papers of the Journal of Biopharmaceutical Statistics on the journal’s website .
Task 3.2 Development and assessment of frequentist methods and Task 3.3.
Development of decision-theoretic methods for subgroup identification
Frequentist and decision theoretic methods were addressed in several steps. On the one hand we focused on the development of confirmatory hypothesis tests that control frequentist error rates in complex trials where several subgroups are investigated. On the other hand we used decision theoretic methods to optimize trial designs that control frequentist false positive rates as required by regulatory authorities. Thus, Tasks 3.3 was addressed combined with Task 3.2.
Development of Frequentist designs. We developed frequentist fixed sample trial designs in a setting where a continuous biomarker is used to define subgroups. Such continuous biomarkers can be either quantitative measurements or biomarker signature scores that summarize the measurement of several individual biomarkers. The continuous biomarker is dichotomized based on a threshold to define biomarker-low and biomarker-high subpopulations. If there is insufficient information on the choice of the threshold, several thresholds are investigated and separate hypothesis test for each of the subgroups can be performed. This causes a multiple testing problem and an appropriate adjustment for multiple testing has to be applied. Because the nested structure of subpopulations defined by different thresholds for a continuous biomarker is similar to the structure of analysis populations in group sequential trials (Jennison and Turnbull, 1999), it has been proposed to use critical boundaries of group sequential designs to test such nested subpopulations (Spiessens and Debois, 2010). We showed that special care has to be taken when applying such group sequential boundaries to test hypothesis for multiple nested subpopulations. Especially, we show that the false positive rate of such tests may be inflated, if the biomarker has a prognostic effect (i.e. it is correlated with the prognosis of the patient but not the treatment effect). In a simulation study we quantify the potential inflation of the type I error rate of testing procedures based on group sequential designs. Furthermore, we propose improved hypotheses testing approaches based on regression models and combination tests and demonstrate that they control the family-wise error rate also in situations where the treatment is not effective but the biomarker is prognostic. A further manuscript, focusing on the design of multiple testing procedures for nested subgroups, that also investigates the case where a continuum of thresholds is considered, is under preparation and will be completed after the project end.
Decision Theoretic Methods to Optimize Frequentist Trial Designs. We investigated the potential of Bayesian decision theoretic methods to optimize single stage designs as well as adaptive designs (the latter are addressed in Task 3.4). While in standard parallel group clinical trials the statistical power to demonstrate a treatment effect is typically the basis for the planning of clinical trials, the consideration of power alone does not sufficiently represent the losses and gains of correct and incorrect test decisions when several subgroups are tested. Subgroup analyses are challenging because several types of risks are associated with inference on subgroups. On the one hand, by disregarding a relevant subpopulation a treatment option may be missed due to a dilution of the treatment effect in the full population. Furthermore, even if the diluted treatment effect can be demonstrated in an overall population, it is not ethical to treat patients that do not benefit from the treatment when they can be identified in advance. On the other hand, selecting a spurious subpopulation increases the risk to restrict an efficacious treatment to a too narrow fraction of a potential benefiting population. In order to quantify these risks utility functions are proposed, representing the benefit of a particular clinical trial from a sponsors and a public health perspective. To quantify the risks and losses in these settings, we derived utility functions that take into account the size of the population for which a claim is made and for which a novel treatment becomes available. We define utilities from the perspective of different stakeholders: the sponsor’s view as well as a public health view. We assume that the utility of the sponsor is the net present value (NPV), while for the public health it is the expected health benefit (adjusted for the cost of the trial). In the planning phase expected utilities for different trial designs and different utility functions can be computed based on a Bayesian approach which relies on a prior distribution on the effect sizes in the subgroup and the full population.
In three articles we investigated optimized single stage designs, focusing on different aspects of the testing procedure: optimizing the sample size, the multiple testing procedure (especially the weights used in the multiple test) and the type of the design. Especially, we considered the classical design, where no biomarker information is used and only the full population is tested (such that no cost for biomarker development and determination of the biomarker status of patients incur), the enrichment design, where only biomarker positive patients are included, a stratified design, where patients from the full population are included and the treatment effect is tested in the subgroup and the full population. In the latter, no biomarker based patient selection takes place such that the prevalence of the subgroups in the trial is the same as the prevalence in the underlying population. Finally, we considered partial enrichment designs, where the prevalence of the subgroup in the trial is a design parameter that can be optimally chosen to maximize the expected utility. The models have been formulated for parallel group designs but can equally be applied to cross-over trials.
We find that the optimal trial designs depend sensitively on the considered scenario, especially on the prevalence of the subgroup, the strength of the prior evidence that the treatment is only effective in the subgroup, as well as on the cost of the biomarker development and determination. Furthermore, we observe that optimal designs for the sponsor and the public health view differ. Trials optimized under the sponsor view tend to have smaller sample sizes and are conducted in the full population also in settings where there is strong evidence that the treatment is effective in the subpopulation only. This is due to the fact that due to the variability of treatment effect estimates a treatment might appear effective in a subpopulation (and bring a gain for the sponsor) even if it is not effective and has no benefit for patients.
The work has been published in articles in Biometrical Journal (Graf et al., 2015) and Plos One (Ondra et al., 2016). The article Graf et al. (2015) is the second most cited article of Biometrical Journal of all articles published in 2015 and 2016 : A further manuscript has been submitted to Statistical Methods in Medical Research (Ondra et al. 2017) .The work on the articles Ondra et al. (2016,2017 ) was performed in cooperation with researchers from the EU-FP7-IDEAL project (Carl-Frederic Burman and Sebastian Jobjörnsson) and Robert A Beckman (Georgetown University) who is co-chair of the small population working stream of the DIA.
Task 3.4. Adaptive enrichment trials
Frequentist adaptive designs. A comprehensive description of the statistical methodology for adaptive frequentist designs in confirmatory trials with multiple objectives has been developed. This work focuses on two-stage designs, i.e. trials with one interim analysis, and settings where multiple subgroups or treatment arms are investigated. We describe the application of adaptive combination tests that guarantee control of the type I error rate in a wide range of scenarios. Especially, the application of the closure principle in the context of adaptive two stage designs is demonstrated. This methodology is the basis for the implementation of adaptive enrichment designs. In these two stage designs, in the first stage patients are recruited from the full population. In an interim analysis, based on the interim data, the trial design of the second stage may be modified. For example, recruitment may be limited to patients in a subgroup of biomarker positive patients and/or the sample sizes in the subgroups may be adapted. The adaptation may be based on all information observed at the interim analysis, including information on secondary and surrogate endpoints and safety information.
The application for different endpoints is discussed, including normally distributed, binary and survival endpoints. In the last section, some extensions and applications of the designs are described and implications for the regulatory assessment of such designs are discussed.
This work will appear as an invited book chapter “Adaptive Designs with Multiple Objectives” in the “Handbook of Statistical Methods for Randomized Controlled Trials" (CRC Press).
Furthermore, for the special case of adaptive designs with a survival endpoint we developed hypothesis tests that allow for early rejections of the null hypothesis in an interim analysis. This work generalizes earlier adaptive procedures that control the familywise type I error rate in the strong sense but have limitations: they either cannot use information from surrogate endpoints for adaptive decision making or do not allow for early rejections in an interim analysis. We derived hypothesis tests for multi-armed and enrichment designs which do not have these restrictions and can be used within a multi-stage group sequential design.
This work is included in Deliverable 3.3 and is complemented by a simulation study in cooperation with the EU-FP7 project Asterix. The work is presented in a topic contributed session at the Joint Statistical Meeting 2017, Baltimore, USA. The research is performed in cooperation with Silke Jörgens (Icon Clinical Research Inc) as external partner.
Development of decision theoretic methods to optimize frequentist adaptive designs. In the articles Graf et al. (2015) and Ondra et al., 2017 (see above), we also addressed the optimization of frequentist adaptive two stage enrichment designs based on a decision theoretic approach. Using extensive simulations (Graf et al., 2016) and a dynamic programming algorithm (Ondra et al. 2017) we optimize interim adaptation rules maximizing utility functions. We show that adaptive enrichment designs can lead to a higher expected utility as compared to single stage designs, especially in settings where there is high uncertainty if the treatment is effective only in a subgroup. As for single stage designs, we observe differences in the optimized sample sizes if trials are optimized under the sponsor or the public health perspective. An important advantage of adaptive designs compared to single stage designs are their increased robustness with regard to a misspecification of the planning assumptions.
Publications in print or in press in peer reviewed publications:
Graf, A., Posch, M. and König, F. (2015). Adaptive designs for subpopulation analysis optimizing utility functions. Biometrical Journal 57, 76–89.
Ondra, T., Dmitrienko, A., Friede, T., Graf, A., Miller, F., Stallard, N. and Posch, M. (2016). Methods for identification and confirmation of targeted subgroups in clinical trials: A systematic review. Journal of Biopharmaceutical Statistics, 26, 99-119, DOI: 10.1080/10543406.2015.1092034
Ondra, T., Jobjörnsson, S., Beckman, R. A., Burman, C. F., König, F., Stallard, N. and Posch, M. (2016). Optimizing trial designs for targeted therapies. PLoS One 11(9): e0163726. DOI:10.1371/journal.pone.0163726
Manuscripts submitted for publication currently under review:
Ondra, T., Jobjörnsson, S., Beckman, R. A., Burman, C. F., König, F., Stallard, N. and Posch, M. Optimizing adaptive enrichment designs (Statistical Methods in Medical Research)
Graf, A.C. Wassmer, G., Friede, T., Gera, R. and Posch, M. Robustness of testing procedures for confirmatory subpopulation analysis based on a continuous biomarker (Statistical Methods in Medical Research)
Wassmer, G., König, F. and Posch, M. Adaptive designs with multiple objectives, Handbook of Statistical Methods for Randomized Controlled Trials, CRC Press.
Manuscripts in preparation:
Graf, Magirr, Dmitrienko, Posch, Clinical trial designs with a threshold selection for a continuous biomarker
Jörgens , S., Wassmer, G., König, F. and Posch, M. Confirmatory adaptive designs with a survival endpoint using a short-term endpoint for treatment arm or population selection.
Work package 4: Research in use of evidence synthesis in the planning and interpretation of clinical trials in small populations
Research in rare diseases is often complicated by the sparseness of empirical information that goes together with the rareness of the indication itself. Evidence synthesis methods therefore play an important role in maximizing the amount of information to be derived from sparse data. Work package 4 aimed at the development of methods to synthesize few separate sources of information and to utilize external evidence to support the analysis of otherwise potentially underpowered clinical trials in rare indications.
While work package 4 was inspired by concrete applications such as the EARLY PRO-TECT trial in Alport disease, it has already fed back into applications for rare diseases, such as neuromyelitis optica (Chataway and Friede, 2016), paediatric myocarditis (Messroghli et al., 2017) and Creutzfeldt-Jakob disease (Varges et al., 2017). Motivated by the methodological challenges commonly faced in rare diseases research, the refinement of analysis methods valid also for the case of small samples was promoted (Konietschke et al., 2016).
Task 4.1 Systematic literature review
In order to gain some insight into the methodological challenges faced by clinical trials in rare diseases, and to get an overview of conventional approaches to their analysis, we conducted a systematic literature review. We selected two rare indications, paediatric multiple sclerosis (MS) and Creutzfeldt-Jakob disease (CJD), as exemplary cases and systematically compiled published information on previously conducted studies, aiming at study designs, endpoints and evaluation methods employed in these rare diseases. Twelve studies in paediatric MS and seven in CJD were included in the qualitative synthesis. We extracted information on design aspects, objectives, endpoints, patient characteristics, randomization, masking, types of interventions, controls, withdrawals and statistical methodology. While studies in paediatric MS (except for one open-label RCT) were mostly observational, several RCTs have been conducted in CJD; so the type of evidence available is variable between rare disorders. Interestingly, trials in rare diseases so far mostly appear to resort to standard designs and data analysis techniques, suggesting that more sophisticated analysis tools may contribute to some progress in these fields (Unkel et al., 2016).
Task 4.2 Evidence synthesis for a single trial
Motivated by the ongoing double-blind, randomized, placebo-controlled "EARLY PRO-TECT" trial in Alport disease, we have investigated methods to support the analysis of a single clinical trial by external observational data. In the present example, external evidence may originate from registry data as well as an open-label arm. Several approaches to synthesizing the information are conceivable, e.g. pooling of data at the level of study arms, or at the level of treatment effects derived from study arms. We investigated several hierarchical modelling strategies to aid evaluation of the trial outcome once the data become available (Unkel et al., in preparation).
Quite commonly, the aim of a meta-analysis is not the derivation of an overall "average" estimate, but rather to utilize several studies to support the estimate of a particular member of the set of synthesized studies. The resulting "shrinkage estimate" constitutes a natural and flexible way of expressing the updated estimate in the light of the additional studies, and may hence be used to implement, e.g. the analysis of an RCT considering additional information, yet explicitly focusing at the trial outcome. We investigated such an approach to make use of additional phase-II data in the analysis of a phase-III study, which promises to constitute a straightforward and transparent solution to the problem (Wandel et al., 2017). It is not obvious to what extent such an approach is helpful in the extreme case of an analysis of only two studies, but our investigations have shown that a quite substantial gain in information is not unrealistic (Röver and Friede, in preparation).
Task 4.3 Borrowing strength between trials
Evidence synthesis methods are commonly implemented based on hierarchical models. A simple and very common example is given by the normal-normal hierarchical model (NNHM), which is also at the basis of a majority of published meta-analyses. Traditionally, meta-analyses are mostly performed within a frequentist framework, where in the NNHM context a number of variations of analysis methods are commonly in use, which often mostly differ in the way the heterogeneity variance component is estimated. We investigated in particular two methods to account for the heterogeneity's estimation uncertainty, and found that a previously proposed adjustment makes a substantial difference and allows for a better control of the type-I error especially for the case of few studies (Röver et al., 2015).
Frequentist meta-analysis methods often rely on asymptotic properties that only hold for large numbers of studies, and hence perform poorly in case of small numbers of studies. Bayesian methods naturally lend itself to the problem, as they provide exact results also for finite numbers of studies and because they allow for the consideration of potentially relevant a priori information. We investigated the use of Bayesian methods motivated by the canonical example of meta-analysis of studies that are summarized in terms of contingency tables and odds ratios. In simulation studies we could show that Bayesian methods using readily motivated prior settings perform well compared to common frequentist methods with respect to confidence/credibility interval coverage and interval length, especially avoiding pathological behaviour in the case of few studies (Friede et al., 2017a; Seide et al., 2017).
The joint analysis and synthesis of only two estimates to a joint result constitutes an extreme case, although it is not particularly uncommon. Common frequentist methods tend to behave pathologically, although in a Bayesian approach, given careful model and prior specifications, it does not pose a fundamental challenge. We investigated and compared the behaviour of different meta-analysis methods in this particular case and found the Bayesian approaches to perform well, especially with respect to type-I error rates and credibility interval widths (Friede et al., 2017b).
Task 4.4 Development of software
While many meta-analysis tools are implemented and readily available, the execution of a Bayesian meta-analysis usually is much more laborious and involves tailored coding of e.g. MCMC methods and diagnostic checks. For the common NNHM, we developed a method to perform the necessary calculations semi-analytically using R, interfacing with existing software, and allowing to conduct a Bayesian meta-analysis without the need to worry about technical details. The software is fully documented, extended with further functionality and examples, and is freely available on CRAN as the "bayesmeta" R package (http://cran.r-project.org/package=bayesmeta). The underlying computational method that we developed is in fact more generally applicable to evaluate mixture distributions; the approach has been presented in detail and demonstrated using several example applications (Röver and Friede, 2017).
Another way to approach the involved computations for a Bayesian meta-analysis is the use of integrated nested Laplace approximations (INLA). Building upon previous research, we implemented an R package utilizing the "r-inla" library to perform network meta-analysis using the common "design by treatment interaction" model. The model was extended to include exact binomial likelihoods in addition to the common normal likelihood, thereby enabling more accurate evaluation especially in cases of rare events, which would otherwise challenge the normal model and require potentially biasing ad-hoc model fixes. An R package ("nmainla") is currently under development and will also be published on CRAN (Günhan et al., 2017).
Publications in print or in press in peer reviewed publications:
Chataway, J. and Friede, T. (2016) The N MOmentum trial: building momentum to advance trial methodology in a rare disease. Multiple Sclerosis Journal. 22, 852-853.
Friede, T., Röver, C., Wandel, S. and Neuenschwander, B.. (2017a) Meta-analysis of few small studies in orphan diseases. Research Synthesis Methods, 8, 79-91.
Messroghli, D.R. Pickardt, T., Fischer, M., Opgen-Rhein, B., Papakostas, K., Böcker, D., Jakob, A., Khalil, M., Mueller, G.C. Schmidt, F., Kaestner, M., Udink ten Cate, F.E.A. Wagner, R., Ruf, B., Kiski, D., Wiegand, G., Degener, F., Bauer, U.M.M. Friede, T. and Schubert, S. (2017) Toward evidence-based diagnosis of myocarditis in children and adolescents: Rationale, design, and first baseline data of MYKKE, a multicenter registry and study platform. American Heart Journal, 187, 133-144.
Röver, C., Knapp, G. and Friede, T. (2015) Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Medical Research Methodology, 15, 99.
Röver, C., and Friede, T. (2017) Discrete approximation of a mixture distribution via restricted divergence. Journal of Computational and Graphical Statistics, 26, 217-222.
Unkel, S., Röver, C., Stallard, N., Benda, N., Posch, M., Zohar, S. and Friede, T. (2016) Systematic reviews in paediatric multiple sclerosis and Creutzfeldt-Jakob disease exemplify shortcomings in methods used to evaluate therapies in rare conditions. Orphanet Journal of Rare Diseases, 11, 16.
Varges, D., Manthey, H., Heinemann, U., Ponto, C., Schmitz, M., Schulz-Schaeffer, W.J. Krasnianski, A., Breithaupt, M., Fincke, F., Kramer, K., Friede, T. and. Zerr, I. (2017) Doxycycline in early CJD - double-blinded randomized phase II and observational study. Journal of Neurology, Neurosurgery and Psychiatry, 88, 119-125.
Friede, T., Röver, C., Wandel, S., Neuenschwander, B. (2017b) Meta-analysis of two studies in the presence of heterogeneity with applications in rare diseases. Biometrical Journal (in press).
Wandel, S., Neuenschwander, B., Friede, T. and Röver, C. (2017). Using phase II data for the analysis of phase III studies: an application in rare diseases. Clinical Trials (in press).
Manuscripts submitted for publication currently under review:
Günhan, B. et al. (2017) A design-by-treatment interaction model for network meta-analysis with integrated nested Laplace approximations. (Submitted for publication).
Konietschke, F., Friede, T. and Pauly, M. (2016) Robust analyses of over-dispersed counts with varying follow-up in small samples and rare diseases. (Submitted for publication).
Publications in preparation
Röver, C. and Friede, T.. Shrinkage estimation in meta-analyses of two studies.
Unkel, S., Röver, C., Gross, O. and Friede, T.. A Bayesian hierarchical framework for evidence synthesis for a single randomized controlled trial and observational data in small populations.
Seide, S., Röver, C. and Friede, T. Likelihood-based meta-analysis of few studies.
Potential Impact:
Plan for exploitation of results
We have produced exploitable foreground by the development and publication of novel methodology and production of open-source software for the implementation of methods we have developed. Exploitable methodology has been developed in four areas, with software developed in the first two of these: evidence synthesis in small studies, dose-finding in small populations, clinical trials in stratified medicine and the design of small population clinical trials using the Value of information approach.
Dissemination and exploitation activities completed during the duration of the project include publications and conference presentations in the methodological, medical and policy areas and described in more detail below.
Exploitation work is also ongoing. Additional publications are under review and in preparation including those indicated in the results section. We are also in discussion with the other two FP7 funded small populations methodology projects, Asterix (www.asterix-fp7.eu) and IDeAl, (www.ideal.rwth-aachen.de) over further joint publications to maximise the impact of the exploitable foreground from all three projects. This may include a series of papers in Orphanet Journal of Rare Diseases as well as a single summary paper in a high-ranking medical journal. Members of the consortium will also continue to present project results at methodological and medical conferences and to engage with regulators, for example through the Small-populations Clinical Trial Task Force of the International Rare Diseases Research Consortium (IRDiRC).
Dissemination and exploitation activities
Impact and dissemination and exploitation of results of the project are focussed on four groups; the academic and scientific community, the pharmaceutical industry, national and international regulatory authorities and patients and the general public.
Dissemination to the academic scientific community is primarily via publications in peer-reviewed academic journals and through presentations at national and international conferences including conferences organised as part of the project. Details of publications are given below and in the associated tables of publications. Publications arising from the project have been listed on the project website and all publications have been or will be made available on an open-access basis to maximise dissemination. As the research work on this project is primarily methodological, it has mostly been disseminated through the statistical literature and conferences, but we have also published and presented for clinical audiences when this is appropriate. Scientific publications and conference presentations are described in more detail below.
The high level of regulation in clinical trials for evaluation of novel healthcare interventions, particularly novel medicinal products, means that dissemination to and engagement with regulatory authorities is an important step in implementation of our research results in clinical trial practice following the publication of innovative methodology in scientific publications. Close links with regulatory groups, particularly the European Medicines Agency, through membership of the project team and our Independent Scientific Advisory Committee has led to engagement with regulators at a number of meetings, as described in more detail below. As anticipated, the timespan for implementation extends beyond that of the project, and this work is ongoing.
Implementation of our research results in clinical trials sponsored by the pharmaceutical industry is likely to be established alongside regulatory acceptance. To this aim, we continue to work closely with industry statisticians. Our development of open-source software to enable implementation of the novel methodology developed, as described in more detail below, will facilitate rapid exploitation of methods arising from the work of the project.
Dissemination to patients and the general public is also discussed in more detail below. Our strategy for dissemination to these groups has focussed on presentation of project aims, progress and results in an accessible format on the project website and distributed via the patient group representatives on the project Independent Scientific Advisory Committee.
Peer-reviewed publications
Publication of research results remains the primary means of dissemination for the academic scientific community, in this case both methodological and applied. A total of 21 articles have been published in peer reviewed journals or have been accepted for publication and are available online ahead of print publication and one book chapter is in press. In addition, a further 10 papers have been submitted for publication and are under review or being revised following reviewers’ comments.
Methodological publications
Scientific and technical results of the research project have been published in high quality peer reviewed methodological journals. This is often also a prerequisite for implementation of novel methods in practice as both investigators and regulators consider the peer-reviewing process to provide a necessary validation of methodology prior to its use in a clinical study.
We have 13 papers published, or accepted for publication in methodological journals. Our focus for methodological publications have been applied statistics journals. Papers have appeared or been accepted to be published in journal including Biometrical Journal, Statistical Methods in Medical Research, Journal of Biopharmaceutical Statistics, Journal of Computational and Graphical Statistics, BMC Medical Research Methodology, Research Synthesis Methods and Plos One. Particularly important methodological publications include the following:
Work package 1:
Ursino M, Zohar S, Lentz F, Alberti C, Friede T, Stallard N and Comets E. Dose-finding methods for Phase I clinical trials using pharmacokinetics in small populations. Biometrical Journal. In press. doi:10.1002/bimj.201600084
Petit C, Samson A, Morita S, Ursino M, Guedj J, Jullien V, Comets E, Zohar S, Unified approach for extrapolation and bridging of adult information in early phase dose-finding paediatric studies , Statistical Methods in Medical Research 2016, doi: 10.1177/0962280216671348
Work package 2:
Hee SW, Willis A, Smith CT, Day S, Miller F, Madan J, Posch M, Zohar S, Stallard N (2017) Does the low prevalence rate affect the sample size of interventional clinical trials of rare diseases? An analysis of data from the Aggregate Analysis of ClinicalTrials.gov. Orphanet Journal of Rare Diseases 12:44.
Stallard, N., Miller, F., Day, S., Hee, S.W. Madan, J., Zohar, S. and Posch, M. Determination of the optimal sample size for a clinical trial accounting for the population size. Biometrical Journal. In press. DOI: 10.1002/bimj.201500228.
Work package 3:
Ondra, T, Jobjörnsson, S, Beckman, R A, Burman, C-F, König, F, Stallard, N and Posch, M (2016) Optimizing trial designs for targeted therapies. PLoS One, 11: e0163726
Ondra T, Dmitrienko A, Friede T, Graf A, Miller F, Stallard N, Posch M. (2016), Methods for identification and confirmation of targeted subgroups in clinical trials: a systematic review. Journal of Biopharmaceutical Statistics, 26, 99-119
Graf, A. C., Posch, M. and Koenig, F. (2014), Adaptive designs for subpopulation analysis optimizing utility functions. Biometrical. Journal, 57, 897-913.
Work package 4:
Friede, T., Röver, C., Wandel, S., and Neuenschwander, B. Meta-analysis of few small studies in orphan diseases. Research Synthesis Methods. In press. doi: 10.1002/jrsm.1217
Friede T, Röver C , Wandel S and Neuenschwander B. Meta-analysis of two studies in the presence of heterogeneity with applications in rare diseases. Biometrical Journal. In press. doi: 10.1002/bimj.201500236
Röver, C, Knapp, G and Friede (2015), Hartung-Knapp Sidik Jonkman Approach and its modification for random-effects meta-analysis with few studies. BMC Medical Research Methodology, 15, 99.
Medical publications
Following on from methodological publications, dissemination of research outputs to the wider scientific community often involves the development and publication of retrospective case studies or prospective applications in high quality peer reviewed journals in the relevant medical field. The illustration of potential and actual applications in this way is often an important step to more widespread use of novel methodology.
We have published case studies and/or applications of methods that we have developed as part of the project in medical journals including the following:
Unkel S, Röver C, Stallard N, Benda N, Posch M, Zohar S and Friede S, (20 Feb 2016) Systematic reviews in paediatric multiple sclerosis and Creutzfeldt-Jakob disease exemplify shortcomings in methods used to evaluate therapies in rare conditions. Orphanet Journal of Rare Diseases, 201611:16.
Chataway J , Friede T (2016) The N-MOmentum trial: Building momentum to advance trial methodology in a rare disease. Multiple Sclerosis, 22: 852-853.
Varges D, Manthey H, Heinemann U, Ponto C, Schmitz M, Schulz-Schaeffer WJ, Krasnianski A, Breithaupt M, Fincke F, Kramer K, Friede T, Zerr. Doxycycline in early CJD: a double-blinded randomised phase II and observational study, Journal of Neurology, Neurosurgery and Psychiatry. In press. doi: 10.1136/jnnp-2016-313541
Petit C, Jullien V, Samson A, Guedj J, Kiechel J-R, Zohar S, Comets E (2015) Designing a Pediatric Study for an Antimalarial Drug by Using Information from Adults. Antimicrobial Agents and Chemotherapy, 60, 1481-1491.
Policy publications
In addition to the project outputs just described, we have also published papers contributing to ongoing discussion regarding the regulatory aspects of clinical trials, particularly in rare diseases. These include the following:
Koenig, F., Slattery, J., Groves, T., Lang, T., Benjamini, Y., Day, S., Bauer, P. and Posch, M. (2014), Sharing clinical trial data on patient level: Opportunities and challenges. Biometrical Journal, 57, 8-26.
Jonker, A.H. Mills, A., Lau, L.P.L. Ando, Y., Baroldi, P., Bretz, F., Burman, C.F. Collignon, O., Hamdani, M., Hemmings, R.J. Hilgers, R.D. Irony, I., Karlsson, M., Kirschner, J., Krischer, J.P. Larsson, K., Leeson-Beevers, K., Molenberghs, G., O’Connor, D., Posch, M., Roes, K.C. Schaefer, F., Scott, J., Senn, S.J. Stallard, N., Thompson, A., Torres, A., Zohar, S., Aymé, S. and Day, S. (Eds.) (2016) Small Population Clinical Trials: Challenges in the Field of Rare Diseases, International Rare Diseases Research Consortium.
These publications, particularly the latter (also under review for publication in Orphanet Journal of Rare Diseases), represent an important first step towards influence of regulatory guidelines and ultimately to improved practice in clinical trials in rare diseases and other small population groups.
National and International conference presentations
Methodology conferences
The work of the InSPiRe project has been presented in more than 50 presentations at national and international conferences on biostatistics and clinical trials methodology. Such presentations have raised the profile of the project and serve an important function in dissemination of research results widely within a rapid timescale.
Of particular note are a number of presentations we have made, or have been invited to make in the future, in sessions at methodological conferences focussing specifically on innovative approaches to the design or analysis of clinical trials in rare diseases or small population groups. These include the following national or international medical statistics meetings:
Invited paper session at IRoeS 2015, Vienna, 2015
Invited paper session at the 36th Annual Meeting of the International Society for Clinical Biostatistics, Utrecht, 2015
Forward looking forum: Clinical trials in small populations, London, 2015
Invited paper session at Trends and Innovations in Clinical Trial Statistics, Durham NC, 2016
Mini-symposium and contributed paper sessions at the 37th Annual Meeting of the International Society for Clinical Biostatistics, ISCB 37th Annual Meeting, Birmingham, 2016
Keynote address and invited paper session at AstraZeneca and Medical Research Council Biostatistics Unit Science Symposium, Cambridge, 2017
Invited paper session at CEN ISBS Joint conference on Biometrics and Biopharmaceutical Statistics, Vienna, 2017
Biometric Colloquium of the German Region of the International Biometric Society, Frankfurt, 2018
The number of such sessions gives an indication of the raising of the profile of methodological developments in rare diseases and small populations amongst the academic medical statistics community associated with the work of the InSPiRe project along with the other two FP7 funded projects Asterix and IDeAl.
Medical conferences
In addition to presenting at methodological conferences, InSPiRe team members have also presented results of the project at a number of medical conferences including the International Conference on Rare Diseases and Orphan Drugs, Mexico City, 2015, the 21st Scientific Symposium of the Austrian Pharmacological Society, APHAR, Graz, 2015 and the Swedish Orphan Biovitrum, Stockholm, 2014.
InSPiRe project conferences
In addition to InSPiRe investigators and researchers making presentations at national and international meetings as described above, the InSPiRe project team have also organised, or contributed to the organisation of, three conferences. These have provided an opportunity to showcase the work of the project, to interact with researchers working in the field and to disseminate research findings.
The first two meetings have been jointly organised by the InSPiRe project and the other two FP7 funded small populations methodology projects, Asterix (www.asterix-fp7.eu) and IDeAl, (www.ideal.rwth-aachen.de). The opportunity to collaboration with these two other projects has provided a route for dissemination on a wider scale than would have been possible for any single project alone and also enabled us to better interact at a European level with key stakeholders including regulatory, industry and patient representative groups.
Symposium on small populations, Vienna, 1-2 July 2014
Co-chaired by InSPiRe project coordinator, Nigel Stallard, Asterix project coordinator, Kit Roes, IDeAl project coordinator, Ralf-Dieter Hilgers, Martin Posch (InSPiRe and Asterix) and Franz Koenig (IDeAl), this conference provided a high-profile start to the InSPiRe, Asterix and IDeAl projects and represented a major initial interaction between the projects.
Attended by representatives of the three projects along with representatives from academia, regulatory authorities, the pharmaceutical industry and patient representative groups with an interest in the small populations research, the main conference was a 1½ day meeting. The first day included plenary sessions introducing the three EU-funded projects and focussing on the challenges of clinical trials in small populations including ethical issues, rare diseases, paediatrics, personalised medicine, data protection and patients’ perspectives, together with parallel break-out sessions for more detailed discussions. The morning of the second day included plenary presentations on regulatory issues, extrapolation, adaptive designs and surrogate markers. In addition to the main conference, participants were able to attend short courses on Surrogate marker evaluation in clinical trials, Randomisation in clinical trials or Adaptive clinical trial designs.
Seventh Framework Programme (FP7) small-population research methods projects and regulatory application workshop, EMA London, 29-30 March 2017
This meeting was jointly organised by the InSPiRe, Asterix and IDeAl projects. Held towards the end of the three projects, it represented a major opportunity for presentation and dissemination of results arising from the project as well as discussion of these results between the three project teams and with other key stakeholders.
The two-day meeting comprised five sessions, each including a series of presentations and guided and open discussion along with introductory and closing sessions with a total of 45 presenters. Session topics covered the range of work of the three projects and were Evidence synthesis, Extrapolation, Levels of evidence and decision theoretic aspects, Study endpoints and statistical analysis and Innovative designs, pharmacometrics, modelling and optimal designs. Members of the InSPiRe team were involved as speakers or discussants in all six sessions.
The meeting was open to the public and was attended by 145 participants with representatives from all major stakeholder groups including academia, regulatory authorities, pharmaceutical industry and patient representative groups. Live streaming of the meeting over the internet also enabled observation of the meeting by others unable to attend in person. It is intended to publish a summary of the meeting jointly authored by the three project coordinators together with other members of the meeting coordinating committee; preparation of a manuscript is in progress and this will be submitted to a high profile medical journal.
Engagement with regulatory authorities was a particularly important aspect of this meeting, with all sessions jointly chaired by a member of one of the three project teams and a representative from EMA or a regulatory committee.
InSPiRe project Conference on Methodology for Clinical Trials in Small Populations and Rare Diseases, Warwick, 26-28 April 2017
Towards the end of the project we organised and hosted at three-day conference at University of Warwick. The aim of this conference was presenting key results from the project and bringing together international experts in innovative clinical trials design and analysis to present recent advances in the methodology for clinical trials in small populations.
The conference was organised with seven invited plenary sessions on Applications in Paediatrics, Research in early phase dose-finding trials in small populations, Decision-theoretic approaches for clinical trials in small populations, Evidence: assessment and synthesis, Research in confirmatory trials for small populations and personalised medicines, Evidence synthesis of clinical trials in small populations and rare diseases and Ongoing work and future challenges in methodology for small population clinical trials. Each session included one or more members of the InSPiRe team presenting project results along with external speakers. External speakers included Géraldine Favrais (University Hospital of Tours), Oliver Gross (Department of Nephrology, Goettingen), Johanna van der Lee (University of Amsterdam), Lisa Hampson (AstraZeneca and University of Lancaster), Andrew Willan (University of Toronto), Christopher Jennison (University of Bath), Leo Held (University of Zurich), Heinz Schmidli (Novartis), Robert Beckman (Georgetown University), Beat Neuenschwander (Novartis), Ralf-Dieter Hilgers (Aachen University and IDeAl project coordinator) and Kit Roes (University Medical Centre, Utrecht and Asterix project coordinator).
External speakers were invited for their expertise in the area and to maximise contributions to the discussion of the work and dissemination of research and technological developments from the project.
In addition to the plenary invited talks, poster presentations were also invited, with eight poster submitted and displayed during breaks and a dedicated poster session. Posters were particularly welcome from research students, with students presenting a poster having their conference registration fee covered by the InSPiRe project. Four of the eight posters were submitted by PhD students.
As a means of dissemination of current best practice, conference delegates were also able to register to attend one of two pre-conference courses given by members of the InSPiRe team. Martin Posch and Gernot Wassmer from Medical University of Vienna gave a course on Adaptive Designs for Studies in Small Populations and Tim Friede from University Medical Centre Goettingen with Simon Wandel from Novartis gave a course on Meta-analysis of few studies. The number of participants attending the two courses were 16 and 12 respectively.
The conference was attended by 73 delegates with representatives from industry, regulatory authorities or patient groups in addition to academia.
Open access software
The availability of software is often a hurdle to the widespread adoption of novel statistical methods. To address this issue, we have produced open access statistical software to implement two of the new approaches that we have developed. We have written programs to run on the freely available software environment R (www.r-project.org) as this is open access, available on variety of platforms and widely used by medical statisticians. Software packages are available for download from the Comprehensive R Archive Network (cran.r-project.org).
We have produced the following R packages:
bayesmeta: Bayesian random effects meta-analysis
(available at http://cran.r-project.org/package=bayesmeta)
Associated with the evidence synthesis work of Work Package 4, this package provides a collection of functions to facilitate easy Bayesian inference in the generic random-effects meta-analysis model. It allows to the user to derive the posterior distribution of the two parameters (effect and heterogeneity), and provides the functionality to evaluate joint and marginal posterior probability distributions, predictive distributions, shrinkage, etc. The main functionality is provided by the bayesmeta function. This takes the data (estimates and associated standard errors) and prior information (effect and heterogeneity priors), and returns an object containing functions that allow to derive posterior quantities like joint or marginal densities, quantiles, etc. Use of the package is demonstrated in a worked-out example available on cran.r-project.org.
dfpk: A Bayesian dose-finding design using pharmacokinetics (PK) for phase I clinical trials
(available at https://cran.r-project.org/web/packages/dfpk/index.html)
Associated with the work of early phase dose-finding studies in Work Package 1, this package provides statistical methods involving PK measures in the dose allocation process during a Phase I clinical trials. These methods enter pharmacokinetics (PK) in the dose finding designs in different ways, including covariates models, dependent variable or hierarchical models. This package provides functions to generate data from several scenarios and functions to run simulations which their objectives are to determine the maximum tolerated dose (MTD) and to investigate the pharmacokinetics of a drug or a combination of drugs. The dfpk R package provides an interface to fit Bayesian generalized (non-)linear mixed models using Stan, which is a C++ package for obtaining Bayesian inference using the No-U-turn sampler (mc-stan.org/).
Engagement with stakeholders including patients and the general public
Engagement with patients and the general public
The InSPiRe project website (http://www.warwick.ac.uk/inspire) provides descriptions of the project aims and has been updated throughout the project to give details of progress and summaries of innovative methods developed in style and language appropriate for the lay reader in addition to more technical details.
Since January 2017 Tim Friede has been founding member of the Center of rare disease Göttingen (ZSEG) where several clinics and departments of the University Medical Center Göttingen collaborate together to disseminate information on rare disease, treatment possibilities, clinical research and events to patients and clinicians. We also have links with patient groups through EURORDIS, who are represented on our Independent Scientific Advisory Committee along with other patient representative groups. All members of this committee also receive regular updates on project progress. Regular face-to-face meetings of this committee provide an opportunity to discuss project progress as well as being able to comment on the research and its direction.
Details of the project have also been the subject of press releases from the University of Warwick and other project partners and have been included on http://www.news-medical.net. Along with the Asterix and IDeAl projects, the InSPiRe project has also featured in the publication Impact (https://impact.pub) raising the profile of the project among the broader scientific policy communities.
Engagement with regulatory authorities and other stakeholders
Engagement with relevant regulatory authorities at a national and international level has been an important and ongoing part of the project as we see this as an essential prerequisite for the broader uptake of novel methodology developed as part of the InSPiRe or other small populations projects. Previous and current regulatory authority and committee members are included both as part of the InSPiRe team and as members of the Independent Scientific Advisory Committee.
A major regulatory development of relevance to clinical trials in small populations during the time of the InSPiRe project has been the production of the draft EMA PDCO Reflection paper on extrapolation of efficacy and safety in paediatric medicine development. Following the 2013 EMA Concept paper, a one-day Workshop of an EMA Extrapolation expert group was held in September 2015. This EMA Extrapolation group included clinicians, pharmacologists, pharmacometricians and statisticians from the EMA, national regulatory authorities and academia. From the InSPiRe project team, Martin Posch was a presenter at this workshop and Tim Friede and Nigel Stallard participated. The output from this meeting led to the production of the draft Reflection paper in March 2016. Following the release of the draft Reflection paper, a two-day public Workshop on extrapolation of efficacy and safety in medicine development across age groups was held by EMA in May 2016. InSPiRe team members were again involved in this meeting, with Martin Posch on the Organising Committee, Tim Friede and Nigel Stallard, the latter on behalf of the InSPiRe, Asterix and IDeAl projects.
The level of interest in and commitment to the InSPiRe, Asterix and IDeAl projects by the EMA is also demonstrated by their willingness to host the joint meeting of the three projects in March 2017 as described in more detail in the section on InSPiRe project conferences above. This meeting provided excellent opportunities to continue the dialogue between regulators, academia and patient groups that has been a feature of these projects, and gives a good indication that this will continue beyond the end of the projects, leading to adoption and/or implementation of the methods developed. Martin Posch has also given a presentation on the work of the InSPiRe project to the Japanese Regulatory Authority (PMDA) in March 2015.
Along with Kit Roes and Ralf-Dieter Hilgers, coordinators of the Asterix and IDeAl projects, Nigel Stallard, the InSPiRe project coordinator, and Simon Day from the InSPiRe team were invited to join the Steering Committee of the Small-populations Clinical Trial Task Force of the International Rare Diseases Research Consortium (IRDiRC), which also include Martin Posch, Tim Friede and Sarah Zohar from the InSPiRe team. The steering group organised a two-day workshop of the task force, held at EMA in March 2016. A report of the workshop has been published on the IRDiRC website (see above) and has been submitted for publication by Orphanet Journal of Rare Diseases.
InSPiRe project website
The InSPiRe project website was created at the outset of the project and has been regularly maintained and updated throughout the duration of the project.
The website explains the focus and main aims of the project in accessible language as well as giving details of the work undertaken in each project Work Package. The website gives contact details for the project coordinator as well as details of all project partners and the Independent Scientific Advisory Committee.
A ‘News’ page includes press releases and upcoming events and dissemination activities, while our publications and details of our open source software, with links to the Comprehensive R Archive Network website where these are available, are given on a ‘Project Results’ page.
The project website also serves as the website for the main InSPiRe project conference, providing a portal for registration as well as full details of the conference.
The InSPiRe project logo as well as the EU logo and funding acknowledgement statement appear on all website pages, ensuring consistent branding is presented.
List of Websites:
The address of the project public website:
www.warwick.ac.uk/inspire
Contact details for project coordinator:
Professor Nigel Stallard
Statistics and Epidemiology
Division of Health Sciences
Warwick Medical School
University of Warwick
Coventry CV4 7AL
UK
Email: n.stallard@warwick.ac.uk
This project has brought together eight European partner organisations to address the statistical challenges of the design and analysis of clinical trials in small populations. Participants are international experts in innovative trial design methods from academia, industry and regulatory authorities. We have focussed on methods to optimise clinical trial design, improve analysis to use data from relevant sources both within and external to clinical trials and to consider levels of evidence appropriate for decision-making. Our work in four specific areas is described below.
Research in early phase dose-finding trials in small populations:
We have focussed on innovative designs for early phase clinical trials and for extrapolation from adult to paediatric studies, developing methods to enable use of pharmacokinetics and pharmacodynamics data to help determine which doses to use both in the study as it progresses and for further evaluation. Our methods enable better estimation of the dose-response curve parameters without increasing the number of patients in the study who receive sub-optimal doses. This work has led to a novel design for an ongoing clinical trial in prevention of seizures in neonates.
Research in decision-theoretic designs for clinical trials in small populations:
We have considered determination of appropriate decision-making methods for small population clinical trials. We have conducted a review of clinical trial designs currently in use in clinical trials in rare diseases as well as exploring novel methods. In particular we have explored the use of a Bayesian decision-theoretic framework to compare the costs of clinical trial evaluation with the potential benefits to current and future patients, assessing how the cost-benefit balance differs between large and small patient populations.
Research in confirmatory trials for small populations and personalised medicines:
We have developed frequentist analysis methods and decision theoretic optimal design approaches for trials that assess targeted treatments based on genetic features or other biomarkers are being developed. These trials enable the simultaneous identification of subgroups of patients for which the benefit risk balance of a treatment is positive and confirmation of treatment effect for these patients without compromising statistical or scientific integrity.
Research in use of evidence synthesis in the planning and interpretation of clinical trials in small populations:
We have developed improved robust meta-analysis methods for synthesis evidence from a small number of trials as would be common in a small population. These methods can be used to support the planning, analysis and interpretation of a trial as well as enabling extrapolation between patient groups. The novel approaches have been applied in a review of studies in sporadic Creutzfeld-Jakob disease.
Dissemination and stakeholder engagement:
Project results have been disseminated through open-access publications in high quality peer reviewed journals, with 21 papers currently published and a further others submitted, along with open-access software applications to facilitate the use of methods we have developed. Results have been presented at national and international conferences and we have organised three conferences to present our results and interact with key stakeholders, two of which have been joint meetings with the other projects, Asterix and IDeAl, funded under the same EU FP7 call. One of the latter meetings was hosted by the European Medicines Agency, facilitating engagement with regulatory authorities along with other key stakeholders including industry and patient representative groups.
Project Context and Objectives:
The HEALTH.2013.4.2-3 call identified a need for new or improved statistical methodology for clinical trials for the efficient assessment of safety and/or efficacy of treatment for small population groups.
Clinical trials in small populations present a number of novel statistical challenges associated with the need to draw inference from the necessarily small sample sizes. The EMA CHMP Guideline on Clinical Trials in Small Populations (CHMP, 2007) and the report of the US Institute of Medicine Committee on Strategies for Small-Number-Participant Clinical Research Trials (Evans and Ildstad, 2001) propose a number of potential statistical approaches. These include efficient trial designs, such as sequential and adaptive methods, and innovative methods that enable data from outside the trial to be used both at the design stage and assimilated with trial data in the final inferential process. Based on the recommendations of these guidelines and building on recent relevant methodological advances in this area, in this project we will develop novel statistical methodology for clinical trials in small populations.
We identified four specific areas where we believed novel methodology for clinical trial design was needed and achievable. These are (i) early phase dose-finding studies in small populations, (ii) decision-theoretic methods for clinical trials in small populations, (iii) confirmatory trials in small populations and personalised medicines, (iv) use of evidence synthesis in the planning and interpretation of clinical trials in small populations and rare diseases. These four areas formed the four main Work Packages, WP1, WP2, WP3 and WP4, of the project. These were augmented by two further work packages on dissemination (WP5) and project management (WP6). The objectives for each work package are given in the following paragraphs.
Work package WP1: Research in early phase dose-finding trials in small populations
Early phase dose-finding studies are the first trials of a new medicinal product in humans and aim to obtain reliable information on an appropriate dose for use in further clinical trials. Sample sizes for such studies are typically small, particularly in a small population context. Recent work has led to efficient designs for dose-finding based on the minimum number of participants. These methods have generally relied primarily on observed safety data, however, and there is a growing awareness of the need to better incorporate pharmacokinetic/pharmacodynamic (PK/PD) information in the dose-finding process. In this project we will develop novel design methods to use PK/PD data more fully. The use of such data will give a fuller picture of drug safety and efficacy during the dose-finding trial, enabling improved estimation of the best dose level for further evaluation whilst ensuring the sample size for early phase trials is kept to a minimum.
Overall objective
O.1 Propose innovative designs for early phase clinical trials taking into account safety, efficacy and pharmacokinetic/pharmacodynamic (PK/PD) measures in order to better estimate the dose level to be recommended based on limited sample sizes and subgroups.
Specific objectives
O.1.1 Develop efficient model-based designs for dose-finding studies using PK/PD information with different continuous and binary outcomes.
O.1.2 Evaluate the performance of the methods in terms of information gain, number of subjects, efficiency, and robustness.
Work package WP2: Research in decision-theoretic designs for clinical trials in small populations
Decision-theoretic methods explicitly enable evaluation of the level of evidence required from a clinical trial to best inform clinical practice. This in turn can lead to efficient and appropriate clinical trial design. In a small population setting, the small available sample size, the fact that recruitment to one clinical trial may limit recruitment to concurrent trials and the small size of the target population may mean that standard clinical trial designs, proposed in the setting of large patient populations, may be inappropriate on impossible to achieve. In this project we will develop new decision-theoretic and value-of-information methods for the design and sequential monitoring of clinical trials in small populations. Application of these methods will lead to smaller studies that are more able to lead to appropriate decision-making.
Overall objective
O.2 Develop methods for small population clinical trials based on a decision-theoretic framework.
Specific objectives
O.2.1 Obtain optimised clinical trial designs based on utility functions that account for the benefits of treatment and the population size.
O.2.2 Incorporate health economic aspects in design of trials in small populations using a value-of-information approach.
O.2.3 Determine appropriate levels of evidence for decision-making in small population clinical trials.
Work package WP3: Research in confirmatory trials for small populations and personalised medicines
Confirmatory trials are usually conducted in a large heterogeneous patient population. The development of stratified and personalised medicine means that drugs may be effective in a small population subgroup, so that such a strategy could be inefficient or lead to erroneous results. In such a setting, use of multiple testing and adaptive design strategies are promising approaches for identifying subgroups of particular interest and assessing the treatment effect in these subgroups. The project objective is to develop efficient study designs and analysis methods for the identification and confirmation of small target subpopulations as well as a clear understanding of their properties.
Overall objective
O.3 Develop frequentist and decision theoretic methods to predict patients’ responses to targeted treatments based on genetic features or other biomarkers such that subgroups of patients for which the benefit risk balance of a treatment is positive can be identified and confirmed.
Specific objectives
O.3.1 Develop frequentist methods for the identification and confirmation of subgroups where the benefit risk balance is positive.
O.3.2 Develop decision theoretic approaches for the identification and confirmation of subgroups.
O.3.3 Develop optimised adaptive enrichment designs and understanding of the potential improvement in efficiency achievable by adaptive designs compared to fixed sample designs.
Work package WP4: Research in use of evidence synthesis in the planning and interpretation of clinical trials in small populations and rare diseases
Ensuring that all available evidence is fully and appropriately synthesised and utilised is particularly important in small populations, where possible sample sizes for clinical trials may be very limited. The use of observational data from disease registries, PK/PD data from early trials and data from uncontrolled or non-randomised studies alongside randomised trial data can improve estimate precision and enable extrapolation, for example between patient subgroups. In this project we will develop novel methods for evidence synthesis, with an emphasis on methods particularly appropriate in the small population setting.
Overall objective
O.4 Develop evidence synthesis methods for small populations and rare diseases to support the planning, analysis and interpretation of a single randomized controlled trial.
Specific objectives
O.4.1 Assess feasibility and utility of the newly developed methods in small populations.
O.4.2 Apply generalized evidence synthesis approaches to paediatric studies and compounds developed for potentially multiple rare indications.
O.4.3 Provide software tools for design and analysis to facilitate application of methods developed.
Work package WP5: Research dissemination
The ultimate aim of all methodological development is the implementation of the novel methods, in this case improving the design, analysis and conduct of clinical trials in small populations. Whilst this can be achieved through the improvement of clinical trial guidelines to reflect the availability of efficient new methods, realistically this may not be achieved within the duration of this project. A key step on the path to guideline change is the peer-reviewed publication of methods and case studies in the statistical and medical literature. This disseminates the work and shows that the methods have been accepted by the scientific community. Our first objective is thus the publication of novel statistical methodology that we have developed for the design, conduct and statistical analysis and interpretation of clinical trials in small populations. This will start with publication of methodological papers in high quality international peer-reviewed statistical journals such as Statistics in Medicine or Biometrics.
We will also disseminate the results widely through relevant conferences and a project website. Involvement of clinical and regulatory experts, pharmaceutical industry statisticians and patient representatives will ensure the relevance of our work to key stakeholders. This will enable our second objective; the development and publishing of case-studies illustrating the application of methodology developed to clinical trials in small populations, which will again be published in high-quality peer-reviewed journals. This will initially be retrospective application to data from completed trials. We see this as an important step towards acceptance of the novel methodology, leading to prospective inclusion in clinical trial protocols. This will ensure that the methodology is disseminated and implemented as widely and rapidly as possible.
Overall objective
O.5 Disseminate and present innovative methodological developments from work packages.
Specific objectives
O.5.1 Build and maintain awareness of the project in interested stakeholder groups including relevant statisticians, clinicians and decision makers in academic, pharmaceutical industry and regulatory settings.
O.5.2 Organise and host a conference for dissemination and discussion of key project research results.
O.5.3 Publish project results in high-quality peer-reviewed journals, with publications made available on an open access basis when possible.
Work package WP6: Research management
In order to ensure the successful execution of the InSPiRe project, we will create and maintain appropriate and effective management structures and procedures, as described more fully in Section 2.1 below.
Overall objective
O.6 Create and maintain an organisational framework that facilitates the successful conduct of the project, guaranteeing that participants are fully integrated in the decision-making, management and delivery of the project and that financial resources are effectively managed.
Specific objectives
O.6.1 Ensure that all milestones are completed and all deliverables are delivered in time and that the consortium’s contractual duties are fulfilled.
O.6.1 Ensure effective communication between project participants and with third parties.
O.6.3 Deliver effective management of financial resources.
O.6.4 Comply with ethical requirements and collate documentation as required.
Project Results:
The aim of the InSPiRe (Innovative Methodology for Small Populations Research) project is to address the statistical challenges of the design and analysis of clinical trials in small populations. Research in these populations is challenging as the small number of patients makes conventional large clinical trials either infeasible or impossible.
A consortium of eight European partners as listed in Table 1 (attached), the InSPiRe project has addressed this challenge through four interrelated main Work Packages in three broad areas as illustrated in Figure 1 (attached).
The four main work packages are as follows:
WP1: Research in early phase dose-finding trials in small populations
WP2: Research in decision-theoretic designs for clinical trials in small populations
WP3: Research in confirmatory trials for small populations and personalised medicines
WP4: Research in use of evidence synthesis in the planning and interpretation of clinical trials in small populations
These have been supported by work packages WP5: Research Dissemination and WP6: Research Administration.
Work packages WP1, WP3 and WP4 address the challenge of small population research in two ways. Work packages WP1 and WP3 focus on improved and optimal clinical trial design. This ensures that the collection of data is as efficient as possible, enabling the most information to be obtained from the small number of patients that can be recruited into a clinical trial in a small population. The main focus of work package WP3 has been the design of clinical trials in personalised or stratified medicine, where an efficient design can both identify a subset of the population for which a new treatment gives a positive risk-benefit balance, and demonstrate the effectiveness of the treatment in that population in a scientifically and statistically rigorous way. Work packages WP1 and WP4 focus on the use of data additional to the primary endpoint data from a clinical trial that are often considered in isolation in the main trial analysis. In WP1, these data are collected from the patients in the trial. In particular, we have considered the use of data on pharmacokinetics and pharmacodynamics in early phase dose-finding studies. Such data are usually collected, but their analysis is not usually combined with that of clinical endpoints or used for ongoing design of the study. This work package has considered how these data can be used to improve study efficiency. In work package WP4, use of additional data from outside the trial is considered. These data may come from other randomised trials or from observation or registry data.
Work package WP2 takes a more general view of the design of clinical trials in small population groups, investigating how the design of such studies might reflect the size of the population under investigation. By considering the value of information obtained from a clinical trial in terms of the benefits for the population, this work package has considered how the level of evidence appropriate for decision-making may vary depending on the population size.
The results from each of these four work packages are described in detail in the remainder of this section.
The InSPiRe project team have worked closely with the two other projects funded under the same EU FP7 call; the Asterix and IDeAl projects, with some work being undertaken jointly by more than one project. A particularly strong collaboration between the IDeAl and InSPiRe projects associated with the work of InSPiRe WP3. A summary of the work planned in all three projects has been published in the Orphanet Journal of Rare Diseases (Hilgers et al., 2016).
Work Package 1: Research in early phase dose-finding trials in small populations
Work package 1 has focussed on the development of innovative design for early phase dose-finding trials in small population. Early phase dose-finding studies are the first trials of a new medicinal product or a combination procedure in humans and aim to obtain reliable information on an appropriate dose for use in further clinical trials. Sample sizes for such studies are typically small, and this is particularly true in a small population context. Recent work in this field has led to efficient designs for dose-finding based on the minimum number of participants. These methods have generally relied primarily on observed safety data, however, and there is a growing awareness of the need to better incorporate PK/PD information. Our methods can be used to better estimate the dose level to be recommended allowing for small population characteristics and prior knowledge of the drug together with innovative use of PK/PD and other external data.
Our objective in WP1 was to include all available information to better design the dose-finding trial and to better estimate the recommended dose level for further evaluation whilst ensuring the sample size for early phase trials is kept to a minimum. This involved (i) the incorporation of the pharmacokinetic/pharmacodynamic (PK/PD) information in the dose-allocation process, (ii) the addition of multiple constraints on cumulative toxicities, (iii) the development of an innovative design for the LEVENEONAT clinical trial for reducing neonatal seizures, (iv) the development of new methods for bridging studies from adults to children and (v) the proposal of a new way of eliciting expert information.
Task 1.1 Combining PK data for optimal dose-finding with categorical PD outcome
We began our work by performing a literature review on already existing methods which combine PK or PK/PD data with dose-finding. Since only few methods with both PK and PD were found, we decide to start our work by including PK data only in the dose allocation of early phase clinical trials.
The aim of phase I clinical trials is to obtain reliable information on safety, tolerability, PK, and mechanism of action of drugs with the objective of determining the maximum tolerated dose (MTD). In most phase I studies, dose-finding and PK analysis are analysed separately and no attempt is made to combine them during dose allocation. In cases such as rare diseases, paediatrics, and studies in a biomarker-defined subgroup of a larger population, the available population size will limit the number of possible clinical trials that can be conducted. Combining dose-finding and PK analyses to allow better estimation of the dose-toxicity curve should then be considered. In this work, we proposed, studied, and compared methods to incorporate PK measures in the dose allocation process during a phase I clinical trial. These methods approach this in different ways, such as using PK observations as a covariate, as the dependent variable or in a hierarchical model. We conducted a large simulation study that showed that adding PK measurements as a covariate only does not improve the efficiency of dose-finding trials either in terms of the number of observed dose limiting toxicities or the probability of correct dose selection. However, incorporating PK measures through a hierarchical model does allow better estimation of the dose-toxicity curve while maintaining the performance in terms of MTD selection compared to dose-finding designs that do not incorporate PK information. In conclusion, we found that using PK information in the dose allocation process enriches the knowledge of the dose-toxicity relationship, facilitating better dose recommendation for subsequent trials.
This work has been accepted for publication by Biometrical Journal and is currently available online ahead of print (Ursino et al., DOI: 10.1002/bimj.201600084).
An R package called “dfpk” was developed to provide a practical implementation of the methods presented in the previous work. The package provides, for each method, a function (nextDose) to estimate the probability of toxicity and to suggest the dose to give to the next cohort, and a function to run trial simulations before starting the trial (nsim). Functions to generate data-sets and to display the results graphically are also provided. The package dfpk can be downloaded freely from the CRAN repository (https://cran.r-project.org/web/packages/dfpk/index.html). A manuscript has been submitted for publication (Toumazi et al., 2017) and is currently under review.
During the first year, there was the possibility to set a collaboration with the Columbia University (NY) in order to develop a novel methodology on including grade 2 toxicity information in the dose-finding design. Even if PK was not introduced in this work, this was considered to be a good idea for the WP1, since we focused on including the maximum available information in the design. Therefore, we developed a dose-finding design where multiple constraints are added on cumulative toxicities. The toxicity profile of newer anticancer treatments such as targeted and immunotherapeutic agents differs from that of chemotherapy. While some of these newer agents cause dose limiting toxicities, others cause lower grade toxicities which may not occur within the first cycle of treatment. Thus, in the early development of these agents it is necessary to account for both lower grade toxicity, as well as, late onset and cumulative toxicity. This new design addressed the concern regarding late-onset dose-limiting toxicities (DLT), moderate toxicities below the threshold of a DLT, and cumulative toxicities that may lead to a DLT, which are either disregarded or handled in an ad-hoc manner in cancer trials. An extension of the Time-to-Event Continual Reassessment Method (TITE-CRM) which allows for multiple constraints, and can account for partial information on both DLT and moderate toxicities was proposed. The method was illustrated in the context of an Erlotinib dose-finding trial with low DLT rates, but a significant number moderate toxicities that led to treatment discontinuation in later cycles. Based on simulations, our method performed well at selecting the dose that satisfies both the DLT and moderate toxicity constraints when the true MTD for the individual constraints do not coincide, and performs better than the TITE-CRM when the true MTDs coincide, by reducing the probability of overdose while having similar probability of correct selection. This work has been submitted for publication (Lee et al., 2017) and is currently under review.
Task 1.2 Using continuous PD data
During the project, we had the opportunity to help designing a new clinical trial in a paediatric context, the LEVNEONAT (NCT 02229123). The aim of the trial is to find the optimal dose of Levetiracetam for reducing neonatal seizures. The LEVNEONAT clinical trial follows the NEMO study (NEonatal seizures with Medication Off-patent, NCT 01434225, FP7-HEALTH-2009-4.2-1 grant agreement 241479, The NEMO Project), which was stopped after that unexpected safety events were measured at long term. The LEVNEONAT was planned for the same indication but with another drug. A novel approach was needed to respond to requirements based on what was observed in NEMO study. Clinical trials in vulnerable populations, such as newborns, are extremely difficult to conduct and this was considered to be a good case-study for the approaches developed in the WP1 package. It gave us the opportunity to evaluate the impact of using a continuous regression for efficacy, at the end of the trial, instead of the usual dichotomised variable.
The LEVNEONAT is a phase I/II trial aiming at finding the recommended dose of Levetiracetam for treating neonatal seizures and it was planned with a maximum sample size of 50. In the trial, initially 4 doses were considered with 3 primary outcomes: efficacy and two types of toxicity that occur at the same time but measured earlier or later in time. In the case of failure, physicians could add a second agent as a rescue medication, which may differ from centre to centre. The primary outcomes were modelled via a logistic model for efficacy and a weighted likelihood with pseudo outcomes for the two toxicities taking into account the dependences under Bayesian inference. Pseudo toxicity outcomes were introduced to consider the possible effect of the second agent. Finally, this model permits sequential patient accrual. Dose escalation rules were based on an adaptive threshold for posterior probabilities, considering only early toxicity in the start-up phase. Simulations were conducted to assess the design properties and the correct dose was recommended more than 60% of the time on average. A manuscript on this work has been submitted for publication and is currently under review (Ursino et al, 2017a). The trial got the regulatory and ethical committee agreement and inclusion should start in September 2017.
In parallel, we extended the work on combining PK data in the dose allocation of early phase clinical trials started in Task 1.1. The extension deals with combining full PK/PD analysis in the dose escalation process in Phase I/II. We started to focus on trials where the toxicity outcome is defined as the area under the curve (AUC) above a specified threshold and the efficacy outcome is defined as the percentage of inhibition of a biomarker. The right recommended dose is defined as the one with the highest probability of efficacy without toxicity, under marginal toxicity and efficacy constraints. We continued to use a Bayesian design which allowed us to add prior information on parameters and to run estimation also in presence of very limited sample size at the beginning of the trial. We proposed a two stage design: in the first stage, the bivariate continual reassessment method is used to guide the dose escalation process, while in the second stage, the full PK/PD model is used to compute posterior probabilities to guide the dose allocation. We conducted a large simulation study that showed that the two stage model is less sensitive to prior misspecification; it is also more conservative, reducing overdosing while increasing the percentage of the right selected dose. We started to adapt the design for Phase I/II studies where the probability of toxicity and efficacy depends on the AUC and the percentage of inhibition, respectively. A paper is in preparation to report this work.
Task 1.3 Combining data from different sources
Task 1.3 focussed on bridging studies, in particular on (1) how to use previous information to set up prior distributions and better design PK, (2) and phase I/II studies for children when PK and toxicity data are available from adults; (3) and on building prior distributions using elicited expert opinion. Parts (1) and (2) were done in collaboration with IDEX (Paris Cité Sorbonne) project.
Looking at bridging studies, we started by focussing on how to design PK studies in paediatric population using adult data and then we extended the work to early phase dose-finding trials.
Regarding PK, the objectives of the study were to design a pharmacokinetic study by using information about adults and evaluate the robustness of the recommended design through a case study, the antimalarial drug mefloquine. Combining PK modelling, extrapolation, and design optimisation led to a design for children with five sampling times. PK parameters were well estimated by this design with small relative standard errors. Although the extrapolated model did not predict the observed mefloquine concentrations in children very accurately, the design optimised based on the extrapolated model allowed precise and unbiased estimates across various model assumptions, contrary to the empirical design. In conclusion, using information from adult studies combined with allometry and maturation can help provide robust designs for paediatric studies.
The resulting paper has been published in Antimicrobial Agents Chemotherapy and is available online as an open access publication (Petit et al., Statistical Methods in Medical Research, 2016a).
This first work shows that even imperfect priors can be used to improve the design of clinical studies in children. The next step is figuring how to account for these priors in dose-finding studies. We propose a unified approach for extrapolation and bridging adult information in early phase dose-finding studies in paediatrics. We investigated calibrating an existing dose-finding model for a paediatric population using adult observations, such as PK, toxicity and efficacy, in order to choose the parameters of the dose-finding model, i.e. the initial guess of toxicity for each dose (working model) and the prior distribution of parameters, and the dose-range given to the paediatric population in the study. We evaluated the resulting method on a drug used against cancer, erlotinib.
This work has been accepted for publication by Statistical Methods in Medical Research and is currently available online ahead of appearing in the printed journal (Petit et al., DOI: 10.1177/0962280216671348).
We have also performed a systematic review on dose-finding research in paediatrics from January 1st 1996 to October 30th 2016 in order to evaluate the use of extrapolation in paediatric designs and to propose related methodologies. We included (1) publications from January 1st 1996 to October 30th 2016 with (2) a paediatric population aged 0 to 12 years old and (3) a drug administration of at least two or more doses, associated with or without another drug treatment. We excluded adult population and unrelated studies. A total number of 167 publications were selected and reviewed. Among them, 119 publications reported an adaptive sequential dose-finding design while 48 studies used a randomisation procedure. Extrapolation was used for the choice of doses in 34% of the publications and in one study for choices related to the dose-finding design. Among them, the extrapolation rule is predominantly made according to an arbitrary rule. This systematic review suggested that the use of an extrapolation criteria is rare and model-based and/or simulations are rarely used for designing a paediatric dose-finding study. A draft of the paper is in preparation.
During the project, we were involved in the analysis of a 70-patients randomized trial to compare two treatments for idiopathic nephrotic syndrome, a rare disease in children.
We proposed a Bayesian methodology for constructing a parametric prior on two treatment effect parameters, based on graphical information elicited from a group of expert physicians. The methodology relies on histograms of the treatment parameters constructed manually by each physician, applying the method of Johnson, et al. (2010). For each physician, a marginal prior for each treatment parameter characterized by location and precision hyper-parameters is fit to the elicited histogram. A bivariate prior is obtained by averaging the marginals over a latent physician effect distribution. An overall prior is constructed as a mixture of the individual physicians’ priors. A simulation study evaluating several versions of the methodology is presented. A framework is given for performing a sensitivity analysis of posterior inferences to prior location and precision, and is illustrated based on the idiopathic nephrotic syndrome trial.
A manuscript on this work has been submitted for publication and is currently under minor revision in Statistical Methods in Medical Research (Thall et al, 2017).
Publications in print or in press in peer reviewed publications (all are available as open access publications):
Ursino, M., Zohar, S., Lentz, F., Alberti, C., Friede, T., Stallard, N. and Comets, E. (2017). Dose-finding methods for Phase I clinical trials using pharmacokinetics in small populations. Biometrical Journal. In press. DOI:10.1002/bimj.201600084
Petit, C., Jullien, V., Samson, A., Guedj, J., Kiechel, J. R., Zohar, S., and Comets, E. (2016a). Designing a Pediatric Study for an Antimalarial Drug by Using Information from Adults. Antimicrobial agents and chemotherapy, 60(3), 1481-1491.
Petit, C., Samson, A., Morita, S., Ursino, M., Guedj, J., Jullien, V., ... and Zohar, S. (2016b). Unified approach for extrapolation and bridging of adult information in early-phase dose-finding paediatric studies. Statistical Methods in Medical Research. In press. DOI: 10.1177/0962280216671348.
Manuscripts submitted for publication in peer reviewed journals currently under review:
Toumazi, A., Comets, E., Alberti, C., Friede, T., Lentz, F., Stallard, N., Zohar, S. and Ursino, M. (2017). An R-package for Bayesian Dose-Finding Design using Pharmacokinetics (PK) for Phase I Clinical Trials.
Shing, L.M. Ursino, M., Cheung, Y.K. and Zohar, S. (2017). Dose-finding designs for cumulative toxicities using multiple constraints.
Ursino, M., Yuan, Y., Alberti, C., Comets, E., Favrais, G., Friede, T., Lentz, F., Stallard, N., and Zohar, S. (2017a). A Dose-finding Design for Seizure Reduction in Neonates.
Thall, P.F. Ursino, M., Baudouin, V., Alberti, C. and Zohar, S. (2017). Bayesian Treatment
Comparison Using Parametric Mixture Priors Computed from Elicited Histograms.
Work Package 2: Research in decision-theoretic designs for clinical trials in small populations
Work package 2 has focussed on the development of decision-theoretic designs for clinical trials in small populations. These methods explicitly enable evaluation of the level of evidence required from a clinical trial to best inform clinical practice enabling efficient clinical trial design. We have also built on the health economic value of information approach, enabling an assessment of the appropriate level of information required for clinical decision-making in the small population setting.
Most clinical trials are designed with no reference to the size of the population in which the research is conducted. Whilst this may be reasonable in a large population, in rare diseases or other small populations it can lead to designs that are inappropriate. If the population under investigation is small, a large proportion of the patient group may be recruited to a clinical trial. Recruitment to one trial may thus have an impact on the conduct of other trials or even reduce significantly the size of the population receiving usual care. The value of one clinical trial must therefore be compared with that of other trials if research is to proceed efficiently. Such a comparison, and resulting appropriate trial design, is enabled through the decision-theoretic methods that we have developed.
Task 2.1 Review of relevant literature
In order to provide an overview of the current state of the art and to establish the context for future research work, the initial work of Work Package 2, Task 2.1 was completion of a review of literature on the use of decision-theoretic approaches in clinical trial design.
After removal of duplicates, a total of 8,326 potentially relevant articles were identified, with 10 further articles identified through reference screening. Screening of article titles, abstracts and, where considered necessary, the whole paper, 67 articles were found to propose decision-theoretic design methods relevant to small clinical trials. The review discusses these 67 articles in detail, classifying them according to the type of study design proposed (single-arm single-stage, single-arm multi- stage, two-arm single-stage, two-arm multi-stage, multi-arm, series of trials and enrichment design) and the type of gain function proposed (simple or more realistic utility based on patient, societal, commercial or unspecified perspective).
The review has been published in Statistical Methods in Medical Research and is available online as an open access publication (Hee et al., Statistical Methods in Medical Research, 2016).
Task 2.2a Development of decision theoretic framework
Task 2.2a focussed on the development of a decision-theoretic model for the clinical evaluation process that incorporates gains both to patients within a trial who receive effective treatment and to future patients receiving either a new treatment or the current standard treatment depending on the conclusion reached at the end of a clinical trial. Based on a model proposed by Cheng et al (Biometrika 2003), attention has focused in particular on how the optimal sample size for a clinical trial depends on the size of the patient population.
Taking a general viewpoint, we have shown that for a wide range of response distributions, including responses with normal, exponential, Bernoulli and Poisson distributions, and gain function forms, the optimal trial sample size is proportional to the square root of the population size, with constant of proportionality depending on the gain function form and prior distribution of the parameters of the distribution of the data. This has important consequences in terms of optimal design as it challenges the usual method of sample size determination based on frequentist error rates to show that in a small population setting a smaller trial than usual may be optimal.
This work has been accepted for publication by Biometrical Journal and is currently available online ahead of appearing in the printed journal (Stallard et al., DOI: 10.1002/bimj.201500228).
In order to determine how our theoretical work relates to current practice in small clinical trials, we have conducted an analysis of trials in rare diseases recorded in the ClinicalTrials.gov database. The aim was to use disease prevalence data from Orphadata to identify trials registered in ClinicalTrials.gov that studied a rare disease and to investigate how the sample sizes for these trials depended on the disease prevalence and on other factors including the gender and age of patients recruited to the trial, trial phase, trial design and trial date. Evaluation of all clinical studies registered on ClinicalTrials.gov up to 27 September 2015 indicated that of the 186,941 trials registered, 1,567 studied a single rare condition with prevalence information on Orphadata. Of these, there were 19 (1.2%) trials studying disease with prevalence <1/1,000,000, 126 (8.0%) trials with 1-9/1,000,000, 791 (50.5%) trials with 1-9/100,000 and 631 (40.3%) trials with 1-5/10,000. Of these, 1160 (74%) were phase 2 trials. Our study found that the sample sizes in phase 2 trials were similar across all prevalence classes after adjusting for other covariates; mean, 15.7 (95% CI, 8.7–28.1) 26.2 (16.1–42.6) 33.8 (22.1–51.7) and 35.6 (23.3–54.3) for prevalence <1/1,000,000, 1-9/1,000,000, 1-9/100,000 and 1-5/10,000, respectively. Estimated size of phase 3 trials of rarer diseases, <1/1,000,000 (19.2 6.9–53.2) and 1-9/1,000,000 (33.1 18.6–58.9) were similar to those in phase 2 but were statistically significant lower than the slightly less rare diseases, 1-9/100,000 (75.3 48.2–117.6) and 1-5/10,000 (77.7 49.6–121.8) trials. This work has been published in Orphanet Journal of Rare Diseases (Hee et al., 2017a) and is currently under review.
We have also developed decision-theoretic methods for the simultaneous design of a series of trials. This work has focussed on optimal design and decision-making in a setting in which a number of treatments are to be evaluated in a small fixed population. This extends the work described above through the use of a joint prior distribution for effects of different treatments that are correlated provides gains in efficiency. Use of the methodology has been illustrated through retrospective application in an example in orthopaedic surgery. A manuscript on this work has been accepted for publication in Biometrical Journal (Hee et al, 2017b).
Task 2.2b Construction of an approximate utility function
Building on the theoretical work of Task 2.2a we have explored the application of the methodology through the development of a number of specific case-studies of clinical trials in rare diseases.
The aim of the work is to explore the application of methods developed in a more general context in Task 2.1 and to compare this with other approaches for the design of clinical trials, particularly in the rare disease setting.
We consider three case studies; Lyell’s disease, adult-onset Still’s disease and cystic fibrosis. In each case the aim to plan the sample size for an upcoming study. Three alternative sample size approaches are described: traditional sample size calculation based on power to show a statistically significant effect, sample size calculation based on assurance and the decision theoretic approach based on the published work from Task 2.2a. The latter has the property that trial sample size depends on the size of the disease population which is interesting in the context of rare diseases. We outline in detail the reasonable choice of parameters for these approaches for each of the three case studies and calculate sample sizes. We stress that the influence of the input parameters need to be investigated in all approaches and recommend investigating different sample size approaches before deciding finally on the sample size.
A manuscript has been submitted for publication (Miller et al, 2017) and is currently under review.
Task 2.3 Development of value-of-information approaches
Building on the work of Task 2.2a we have developed methodology for the use of a decision-theoretic value of information method for a confirmatory phase III trial, particularly in the small population setting. In particular, we determine the optimal significance level that should be used if a frequentist hypothesis test is used for decision-making at the end of the trial, and investigate how this significance level changes with the size of the disease population. We show how decision-theoretic VOI analysis suggests a more flexible approach with both type I error rate and power (or equivalently trial sample size) depending on the size of the future population for whom the treatment under investigation is intended.
We focus on a primary endpoint assumed to be continuous and normally distributed with unknown mean with some normal prior distribution, the latter representing information on the anticipated effectiveness of the therapy available from sources external to the trial itself. We explicitly specify the gain in terms of improvement in primary outcome for patients treated with the new therapy and compared this with the costs, both financial and in terms of risk of potential harm, of treating patients, either in the trial or in the future if the therapy is approved.
In line with our previous work, we find that as the size of the population increases, the optimal sample size for the clinical trial also increases. For non-zero cost of treating future patients, either monetary or potential harmful effects, a more stringent significance level, that is stronger evidence, is required for approval as the population size increases, though this is not the case if the costs of treating future patients are ignored.
A manuscript describing the work and also a retrospective application in a case study, has been submitted to a peer reviewed publication and is currently under review.
Task 2.4. Consideration of levels of evidence appropriate for decision-making in small populations
Building on Tasks 2.2 and 2.3 we have extended the models developed there in three ways. First, we relax the assumption that the population size is known. Rather we assume that we can specify a prior distribution for the incidence of new disease cases, and that we learn about this prevalence through the rate at which patients enter the trial. As the incidence of the disease is considered unknown, the size of the trial is specified not in terms of the number of patients to be recruited, but the duration in time over which recruitment will take place, so that the optimal length of trial is obtained as a proportion of patient horizon specified in terms of the total time that the new treatment might be considered to be in use. The second extension is to consider a two-stage trial, in which a decision is made at an interim analysis as to whether to stop the trial, either to recommend the new treatment or to abandon development, or to continue with a second stage of the trial prior to making a final decision. If the trial continues, the size of the second stage can be chosen optimally based on the first stage data. A manuscript describing this work is in preparation and will be submitted for publication.
The third extension to the model developed in Tasks 2.2 and 2.3 is to extends the utility function formulation to the setting in which we consider two decision-makers; the “sponsor” responsible for decisions regarding the design and conduct of the trial and “society”, who can make decisions concerning reimbursement offered to the sponsor on the basis of potential benefit from an effective treatment. We explore the consequences of this formulation of the decision-making problem, both in settings in which the sponsor and society each seek to maximise their own benefit and when they can cooperate. Again, a manuscript to be submitted for publication is in preparation.
Publications in print or in press in peer reviewed journals:
Hee, S.W. Hamborg, T., Day, S., Madan, J., Miller, F., Posch, M., Zohar, S. and Stallard, N. (2016) Decision theoretic designs for small trials and pilot studies: a review. Statistical Methods in Medical Research, 25, 1022-1038.
Kunz CU, Stallard N, Parsons N, Todd S, Friede T (2016) Blinded versus unblinded estimation of a correlation coefficient to inform interim design adaptations. Biometrical Journal 2, 344–357.
Hee, S.W. Willis, A., Tudur-Smith, C., Day, S., Miller, F., Madan, J., Posch, M., Zohar, S. and Stallard, N. (2017a) Does the low prevalence rate affect the sample size of interventional clinical trials of rare diseases? An analysis of data from the Aggregate Analysis of ClinicalTrials.gov. Orphanet Journal of Rare Diseases, 12, 44.
Stallard, N., Miller, F., Day, S., Hee, S.W. Madan, J., Zohar, S. and Posch, M. Determination of the optimal sample size for a clinical trial accounting for the population size. Biometrical Journal. In press. DOI: 10.1002/bimj.201500228.
Hee, S.W. Parsons, N. and Stallard, N. (2017b) Designing a series of decision-theoretic phase II trials when treatment effects are correlated with the Sarmanov multivariate beta-binomial distribution. Biometrical Journal. In press.
Manuscripts submitted for publication in peer reviewed journals currently under review:
Miller, F., Day, S., Hee, S.W. Madan, J., Pearce, M., Posch, M., Stallard, N., Vågerö, M. and Zohar, S. (2017) Approaches to sample size calculation for clinical trials in small populations.
Pearce, M., Hee, S.W. Madan, J., Posch, M., Day, S., Miller, F., Zohar, S. and Stallard, N. Value of information methods to design a clinical trial in a small population to optimize a health economic utility function.
Work Package 3: Research in confirmatory trials for small populations and personalised medicines
In work package 3 frequentist and decision theoretic methods were developed in order to predict patients’ responses to targeted treatments based on genetic features or other biomarkers such that subgroups of patients for which the benefit risk balance of a treatment is positive can be identified and confirmed.
Task 3.1. Literature review
In a first step we conducted a literature review describing the current trial designs and statistical approaches for the development of targeted therapies. These methods allow for the identification and confirmation of subpopulations of patients where the treatment under investigation is effective and has a positive benefit-risk balance. The identified methods were classified based on characteristics of the proposed trial designs and analysis methods. In particular we distinguished exploratory and confirmatory subgroup analysis and frequentist, Bayesian and decision theoretic approaches. Furthermore we reported on fixed-sample, group-sequential, and adaptive designs and illustrated the available trial designs and analysis strategies with published case studies.
The literature search was based on PubMed and was restricted to methodological journals. We first conducted a keyword search to identify an initial set of articles and then screened the references listed in these papers. Furthermore, relevant literature identified by manual searches was added to the review. In total 86 papers were classified as relevant and included in the survey.
The review has been published in the Journal of Biopharmaceutical Statistics and is available online as open access publication (Ondra et al., Journal of Biopharmaceutical Statistics, 2016). The publication is listed among the top 3 most read papers of the Journal of Biopharmaceutical Statistics on the journal’s website .
Task 3.2 Development and assessment of frequentist methods and Task 3.3.
Development of decision-theoretic methods for subgroup identification
Frequentist and decision theoretic methods were addressed in several steps. On the one hand we focused on the development of confirmatory hypothesis tests that control frequentist error rates in complex trials where several subgroups are investigated. On the other hand we used decision theoretic methods to optimize trial designs that control frequentist false positive rates as required by regulatory authorities. Thus, Tasks 3.3 was addressed combined with Task 3.2.
Development of Frequentist designs. We developed frequentist fixed sample trial designs in a setting where a continuous biomarker is used to define subgroups. Such continuous biomarkers can be either quantitative measurements or biomarker signature scores that summarize the measurement of several individual biomarkers. The continuous biomarker is dichotomized based on a threshold to define biomarker-low and biomarker-high subpopulations. If there is insufficient information on the choice of the threshold, several thresholds are investigated and separate hypothesis test for each of the subgroups can be performed. This causes a multiple testing problem and an appropriate adjustment for multiple testing has to be applied. Because the nested structure of subpopulations defined by different thresholds for a continuous biomarker is similar to the structure of analysis populations in group sequential trials (Jennison and Turnbull, 1999), it has been proposed to use critical boundaries of group sequential designs to test such nested subpopulations (Spiessens and Debois, 2010). We showed that special care has to be taken when applying such group sequential boundaries to test hypothesis for multiple nested subpopulations. Especially, we show that the false positive rate of such tests may be inflated, if the biomarker has a prognostic effect (i.e. it is correlated with the prognosis of the patient but not the treatment effect). In a simulation study we quantify the potential inflation of the type I error rate of testing procedures based on group sequential designs. Furthermore, we propose improved hypotheses testing approaches based on regression models and combination tests and demonstrate that they control the family-wise error rate also in situations where the treatment is not effective but the biomarker is prognostic. A further manuscript, focusing on the design of multiple testing procedures for nested subgroups, that also investigates the case where a continuum of thresholds is considered, is under preparation and will be completed after the project end.
Decision Theoretic Methods to Optimize Frequentist Trial Designs. We investigated the potential of Bayesian decision theoretic methods to optimize single stage designs as well as adaptive designs (the latter are addressed in Task 3.4). While in standard parallel group clinical trials the statistical power to demonstrate a treatment effect is typically the basis for the planning of clinical trials, the consideration of power alone does not sufficiently represent the losses and gains of correct and incorrect test decisions when several subgroups are tested. Subgroup analyses are challenging because several types of risks are associated with inference on subgroups. On the one hand, by disregarding a relevant subpopulation a treatment option may be missed due to a dilution of the treatment effect in the full population. Furthermore, even if the diluted treatment effect can be demonstrated in an overall population, it is not ethical to treat patients that do not benefit from the treatment when they can be identified in advance. On the other hand, selecting a spurious subpopulation increases the risk to restrict an efficacious treatment to a too narrow fraction of a potential benefiting population. In order to quantify these risks utility functions are proposed, representing the benefit of a particular clinical trial from a sponsors and a public health perspective. To quantify the risks and losses in these settings, we derived utility functions that take into account the size of the population for which a claim is made and for which a novel treatment becomes available. We define utilities from the perspective of different stakeholders: the sponsor’s view as well as a public health view. We assume that the utility of the sponsor is the net present value (NPV), while for the public health it is the expected health benefit (adjusted for the cost of the trial). In the planning phase expected utilities for different trial designs and different utility functions can be computed based on a Bayesian approach which relies on a prior distribution on the effect sizes in the subgroup and the full population.
In three articles we investigated optimized single stage designs, focusing on different aspects of the testing procedure: optimizing the sample size, the multiple testing procedure (especially the weights used in the multiple test) and the type of the design. Especially, we considered the classical design, where no biomarker information is used and only the full population is tested (such that no cost for biomarker development and determination of the biomarker status of patients incur), the enrichment design, where only biomarker positive patients are included, a stratified design, where patients from the full population are included and the treatment effect is tested in the subgroup and the full population. In the latter, no biomarker based patient selection takes place such that the prevalence of the subgroups in the trial is the same as the prevalence in the underlying population. Finally, we considered partial enrichment designs, where the prevalence of the subgroup in the trial is a design parameter that can be optimally chosen to maximize the expected utility. The models have been formulated for parallel group designs but can equally be applied to cross-over trials.
We find that the optimal trial designs depend sensitively on the considered scenario, especially on the prevalence of the subgroup, the strength of the prior evidence that the treatment is only effective in the subgroup, as well as on the cost of the biomarker development and determination. Furthermore, we observe that optimal designs for the sponsor and the public health view differ. Trials optimized under the sponsor view tend to have smaller sample sizes and are conducted in the full population also in settings where there is strong evidence that the treatment is effective in the subpopulation only. This is due to the fact that due to the variability of treatment effect estimates a treatment might appear effective in a subpopulation (and bring a gain for the sponsor) even if it is not effective and has no benefit for patients.
The work has been published in articles in Biometrical Journal (Graf et al., 2015) and Plos One (Ondra et al., 2016). The article Graf et al. (2015) is the second most cited article of Biometrical Journal of all articles published in 2015 and 2016 : A further manuscript has been submitted to Statistical Methods in Medical Research (Ondra et al. 2017) .The work on the articles Ondra et al. (2016,2017 ) was performed in cooperation with researchers from the EU-FP7-IDEAL project (Carl-Frederic Burman and Sebastian Jobjörnsson) and Robert A Beckman (Georgetown University) who is co-chair of the small population working stream of the DIA.
Task 3.4. Adaptive enrichment trials
Frequentist adaptive designs. A comprehensive description of the statistical methodology for adaptive frequentist designs in confirmatory trials with multiple objectives has been developed. This work focuses on two-stage designs, i.e. trials with one interim analysis, and settings where multiple subgroups or treatment arms are investigated. We describe the application of adaptive combination tests that guarantee control of the type I error rate in a wide range of scenarios. Especially, the application of the closure principle in the context of adaptive two stage designs is demonstrated. This methodology is the basis for the implementation of adaptive enrichment designs. In these two stage designs, in the first stage patients are recruited from the full population. In an interim analysis, based on the interim data, the trial design of the second stage may be modified. For example, recruitment may be limited to patients in a subgroup of biomarker positive patients and/or the sample sizes in the subgroups may be adapted. The adaptation may be based on all information observed at the interim analysis, including information on secondary and surrogate endpoints and safety information.
The application for different endpoints is discussed, including normally distributed, binary and survival endpoints. In the last section, some extensions and applications of the designs are described and implications for the regulatory assessment of such designs are discussed.
This work will appear as an invited book chapter “Adaptive Designs with Multiple Objectives” in the “Handbook of Statistical Methods for Randomized Controlled Trials" (CRC Press).
Furthermore, for the special case of adaptive designs with a survival endpoint we developed hypothesis tests that allow for early rejections of the null hypothesis in an interim analysis. This work generalizes earlier adaptive procedures that control the familywise type I error rate in the strong sense but have limitations: they either cannot use information from surrogate endpoints for adaptive decision making or do not allow for early rejections in an interim analysis. We derived hypothesis tests for multi-armed and enrichment designs which do not have these restrictions and can be used within a multi-stage group sequential design.
This work is included in Deliverable 3.3 and is complemented by a simulation study in cooperation with the EU-FP7 project Asterix. The work is presented in a topic contributed session at the Joint Statistical Meeting 2017, Baltimore, USA. The research is performed in cooperation with Silke Jörgens (Icon Clinical Research Inc) as external partner.
Development of decision theoretic methods to optimize frequentist adaptive designs. In the articles Graf et al. (2015) and Ondra et al., 2017 (see above), we also addressed the optimization of frequentist adaptive two stage enrichment designs based on a decision theoretic approach. Using extensive simulations (Graf et al., 2016) and a dynamic programming algorithm (Ondra et al. 2017) we optimize interim adaptation rules maximizing utility functions. We show that adaptive enrichment designs can lead to a higher expected utility as compared to single stage designs, especially in settings where there is high uncertainty if the treatment is effective only in a subgroup. As for single stage designs, we observe differences in the optimized sample sizes if trials are optimized under the sponsor or the public health perspective. An important advantage of adaptive designs compared to single stage designs are their increased robustness with regard to a misspecification of the planning assumptions.
Publications in print or in press in peer reviewed publications:
Graf, A., Posch, M. and König, F. (2015). Adaptive designs for subpopulation analysis optimizing utility functions. Biometrical Journal 57, 76–89.
Ondra, T., Dmitrienko, A., Friede, T., Graf, A., Miller, F., Stallard, N. and Posch, M. (2016). Methods for identification and confirmation of targeted subgroups in clinical trials: A systematic review. Journal of Biopharmaceutical Statistics, 26, 99-119, DOI: 10.1080/10543406.2015.1092034
Ondra, T., Jobjörnsson, S., Beckman, R. A., Burman, C. F., König, F., Stallard, N. and Posch, M. (2016). Optimizing trial designs for targeted therapies. PLoS One 11(9): e0163726. DOI:10.1371/journal.pone.0163726
Manuscripts submitted for publication currently under review:
Ondra, T., Jobjörnsson, S., Beckman, R. A., Burman, C. F., König, F., Stallard, N. and Posch, M. Optimizing adaptive enrichment designs (Statistical Methods in Medical Research)
Graf, A.C. Wassmer, G., Friede, T., Gera, R. and Posch, M. Robustness of testing procedures for confirmatory subpopulation analysis based on a continuous biomarker (Statistical Methods in Medical Research)
Wassmer, G., König, F. and Posch, M. Adaptive designs with multiple objectives, Handbook of Statistical Methods for Randomized Controlled Trials, CRC Press.
Manuscripts in preparation:
Graf, Magirr, Dmitrienko, Posch, Clinical trial designs with a threshold selection for a continuous biomarker
Jörgens , S., Wassmer, G., König, F. and Posch, M. Confirmatory adaptive designs with a survival endpoint using a short-term endpoint for treatment arm or population selection.
Work package 4: Research in use of evidence synthesis in the planning and interpretation of clinical trials in small populations
Research in rare diseases is often complicated by the sparseness of empirical information that goes together with the rareness of the indication itself. Evidence synthesis methods therefore play an important role in maximizing the amount of information to be derived from sparse data. Work package 4 aimed at the development of methods to synthesize few separate sources of information and to utilize external evidence to support the analysis of otherwise potentially underpowered clinical trials in rare indications.
While work package 4 was inspired by concrete applications such as the EARLY PRO-TECT trial in Alport disease, it has already fed back into applications for rare diseases, such as neuromyelitis optica (Chataway and Friede, 2016), paediatric myocarditis (Messroghli et al., 2017) and Creutzfeldt-Jakob disease (Varges et al., 2017). Motivated by the methodological challenges commonly faced in rare diseases research, the refinement of analysis methods valid also for the case of small samples was promoted (Konietschke et al., 2016).
Task 4.1 Systematic literature review
In order to gain some insight into the methodological challenges faced by clinical trials in rare diseases, and to get an overview of conventional approaches to their analysis, we conducted a systematic literature review. We selected two rare indications, paediatric multiple sclerosis (MS) and Creutzfeldt-Jakob disease (CJD), as exemplary cases and systematically compiled published information on previously conducted studies, aiming at study designs, endpoints and evaluation methods employed in these rare diseases. Twelve studies in paediatric MS and seven in CJD were included in the qualitative synthesis. We extracted information on design aspects, objectives, endpoints, patient characteristics, randomization, masking, types of interventions, controls, withdrawals and statistical methodology. While studies in paediatric MS (except for one open-label RCT) were mostly observational, several RCTs have been conducted in CJD; so the type of evidence available is variable between rare disorders. Interestingly, trials in rare diseases so far mostly appear to resort to standard designs and data analysis techniques, suggesting that more sophisticated analysis tools may contribute to some progress in these fields (Unkel et al., 2016).
Task 4.2 Evidence synthesis for a single trial
Motivated by the ongoing double-blind, randomized, placebo-controlled "EARLY PRO-TECT" trial in Alport disease, we have investigated methods to support the analysis of a single clinical trial by external observational data. In the present example, external evidence may originate from registry data as well as an open-label arm. Several approaches to synthesizing the information are conceivable, e.g. pooling of data at the level of study arms, or at the level of treatment effects derived from study arms. We investigated several hierarchical modelling strategies to aid evaluation of the trial outcome once the data become available (Unkel et al., in preparation).
Quite commonly, the aim of a meta-analysis is not the derivation of an overall "average" estimate, but rather to utilize several studies to support the estimate of a particular member of the set of synthesized studies. The resulting "shrinkage estimate" constitutes a natural and flexible way of expressing the updated estimate in the light of the additional studies, and may hence be used to implement, e.g. the analysis of an RCT considering additional information, yet explicitly focusing at the trial outcome. We investigated such an approach to make use of additional phase-II data in the analysis of a phase-III study, which promises to constitute a straightforward and transparent solution to the problem (Wandel et al., 2017). It is not obvious to what extent such an approach is helpful in the extreme case of an analysis of only two studies, but our investigations have shown that a quite substantial gain in information is not unrealistic (Röver and Friede, in preparation).
Task 4.3 Borrowing strength between trials
Evidence synthesis methods are commonly implemented based on hierarchical models. A simple and very common example is given by the normal-normal hierarchical model (NNHM), which is also at the basis of a majority of published meta-analyses. Traditionally, meta-analyses are mostly performed within a frequentist framework, where in the NNHM context a number of variations of analysis methods are commonly in use, which often mostly differ in the way the heterogeneity variance component is estimated. We investigated in particular two methods to account for the heterogeneity's estimation uncertainty, and found that a previously proposed adjustment makes a substantial difference and allows for a better control of the type-I error especially for the case of few studies (Röver et al., 2015).
Frequentist meta-analysis methods often rely on asymptotic properties that only hold for large numbers of studies, and hence perform poorly in case of small numbers of studies. Bayesian methods naturally lend itself to the problem, as they provide exact results also for finite numbers of studies and because they allow for the consideration of potentially relevant a priori information. We investigated the use of Bayesian methods motivated by the canonical example of meta-analysis of studies that are summarized in terms of contingency tables and odds ratios. In simulation studies we could show that Bayesian methods using readily motivated prior settings perform well compared to common frequentist methods with respect to confidence/credibility interval coverage and interval length, especially avoiding pathological behaviour in the case of few studies (Friede et al., 2017a; Seide et al., 2017).
The joint analysis and synthesis of only two estimates to a joint result constitutes an extreme case, although it is not particularly uncommon. Common frequentist methods tend to behave pathologically, although in a Bayesian approach, given careful model and prior specifications, it does not pose a fundamental challenge. We investigated and compared the behaviour of different meta-analysis methods in this particular case and found the Bayesian approaches to perform well, especially with respect to type-I error rates and credibility interval widths (Friede et al., 2017b).
Task 4.4 Development of software
While many meta-analysis tools are implemented and readily available, the execution of a Bayesian meta-analysis usually is much more laborious and involves tailored coding of e.g. MCMC methods and diagnostic checks. For the common NNHM, we developed a method to perform the necessary calculations semi-analytically using R, interfacing with existing software, and allowing to conduct a Bayesian meta-analysis without the need to worry about technical details. The software is fully documented, extended with further functionality and examples, and is freely available on CRAN as the "bayesmeta" R package (http://cran.r-project.org/package=bayesmeta). The underlying computational method that we developed is in fact more generally applicable to evaluate mixture distributions; the approach has been presented in detail and demonstrated using several example applications (Röver and Friede, 2017).
Another way to approach the involved computations for a Bayesian meta-analysis is the use of integrated nested Laplace approximations (INLA). Building upon previous research, we implemented an R package utilizing the "r-inla" library to perform network meta-analysis using the common "design by treatment interaction" model. The model was extended to include exact binomial likelihoods in addition to the common normal likelihood, thereby enabling more accurate evaluation especially in cases of rare events, which would otherwise challenge the normal model and require potentially biasing ad-hoc model fixes. An R package ("nmainla") is currently under development and will also be published on CRAN (Günhan et al., 2017).
Publications in print or in press in peer reviewed publications:
Chataway, J. and Friede, T. (2016) The N MOmentum trial: building momentum to advance trial methodology in a rare disease. Multiple Sclerosis Journal. 22, 852-853.
Friede, T., Röver, C., Wandel, S. and Neuenschwander, B.. (2017a) Meta-analysis of few small studies in orphan diseases. Research Synthesis Methods, 8, 79-91.
Messroghli, D.R. Pickardt, T., Fischer, M., Opgen-Rhein, B., Papakostas, K., Böcker, D., Jakob, A., Khalil, M., Mueller, G.C. Schmidt, F., Kaestner, M., Udink ten Cate, F.E.A. Wagner, R., Ruf, B., Kiski, D., Wiegand, G., Degener, F., Bauer, U.M.M. Friede, T. and Schubert, S. (2017) Toward evidence-based diagnosis of myocarditis in children and adolescents: Rationale, design, and first baseline data of MYKKE, a multicenter registry and study platform. American Heart Journal, 187, 133-144.
Röver, C., Knapp, G. and Friede, T. (2015) Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Medical Research Methodology, 15, 99.
Röver, C., and Friede, T. (2017) Discrete approximation of a mixture distribution via restricted divergence. Journal of Computational and Graphical Statistics, 26, 217-222.
Unkel, S., Röver, C., Stallard, N., Benda, N., Posch, M., Zohar, S. and Friede, T. (2016) Systematic reviews in paediatric multiple sclerosis and Creutzfeldt-Jakob disease exemplify shortcomings in methods used to evaluate therapies in rare conditions. Orphanet Journal of Rare Diseases, 11, 16.
Varges, D., Manthey, H., Heinemann, U., Ponto, C., Schmitz, M., Schulz-Schaeffer, W.J. Krasnianski, A., Breithaupt, M., Fincke, F., Kramer, K., Friede, T. and. Zerr, I. (2017) Doxycycline in early CJD - double-blinded randomized phase II and observational study. Journal of Neurology, Neurosurgery and Psychiatry, 88, 119-125.
Friede, T., Röver, C., Wandel, S., Neuenschwander, B. (2017b) Meta-analysis of two studies in the presence of heterogeneity with applications in rare diseases. Biometrical Journal (in press).
Wandel, S., Neuenschwander, B., Friede, T. and Röver, C. (2017). Using phase II data for the analysis of phase III studies: an application in rare diseases. Clinical Trials (in press).
Manuscripts submitted for publication currently under review:
Günhan, B. et al. (2017) A design-by-treatment interaction model for network meta-analysis with integrated nested Laplace approximations. (Submitted for publication).
Konietschke, F., Friede, T. and Pauly, M. (2016) Robust analyses of over-dispersed counts with varying follow-up in small samples and rare diseases. (Submitted for publication).
Publications in preparation
Röver, C. and Friede, T.. Shrinkage estimation in meta-analyses of two studies.
Unkel, S., Röver, C., Gross, O. and Friede, T.. A Bayesian hierarchical framework for evidence synthesis for a single randomized controlled trial and observational data in small populations.
Seide, S., Röver, C. and Friede, T. Likelihood-based meta-analysis of few studies.
Potential Impact:
Plan for exploitation of results
We have produced exploitable foreground by the development and publication of novel methodology and production of open-source software for the implementation of methods we have developed. Exploitable methodology has been developed in four areas, with software developed in the first two of these: evidence synthesis in small studies, dose-finding in small populations, clinical trials in stratified medicine and the design of small population clinical trials using the Value of information approach.
Dissemination and exploitation activities completed during the duration of the project include publications and conference presentations in the methodological, medical and policy areas and described in more detail below.
Exploitation work is also ongoing. Additional publications are under review and in preparation including those indicated in the results section. We are also in discussion with the other two FP7 funded small populations methodology projects, Asterix (www.asterix-fp7.eu) and IDeAl, (www.ideal.rwth-aachen.de) over further joint publications to maximise the impact of the exploitable foreground from all three projects. This may include a series of papers in Orphanet Journal of Rare Diseases as well as a single summary paper in a high-ranking medical journal. Members of the consortium will also continue to present project results at methodological and medical conferences and to engage with regulators, for example through the Small-populations Clinical Trial Task Force of the International Rare Diseases Research Consortium (IRDiRC).
Dissemination and exploitation activities
Impact and dissemination and exploitation of results of the project are focussed on four groups; the academic and scientific community, the pharmaceutical industry, national and international regulatory authorities and patients and the general public.
Dissemination to the academic scientific community is primarily via publications in peer-reviewed academic journals and through presentations at national and international conferences including conferences organised as part of the project. Details of publications are given below and in the associated tables of publications. Publications arising from the project have been listed on the project website and all publications have been or will be made available on an open-access basis to maximise dissemination. As the research work on this project is primarily methodological, it has mostly been disseminated through the statistical literature and conferences, but we have also published and presented for clinical audiences when this is appropriate. Scientific publications and conference presentations are described in more detail below.
The high level of regulation in clinical trials for evaluation of novel healthcare interventions, particularly novel medicinal products, means that dissemination to and engagement with regulatory authorities is an important step in implementation of our research results in clinical trial practice following the publication of innovative methodology in scientific publications. Close links with regulatory groups, particularly the European Medicines Agency, through membership of the project team and our Independent Scientific Advisory Committee has led to engagement with regulators at a number of meetings, as described in more detail below. As anticipated, the timespan for implementation extends beyond that of the project, and this work is ongoing.
Implementation of our research results in clinical trials sponsored by the pharmaceutical industry is likely to be established alongside regulatory acceptance. To this aim, we continue to work closely with industry statisticians. Our development of open-source software to enable implementation of the novel methodology developed, as described in more detail below, will facilitate rapid exploitation of methods arising from the work of the project.
Dissemination to patients and the general public is also discussed in more detail below. Our strategy for dissemination to these groups has focussed on presentation of project aims, progress and results in an accessible format on the project website and distributed via the patient group representatives on the project Independent Scientific Advisory Committee.
Peer-reviewed publications
Publication of research results remains the primary means of dissemination for the academic scientific community, in this case both methodological and applied. A total of 21 articles have been published in peer reviewed journals or have been accepted for publication and are available online ahead of print publication and one book chapter is in press. In addition, a further 10 papers have been submitted for publication and are under review or being revised following reviewers’ comments.
Methodological publications
Scientific and technical results of the research project have been published in high quality peer reviewed methodological journals. This is often also a prerequisite for implementation of novel methods in practice as both investigators and regulators consider the peer-reviewing process to provide a necessary validation of methodology prior to its use in a clinical study.
We have 13 papers published, or accepted for publication in methodological journals. Our focus for methodological publications have been applied statistics journals. Papers have appeared or been accepted to be published in journal including Biometrical Journal, Statistical Methods in Medical Research, Journal of Biopharmaceutical Statistics, Journal of Computational and Graphical Statistics, BMC Medical Research Methodology, Research Synthesis Methods and Plos One. Particularly important methodological publications include the following:
Work package 1:
Ursino M, Zohar S, Lentz F, Alberti C, Friede T, Stallard N and Comets E. Dose-finding methods for Phase I clinical trials using pharmacokinetics in small populations. Biometrical Journal. In press. doi:10.1002/bimj.201600084
Petit C, Samson A, Morita S, Ursino M, Guedj J, Jullien V, Comets E, Zohar S, Unified approach for extrapolation and bridging of adult information in early phase dose-finding paediatric studies , Statistical Methods in Medical Research 2016, doi: 10.1177/0962280216671348
Work package 2:
Hee SW, Willis A, Smith CT, Day S, Miller F, Madan J, Posch M, Zohar S, Stallard N (2017) Does the low prevalence rate affect the sample size of interventional clinical trials of rare diseases? An analysis of data from the Aggregate Analysis of ClinicalTrials.gov. Orphanet Journal of Rare Diseases 12:44.
Stallard, N., Miller, F., Day, S., Hee, S.W. Madan, J., Zohar, S. and Posch, M. Determination of the optimal sample size for a clinical trial accounting for the population size. Biometrical Journal. In press. DOI: 10.1002/bimj.201500228.
Work package 3:
Ondra, T, Jobjörnsson, S, Beckman, R A, Burman, C-F, König, F, Stallard, N and Posch, M (2016) Optimizing trial designs for targeted therapies. PLoS One, 11: e0163726
Ondra T, Dmitrienko A, Friede T, Graf A, Miller F, Stallard N, Posch M. (2016), Methods for identification and confirmation of targeted subgroups in clinical trials: a systematic review. Journal of Biopharmaceutical Statistics, 26, 99-119
Graf, A. C., Posch, M. and Koenig, F. (2014), Adaptive designs for subpopulation analysis optimizing utility functions. Biometrical. Journal, 57, 897-913.
Work package 4:
Friede, T., Röver, C., Wandel, S., and Neuenschwander, B. Meta-analysis of few small studies in orphan diseases. Research Synthesis Methods. In press. doi: 10.1002/jrsm.1217
Friede T, Röver C , Wandel S and Neuenschwander B. Meta-analysis of two studies in the presence of heterogeneity with applications in rare diseases. Biometrical Journal. In press. doi: 10.1002/bimj.201500236
Röver, C, Knapp, G and Friede (2015), Hartung-Knapp Sidik Jonkman Approach and its modification for random-effects meta-analysis with few studies. BMC Medical Research Methodology, 15, 99.
Medical publications
Following on from methodological publications, dissemination of research outputs to the wider scientific community often involves the development and publication of retrospective case studies or prospective applications in high quality peer reviewed journals in the relevant medical field. The illustration of potential and actual applications in this way is often an important step to more widespread use of novel methodology.
We have published case studies and/or applications of methods that we have developed as part of the project in medical journals including the following:
Unkel S, Röver C, Stallard N, Benda N, Posch M, Zohar S and Friede S, (20 Feb 2016) Systematic reviews in paediatric multiple sclerosis and Creutzfeldt-Jakob disease exemplify shortcomings in methods used to evaluate therapies in rare conditions. Orphanet Journal of Rare Diseases, 201611:16.
Chataway J , Friede T (2016) The N-MOmentum trial: Building momentum to advance trial methodology in a rare disease. Multiple Sclerosis, 22: 852-853.
Varges D, Manthey H, Heinemann U, Ponto C, Schmitz M, Schulz-Schaeffer WJ, Krasnianski A, Breithaupt M, Fincke F, Kramer K, Friede T, Zerr. Doxycycline in early CJD: a double-blinded randomised phase II and observational study, Journal of Neurology, Neurosurgery and Psychiatry. In press. doi: 10.1136/jnnp-2016-313541
Petit C, Jullien V, Samson A, Guedj J, Kiechel J-R, Zohar S, Comets E (2015) Designing a Pediatric Study for an Antimalarial Drug by Using Information from Adults. Antimicrobial Agents and Chemotherapy, 60, 1481-1491.
Policy publications
In addition to the project outputs just described, we have also published papers contributing to ongoing discussion regarding the regulatory aspects of clinical trials, particularly in rare diseases. These include the following:
Koenig, F., Slattery, J., Groves, T., Lang, T., Benjamini, Y., Day, S., Bauer, P. and Posch, M. (2014), Sharing clinical trial data on patient level: Opportunities and challenges. Biometrical Journal, 57, 8-26.
Jonker, A.H. Mills, A., Lau, L.P.L. Ando, Y., Baroldi, P., Bretz, F., Burman, C.F. Collignon, O., Hamdani, M., Hemmings, R.J. Hilgers, R.D. Irony, I., Karlsson, M., Kirschner, J., Krischer, J.P. Larsson, K., Leeson-Beevers, K., Molenberghs, G., O’Connor, D., Posch, M., Roes, K.C. Schaefer, F., Scott, J., Senn, S.J. Stallard, N., Thompson, A., Torres, A., Zohar, S., Aymé, S. and Day, S. (Eds.) (2016) Small Population Clinical Trials: Challenges in the Field of Rare Diseases, International Rare Diseases Research Consortium.
These publications, particularly the latter (also under review for publication in Orphanet Journal of Rare Diseases), represent an important first step towards influence of regulatory guidelines and ultimately to improved practice in clinical trials in rare diseases and other small population groups.
National and International conference presentations
Methodology conferences
The work of the InSPiRe project has been presented in more than 50 presentations at national and international conferences on biostatistics and clinical trials methodology. Such presentations have raised the profile of the project and serve an important function in dissemination of research results widely within a rapid timescale.
Of particular note are a number of presentations we have made, or have been invited to make in the future, in sessions at methodological conferences focussing specifically on innovative approaches to the design or analysis of clinical trials in rare diseases or small population groups. These include the following national or international medical statistics meetings:
Invited paper session at IRoeS 2015, Vienna, 2015
Invited paper session at the 36th Annual Meeting of the International Society for Clinical Biostatistics, Utrecht, 2015
Forward looking forum: Clinical trials in small populations, London, 2015
Invited paper session at Trends and Innovations in Clinical Trial Statistics, Durham NC, 2016
Mini-symposium and contributed paper sessions at the 37th Annual Meeting of the International Society for Clinical Biostatistics, ISCB 37th Annual Meeting, Birmingham, 2016
Keynote address and invited paper session at AstraZeneca and Medical Research Council Biostatistics Unit Science Symposium, Cambridge, 2017
Invited paper session at CEN ISBS Joint conference on Biometrics and Biopharmaceutical Statistics, Vienna, 2017
Biometric Colloquium of the German Region of the International Biometric Society, Frankfurt, 2018
The number of such sessions gives an indication of the raising of the profile of methodological developments in rare diseases and small populations amongst the academic medical statistics community associated with the work of the InSPiRe project along with the other two FP7 funded projects Asterix and IDeAl.
Medical conferences
In addition to presenting at methodological conferences, InSPiRe team members have also presented results of the project at a number of medical conferences including the International Conference on Rare Diseases and Orphan Drugs, Mexico City, 2015, the 21st Scientific Symposium of the Austrian Pharmacological Society, APHAR, Graz, 2015 and the Swedish Orphan Biovitrum, Stockholm, 2014.
InSPiRe project conferences
In addition to InSPiRe investigators and researchers making presentations at national and international meetings as described above, the InSPiRe project team have also organised, or contributed to the organisation of, three conferences. These have provided an opportunity to showcase the work of the project, to interact with researchers working in the field and to disseminate research findings.
The first two meetings have been jointly organised by the InSPiRe project and the other two FP7 funded small populations methodology projects, Asterix (www.asterix-fp7.eu) and IDeAl, (www.ideal.rwth-aachen.de). The opportunity to collaboration with these two other projects has provided a route for dissemination on a wider scale than would have been possible for any single project alone and also enabled us to better interact at a European level with key stakeholders including regulatory, industry and patient representative groups.
Symposium on small populations, Vienna, 1-2 July 2014
Co-chaired by InSPiRe project coordinator, Nigel Stallard, Asterix project coordinator, Kit Roes, IDeAl project coordinator, Ralf-Dieter Hilgers, Martin Posch (InSPiRe and Asterix) and Franz Koenig (IDeAl), this conference provided a high-profile start to the InSPiRe, Asterix and IDeAl projects and represented a major initial interaction between the projects.
Attended by representatives of the three projects along with representatives from academia, regulatory authorities, the pharmaceutical industry and patient representative groups with an interest in the small populations research, the main conference was a 1½ day meeting. The first day included plenary sessions introducing the three EU-funded projects and focussing on the challenges of clinical trials in small populations including ethical issues, rare diseases, paediatrics, personalised medicine, data protection and patients’ perspectives, together with parallel break-out sessions for more detailed discussions. The morning of the second day included plenary presentations on regulatory issues, extrapolation, adaptive designs and surrogate markers. In addition to the main conference, participants were able to attend short courses on Surrogate marker evaluation in clinical trials, Randomisation in clinical trials or Adaptive clinical trial designs.
Seventh Framework Programme (FP7) small-population research methods projects and regulatory application workshop, EMA London, 29-30 March 2017
This meeting was jointly organised by the InSPiRe, Asterix and IDeAl projects. Held towards the end of the three projects, it represented a major opportunity for presentation and dissemination of results arising from the project as well as discussion of these results between the three project teams and with other key stakeholders.
The two-day meeting comprised five sessions, each including a series of presentations and guided and open discussion along with introductory and closing sessions with a total of 45 presenters. Session topics covered the range of work of the three projects and were Evidence synthesis, Extrapolation, Levels of evidence and decision theoretic aspects, Study endpoints and statistical analysis and Innovative designs, pharmacometrics, modelling and optimal designs. Members of the InSPiRe team were involved as speakers or discussants in all six sessions.
The meeting was open to the public and was attended by 145 participants with representatives from all major stakeholder groups including academia, regulatory authorities, pharmaceutical industry and patient representative groups. Live streaming of the meeting over the internet also enabled observation of the meeting by others unable to attend in person. It is intended to publish a summary of the meeting jointly authored by the three project coordinators together with other members of the meeting coordinating committee; preparation of a manuscript is in progress and this will be submitted to a high profile medical journal.
Engagement with regulatory authorities was a particularly important aspect of this meeting, with all sessions jointly chaired by a member of one of the three project teams and a representative from EMA or a regulatory committee.
InSPiRe project Conference on Methodology for Clinical Trials in Small Populations and Rare Diseases, Warwick, 26-28 April 2017
Towards the end of the project we organised and hosted at three-day conference at University of Warwick. The aim of this conference was presenting key results from the project and bringing together international experts in innovative clinical trials design and analysis to present recent advances in the methodology for clinical trials in small populations.
The conference was organised with seven invited plenary sessions on Applications in Paediatrics, Research in early phase dose-finding trials in small populations, Decision-theoretic approaches for clinical trials in small populations, Evidence: assessment and synthesis, Research in confirmatory trials for small populations and personalised medicines, Evidence synthesis of clinical trials in small populations and rare diseases and Ongoing work and future challenges in methodology for small population clinical trials. Each session included one or more members of the InSPiRe team presenting project results along with external speakers. External speakers included Géraldine Favrais (University Hospital of Tours), Oliver Gross (Department of Nephrology, Goettingen), Johanna van der Lee (University of Amsterdam), Lisa Hampson (AstraZeneca and University of Lancaster), Andrew Willan (University of Toronto), Christopher Jennison (University of Bath), Leo Held (University of Zurich), Heinz Schmidli (Novartis), Robert Beckman (Georgetown University), Beat Neuenschwander (Novartis), Ralf-Dieter Hilgers (Aachen University and IDeAl project coordinator) and Kit Roes (University Medical Centre, Utrecht and Asterix project coordinator).
External speakers were invited for their expertise in the area and to maximise contributions to the discussion of the work and dissemination of research and technological developments from the project.
In addition to the plenary invited talks, poster presentations were also invited, with eight poster submitted and displayed during breaks and a dedicated poster session. Posters were particularly welcome from research students, with students presenting a poster having their conference registration fee covered by the InSPiRe project. Four of the eight posters were submitted by PhD students.
As a means of dissemination of current best practice, conference delegates were also able to register to attend one of two pre-conference courses given by members of the InSPiRe team. Martin Posch and Gernot Wassmer from Medical University of Vienna gave a course on Adaptive Designs for Studies in Small Populations and Tim Friede from University Medical Centre Goettingen with Simon Wandel from Novartis gave a course on Meta-analysis of few studies. The number of participants attending the two courses were 16 and 12 respectively.
The conference was attended by 73 delegates with representatives from industry, regulatory authorities or patient groups in addition to academia.
Open access software
The availability of software is often a hurdle to the widespread adoption of novel statistical methods. To address this issue, we have produced open access statistical software to implement two of the new approaches that we have developed. We have written programs to run on the freely available software environment R (www.r-project.org) as this is open access, available on variety of platforms and widely used by medical statisticians. Software packages are available for download from the Comprehensive R Archive Network (cran.r-project.org).
We have produced the following R packages:
bayesmeta: Bayesian random effects meta-analysis
(available at http://cran.r-project.org/package=bayesmeta)
Associated with the evidence synthesis work of Work Package 4, this package provides a collection of functions to facilitate easy Bayesian inference in the generic random-effects meta-analysis model. It allows to the user to derive the posterior distribution of the two parameters (effect and heterogeneity), and provides the functionality to evaluate joint and marginal posterior probability distributions, predictive distributions, shrinkage, etc. The main functionality is provided by the bayesmeta function. This takes the data (estimates and associated standard errors) and prior information (effect and heterogeneity priors), and returns an object containing functions that allow to derive posterior quantities like joint or marginal densities, quantiles, etc. Use of the package is demonstrated in a worked-out example available on cran.r-project.org.
dfpk: A Bayesian dose-finding design using pharmacokinetics (PK) for phase I clinical trials
(available at https://cran.r-project.org/web/packages/dfpk/index.html)
Associated with the work of early phase dose-finding studies in Work Package 1, this package provides statistical methods involving PK measures in the dose allocation process during a Phase I clinical trials. These methods enter pharmacokinetics (PK) in the dose finding designs in different ways, including covariates models, dependent variable or hierarchical models. This package provides functions to generate data from several scenarios and functions to run simulations which their objectives are to determine the maximum tolerated dose (MTD) and to investigate the pharmacokinetics of a drug or a combination of drugs. The dfpk R package provides an interface to fit Bayesian generalized (non-)linear mixed models using Stan, which is a C++ package for obtaining Bayesian inference using the No-U-turn sampler (mc-stan.org/).
Engagement with stakeholders including patients and the general public
Engagement with patients and the general public
The InSPiRe project website (http://www.warwick.ac.uk/inspire) provides descriptions of the project aims and has been updated throughout the project to give details of progress and summaries of innovative methods developed in style and language appropriate for the lay reader in addition to more technical details.
Since January 2017 Tim Friede has been founding member of the Center of rare disease Göttingen (ZSEG) where several clinics and departments of the University Medical Center Göttingen collaborate together to disseminate information on rare disease, treatment possibilities, clinical research and events to patients and clinicians. We also have links with patient groups through EURORDIS, who are represented on our Independent Scientific Advisory Committee along with other patient representative groups. All members of this committee also receive regular updates on project progress. Regular face-to-face meetings of this committee provide an opportunity to discuss project progress as well as being able to comment on the research and its direction.
Details of the project have also been the subject of press releases from the University of Warwick and other project partners and have been included on http://www.news-medical.net. Along with the Asterix and IDeAl projects, the InSPiRe project has also featured in the publication Impact (https://impact.pub) raising the profile of the project among the broader scientific policy communities.
Engagement with regulatory authorities and other stakeholders
Engagement with relevant regulatory authorities at a national and international level has been an important and ongoing part of the project as we see this as an essential prerequisite for the broader uptake of novel methodology developed as part of the InSPiRe or other small populations projects. Previous and current regulatory authority and committee members are included both as part of the InSPiRe team and as members of the Independent Scientific Advisory Committee.
A major regulatory development of relevance to clinical trials in small populations during the time of the InSPiRe project has been the production of the draft EMA PDCO Reflection paper on extrapolation of efficacy and safety in paediatric medicine development. Following the 2013 EMA Concept paper, a one-day Workshop of an EMA Extrapolation expert group was held in September 2015. This EMA Extrapolation group included clinicians, pharmacologists, pharmacometricians and statisticians from the EMA, national regulatory authorities and academia. From the InSPiRe project team, Martin Posch was a presenter at this workshop and Tim Friede and Nigel Stallard participated. The output from this meeting led to the production of the draft Reflection paper in March 2016. Following the release of the draft Reflection paper, a two-day public Workshop on extrapolation of efficacy and safety in medicine development across age groups was held by EMA in May 2016. InSPiRe team members were again involved in this meeting, with Martin Posch on the Organising Committee, Tim Friede and Nigel Stallard, the latter on behalf of the InSPiRe, Asterix and IDeAl projects.
The level of interest in and commitment to the InSPiRe, Asterix and IDeAl projects by the EMA is also demonstrated by their willingness to host the joint meeting of the three projects in March 2017 as described in more detail in the section on InSPiRe project conferences above. This meeting provided excellent opportunities to continue the dialogue between regulators, academia and patient groups that has been a feature of these projects, and gives a good indication that this will continue beyond the end of the projects, leading to adoption and/or implementation of the methods developed. Martin Posch has also given a presentation on the work of the InSPiRe project to the Japanese Regulatory Authority (PMDA) in March 2015.
Along with Kit Roes and Ralf-Dieter Hilgers, coordinators of the Asterix and IDeAl projects, Nigel Stallard, the InSPiRe project coordinator, and Simon Day from the InSPiRe team were invited to join the Steering Committee of the Small-populations Clinical Trial Task Force of the International Rare Diseases Research Consortium (IRDiRC), which also include Martin Posch, Tim Friede and Sarah Zohar from the InSPiRe team. The steering group organised a two-day workshop of the task force, held at EMA in March 2016. A report of the workshop has been published on the IRDiRC website (see above) and has been submitted for publication by Orphanet Journal of Rare Diseases.
InSPiRe project website
The InSPiRe project website was created at the outset of the project and has been regularly maintained and updated throughout the duration of the project.
The website explains the focus and main aims of the project in accessible language as well as giving details of the work undertaken in each project Work Package. The website gives contact details for the project coordinator as well as details of all project partners and the Independent Scientific Advisory Committee.
A ‘News’ page includes press releases and upcoming events and dissemination activities, while our publications and details of our open source software, with links to the Comprehensive R Archive Network website where these are available, are given on a ‘Project Results’ page.
The project website also serves as the website for the main InSPiRe project conference, providing a portal for registration as well as full details of the conference.
The InSPiRe project logo as well as the EU logo and funding acknowledgement statement appear on all website pages, ensuring consistent branding is presented.
List of Websites:
The address of the project public website:
www.warwick.ac.uk/inspire
Contact details for project coordinator:
Professor Nigel Stallard
Statistics and Epidemiology
Division of Health Sciences
Warwick Medical School
University of Warwick
Coventry CV4 7AL
UK
Email: n.stallard@warwick.ac.uk