Drug development for rare diseases presents significant challenges, particularly in generating reliable data from clinical trials (CTs) conducted on small samples. Small trials often yield results with limited precision, making it difficult to interpret the treatment effects or achieve statistical significance. To address this situation, alternative trial designs that effectively use both internal and external data on treatment efficacy and safety should be considered a necessity to maximise the ability to draw clinically relevant conclusions. Bayesian methods have been proposed as a valuable framework for investigating interventions in such small samples.
Utility of Bayesian Methods in Rare Disease Clinical Trials
Traditionally, CT analysis has relied on frequentist statistics, which are heavily dependent on long-run properties of repeated experiments and are rooted in the concept of the p-value. However, this approach has several limitations, including challenges in interpreting the proper meaning of the p-value and confidence intervals, difficulties in estimating the probability of clinical benefit, and a lack of straightforward mechanisms for integrating external information with internal trial data. Bayesian statistics, on the other hand, are well-suited to overcome these challenges.
Bayesian statistics are particularly relevant in rare disease research because they provide a clear probability of a clinically meaningful treatment benefit. Additionally, the Bayesian approach offers a formal framework for incorporating external information – such as historical data, published literature, ongoing trials, and other real-world evidence – into the statistical analysis of a CT. Since small sample trials are often underpowered, including external information can provide the necessary context for making robust decisions. This approach also helps avoid prematurely dismissing interventions that may require additional data to demonstrate efficacy.
Using external data in CT has been applied in various stages of drug development, including earlier phases, occasionally in Phase III trials, and specialised areas such as medical devices, orphan indications, and extrapolation in paediatric studies. Several recent regulatory discussions have highlighted both the opportunities and challenges of using external information in trial design and analysis.
Bayesian statistics offer a systematic approach to combine all available evidence. The process begins with creating a prior distribution of the parameters of interest (Fig. 1) based on external data and expert opinion, reflecting the existing knowledge collected from previous trials, published literature, ongoing trials, and other real-world data. As the current CT progresses, new data is gathered, which provides the likelihood of the treatment’s effect. This new information is then combined with the prior knowledge to form a posterior distribution of the treatment effect, which is used to quantify results and draw conclusions about the treatment’s effectiveness.
Fig. 1. Here we see from the posterior distribution that there is 99% probability that the treatment effect is greater than 10. From Kidwell K.M. et al. Orphanet Journal of Rare Diseases (2022).
Limitations of Bayesian Methods in Rare Disease Trials
Despite the strong methodological foundation and advantages of Bayesian methods in rare disease drug development, there are certain limitations. One potential issue is an increased risk of type I error* when there is significant conflict between external control data and trial data. Additionally, Bayesian trial design and analysis require careful planning and early discussions with regulators before the trial begins. Communicating Bayesian results to non-statisticians, who may be more familiar with traditional p-value-based analyses, can also be challenging.
Conclusion
Conducting clinical studies for rare diseases is inherently challenging due to the limited number of patients. In many cases, traditional trial designs and analytic methods may not be feasible for generating the substantial evidence needed for regulatory agency approval. The use of innovative trial designs, such as master protocols and complex adaptive designs, in combination with Bayesian methods, can help reduce sample sizes, select appropriate treatments and populations, and accurately assess treatment effects in the context of rare diseases.
*: A type I error occurs when we reject the null hypothesis and erroneously state that the study found significant differences when there indeed was no difference. In other words, it is equivalent to saying that the groups or variables differ when, in fact, they do not or having false positives.
Veronique Ropion, MD
Strategic Projects Director, Pharmalys Ltd
Bibliography:
- European Medicines Agency. Guideline on clinical trials in small populations [Internet]. 2006. Report No.: CHMP/EWP/83561/2005. https://www.ema.europa.eu/en/documents/scientific-guideline/guide line-clinical-trials-small-populations_en.pdf.
- Food and Drug Administration. Adaptive designs for clinical trials of drugs and biologics: guidance for industry. Silver Spring, MD; 2019. Report No.: FDA-2018-D-3124.
- Food and Drug Administration. Interacting with the FDA on complex innovative trial designs for drugs and biological products: guidance for industry. Silver Spring, MD; 2020. Report No.: FDA-2019-D-3679.
- Kidwell K.M. et al. Application of Bayesian methods to accelerate rare disease drug development: scopes and hurdles. Orphanet Journal of Rare Diseases (2022) 17:186. https://doi.org/10.1186/s13023-022-02342-5
- Shreffler J., Huecker M.R. Type I and Type II Errors and Statistical Power. NIH, National Library of Medicine, National Center for Biotechnology Information. Last Update: March 13, 2023.