Landscape Analysis of the Application of Artificial Intelligence and Machine Learning in Regulatory Submissions for Drug Development From 2016 to 2021
[The copyright line for this article was changed on 07 July 2022 after original online publication.]
An analysis of regulatory submissions of drug and biological products to the US Food and Drug Administration from 2016 to 2021 demonstrated an increasing number of submissions that included artificial intelligence/machine learning (AI/ML). AI/ML was used to perform a variety of tasks, such as informing drug discovery/repurposing, enhancing clinical trial design elements, dose optimization, enhancing adherence to drug regimen, end-point/biomarker assessment, and postmarketing surveillance. AI/ML is being increasingly explored to facilitate drug development.
BACKGROUND
Over the past decade, there has been a rapid expansion of artificial intelligence/machine learning (AI/ML) applications in biomedical research and therapeutic development. In 2019, Liu et al. provided an overview of how AI/ML was used to support drug development and regulatory submissions to the US Food and Drug Administration (FDA). The authors envisioned that AI/ML would play an increasingly important role in drug development.1 That prediction has now been confirmed by this landscape analysis based on drug and biologic regulatory submissions to the FDA from 2016 to 2021.
THE TREND OF INCREASING AI/ML–RELATED SUBMISSIONS AT THE FDA'S CENTER FOR DRUG EVALUATION AND RESEARCH
This analysis was performed by searching for submissions with key terms “machine learning” or “artificial intelligence” in Center for Drug Evaluation and Research (CDER) internal databases for Investigational New Drug applications, New Drug Applications, Abbreviated New Drug Applications, and Biologic License Applications, as well as submissions for Critical Path Innovation Meeting and the Drug Development Tools Program. We evaluated all data from 2016 to 2021. Figure 1a demonstrates that submissions with AI/ML components have increased rapidly in the past few years. In 2016 and 2017, we identified only one such submission each year. From 2017 to 2020, the numbers of submissions increased by approximately twofold to threefold yearly. Then in 2021, the number of submissions increased sharply to 132 (approximately 10-fold as compared with that in 2020). This trend of increasing submissions with AI/ML components is consistent with our expectation based on the observed increasing collaborations between the pharmaceutical and technology industries.
Figure 1b illustrates the distributions of these submissions by therapeutic area. Oncology, psychiatry, gastroenterology, and neurology were the disciplines with the most AI/ML–related submissions from 2016 to 2021.
Figure 1c summarizes the distributions of these submissions by the stage of therapeutic development life cycle. In these submissions, most of the AI/ML applications happen at the clinical drug development stage, but they also happen at the drug discovery, preclinical drug development, and postmarketing stages. It is important to note that the frequency by which AI/ML is mentioned in regulatory submissions to the FDA likely only represents a fraction of its increasingly widespread use in drug discovery.
COMMON ANALYSIS TYPES AND OBJECTIVES FOR AI/ML–RELATED SUBMISSIONS AT CDER/FDA
- Outcome prediction: AI/ML was often used for prediction of clinical outcome, including disease prognosis and treatment response (for both efficacy and safety) based on characteristics of patients as well as treatments (e.g., drug and dose). This is not surprising since one of the most mature areas where AI/ML has shown significant promise generally is its predictive capabilities and its ability to handle a large number of input variables.
- Covariate selection/confounding adjustment: AI/ML was also commonly used for covariate selection and confounding adjustment. For example, in some submissions, AI algorithms, such as decision tree–based algorithms, were used to screen through a large amount of baseline information (e.g., patient demographic data and lab values) to find important factors that impact patients' prognosis as well as exposure or response to a drug. In some cases, a simpler model trained with the factors selected by the AI/ML algorithm was built to guide treatment or monitoring of participants.
- Pharmacometric modeling: AI/ML has been used for pharmacometric modeling, consistent with predictive capabilities of AI/ML algorithms and the increasing interest in AI/ML among the pharmacometrics community.2-4
- Anomaly detection: There were a number of submissions that described the use of AI/ML for anomaly detection. In fact, for high-dimensional data, AI/ML techniques, such as the Isolation Forest Algorithm, are being proposed as more powerful than traditional statistical analyses or graphical methods in identifying potential outliers or anomalies.
- Imaging, video, and voice analysis: Another area where AI/ML has long shown promise is the analyses and evaluation of imaging data. We identified some submissions that included AI/ML, usually deep learning, for the analyses of imaging data,5, 6 videos, or voices.
- RWD phenotyping/Natural Language Processing: Some submissions explored the use of AI/ML (natural language processing) to support the phenotyping of real-world data from certain sources.
- Drug discovery/repurposing: AI/ML has been demonstrated to be a useful tool for drug discovery and repurposing. Although information about the discovery phase may not be typically in a submission to the FDA, we have seen submissions where sponsors stated that the selection of the therapeutic targets or drug candidates was supported by AI/ML.
- Drug toxicity prediction: In some submissions, AI/ML was used to predict the potential safety risk of a drug based on its structure, physiochemical properties, or affinity for targets.7
- Enrichment design: One common application of AI/ML in drug development is to facilitate enrichment trial design. Enrichment is the prospective use of any patient characteristic to select a study population in which detection of a drug effect is more likely than it would be in an unselected population, if there is in fact a drug effect.8 Generally speaking, there are three categories of enrichment strategies: (i) decreasing variability, (ii) prognostic enrichment strategies, and (iii) predictive enrichment strategies.8 AI/ML has been used for all three enrichment strategies. For example, a sponsor proposed to use an AI/ML–based clinical trial inclusion/exclusion criterion to select canonical patients whose symptoms are characterized by greater similarity to the typical patients in the target population in order to decrease intersubject variability. Another sponsor proposed to use AI/ML to generate a prognostic score to identify patients for primary efficacy analyses for a phase III trial.
- Patient risk stratification and management: Certain submissions explored the use of AI/ML for patient risk stratification and management, as well as treatment or dose selection/optimization. For example, a sponsor proposed AI/ML to predict patients' risk for a specific severe adverse event based on patient baseline information, and then they proposed to use this prediction to help determine the need of inpatient or outpatient monitoring for each patient.
- Dose selection/optimization: AI/ML was proposed for the selection/optimization of the drug or dose based on patient characteristics.
- Adherence to drug regimen: AI/ML–based monitoring platforms were used in trials to aid and confirm adherence to an investigational drug regimen. It is worth mentioning that this kind of approach has been proposed in trials across phases (i.e., phases I, II, and III trials).
- Synthetic Control: AI/ML has been proposed to generate synthetic controls. One example is to use ML to create digital twins to predict what would happen to a specific participant in a clinical trial if he or she had received a placebo.9
- End-point/biomarker assessment: AI/ML was proposed as components of drug development tools, such as clinical outcome assessments and biomarker identification. For example, a sponsor proposed that an AI/ML algorithm be used as part of a digital health technology tool to serve as a clinical outcome assessment to support the evaluation of the effectiveness of therapeutic interventions in participants with a skin condition. Another sponsor proposed to use AI/ML analysis of video and audio recordings of patients to calculate visual and auditory markers of schizophrenia symptomatology and use them as exploratory efficacy end points to measure change from baseline in disease severity. In another submission, a sponsor proposed using AI/ML to discover radiographic biomarkers that correlate with survival and adverse events after the cancer treatment. In addition, a recent biomarker qualification-related submission proposed a biomarker consisting of the features comprising the Non-Alcoholic Fatty Liver Disease Activity Score and fibrosis staging, assessed on liver biopsy as interpreted by a convolutional neural network.5, 6
- Postmarketing Surveillance: AI/ML has also been used in postmarketing surveillance. For example, a sponsor proposed using AI/ML on real-world data for a postmarketing requirement pregnancy outcomes study.
ENVISIONING THE FUTURE
As demonstrated from this analysis, AI/ML is being utilized across many aspects of the drug development life cycle. AI/ML holds great promise to help improve both the efficiency of drug development and to further inform the understanding of the efficacy and safety of the treatment. There is an increasing trend of AI/ML applications for drug development in recent years, and the authors anticipate that this trend will likely only increase over time. Both opportunities and challenges lie ahead for the potential uses of AI/ML, and pharmaceutical and technology companies are actively investing in this area. Moreover, academic researchers are continuing to investigate current and future applications. The FDA has also been preparing to manage and evaluate AI/ML uses by engaging with a broad set of stakeholders on these issues and building its capacity in these scientific fields, in order to promote responsible innovation in this area. In 2021, the FDA and other regulatory agencies jointly identified 10 guiding principles that can inform the development of Good Machine Learning Practice to help promote safe, effective, and high-quality medical devices that use AI/ML.10 Although these Good Machine Learning Practice guiding principles were developed for medical device development, many of them (e.g., multi-disciplinary collaboration; data quality assurance, data management, and robust cybersecurity practices; representativeness of study participants and data sets; independence of the training and testing data sets) are also applicable to drug development. Liu et al. discussed some expectations for the application of AI/ML in drug development (e.g., fit-for-purpose and risk-based expectations, proper validation, generalizability, explainability, etc.).1 It is important to note that the regulatory considerations for the application of AI/ML in drug development are evolving and will require input from all stakeholders in various disciplines. Effective communication and active collaboration will serve an increasingly important role in fostering innovation, helping to advance regulatory science, and aiding in the promotion and protection of public health in the United States and the world.
ACKNOWLEDGMENTS
The authors thank Rajanikanth Madabushi for critical review of the manuscript, and Giang Ho and Kimberly Bergman for their assistance in the production of Figure 2. This work was supported in part by an appointment to the Research Participation Program at the Office of Clinical Pharmacology/Center for Drug Evaluation and Research, US Food and Drug Administration, administered by the Oak Ridge Institute for Science and Education (ORISE) through an interagency agreement between the US Department of Energy and the US Food and Drug Administration. Julie Hsieh and Mo Tiwari were ORISE fellows contributing to this work.
FUNDING
No funding was received for this work.
CONFLICT OF INTEREST
The authors declared no competing interests for this work.
DISCLAIMER
The contents of this article reflect the views of the authors and should not be construed to represent the FDA's views or policies. No official support or endorsement by the FDA is intended or should be inferred.
