Linked Pharmacometric‐Pharmacoeconomic Modeling and Simulation in Clinical Drug Development

Market access and pricing of pharmaceuticals are increasingly contingent on the ability to demonstrate comparative effectiveness and cost‐effectiveness. As such, it is widely recognized that predictions of the economic potential of drug candidates in development could inform decisions across the product life cycle. This may be challenging when safety and efficacy profiles in terms of the relevant clinical outcomes are unknown or highly uncertain early in product development. Linking pharmacometrics and pharmacoeconomics, such that outputs from pharmacometric models serve as inputs to pharmacoeconomic models, may provide a framework for extrapolating from early‐phase studies to predict economic outcomes and characterize decision uncertainty. This article reviews the published studies that have implemented this methodology and used simulation to inform drug development decisions and/or to optimize the use of drug treatments. Some of the key practical issues involved in linking pharmacometrics and pharmacoeconomics, including the choice of final outcome measures, methods of incorporating evidence on comparator treatments, approaches to handling multiple intermediate end points, approaches to quantifying uncertainty, and issues of model validation are also discussed. Finally, we have considered the potential barriers that may have limited the adoption of this methodology and suggest that closer alignment between the disciplines of clinical pharmacology, pharmacometrics, and pharmacoeconomics, may help to realize the potential benefits associated with linked pharmacometric‐pharmacoeconomic modeling and simulation.


BACKGROUND
Many jurisdictions across Organization for Economic Cooperation and Development (OECD) countries require evidence of cost-effectiveness to secure the reimbursement and formulary inclusion of new medicinal products. 1 Economic evaluations, conducted as part of a broader health technology assessment (HTA), are often the preferred approach to demonstrate value; that is, whether the benefits of a new treatment are sufficient to justify any additional cost. There is a trend of increasing requirements for cost-effectiveness to secure reimbursement, including in the United States. 2 As pricing and market access has become increasingly contingent on the ability to demonstrate cost-effectiveness, there has emerged the potential for the pharmaceutical industry to benefit from methods of quantifying the economic potential of drug candidates earlier in development.
Earlier value assessments are especially pertinent in the context of the high and increasing cost of drug development that is skewed toward the later phases. The return on investment in pharmaceutical research and development (R&D) has also steadily declined since 2010, 3 although returns would not appear to have yet fallen below those for other large companies. 4 The mean cost of phase III development has been estimated to exceed the cost of phase II by more than a factor of four. 5 It is, therefore, desirable to fail development candidates at the earliest stage possible before incurring large costs-hence the mantra "fail fast, fail cheap." While it does appear that there has been some success in shifting project terminations towards earlier phases in recent decades, the proportion failing at phase III remains high. 6 Studies of attrition rates show inadequate efficacy at phase III to be the primary reason for termination but that 22% failed for commercial reasons 7 and 24% for strategic reasons. 8 Even if marketing authorization is granted, there may be difficulty securing reimbursement. In the United Kingdom, for example, in a sample of drug appraisals only around half of funding decisions were positive without restrictions. 9 Considerations of the requirements for drug reimbursement ought to be integrated within the drug development decision-making process. Failure to do so may risk late-stage termination of projects that are deemed not commercially viable only after substantial resources have been invested, or in medicines gaining regulatory approval only to prove unmarketable at a commercially viable price or failing to gain substantive market share.
Early economic evaluation during drug development aims to increase R&D efficiency through earlier termination of commercially nonviable drugs and to inform pricing strategies by predicting the price reimbursement that authorities would be willing to pay. 10 Economic evaluation should ideally be applied iteratively, beginning in the preclinical phase and subsequently updated as clinical data emerge, 11,12 and used beyond go/no-go and pricing decisions, to also inform strategic data collection and risk management decisions. 11 However, the methods and data sources required to parameterize early economic models have not been well described, including how to address the issue that outcomes in terms of the safety and efficacy of treatments will either be unknown or highly uncertain.
It has been proposed that safety and efficacy inputs to early economic models may be obtained using pharmacometric models, by extrapolating from early-phase data along with the associated uncertainty. 13 In this approach, pharmacometric models, developed routinely in the early clinical phases of drug development, are used to simulate the potential range in outcomes from proposed pivotal clinical trials, 14 which in turn serve as inputs to a pharmacoeconomic model. As pharmacometric models provide a mathematical description of the exposure-response relationships, while also accounting for interindividual variability, inputs to pharmacoeconomic models could be simulated for specific patient subgroups, doses, regimens, or drug adherence scenarios. It also provides a framework that allows uncertainty in safety and efficacy to be captured and propagated through to the economic end points, which is essential in order to quantify decision uncertainty and the value of conducting further research. 15 In this article we refer to this methodology as linked pharmacometric-pharmacoeconomic (PMPE) modeling and simulation. Our aim in this study was to review the published literature for applications of linked PMPE modeling and simulations since the original publications in this area; to identify categories of decisions and timings when this approach may have potential to add value to decisions making; to describe the key challenges in implementing the methods in practice; and consider what barriers may have limited the extent to which this has been applied during drug development.

TYPES OF EARLY ECONOMIC EVALUATION IN DRUG DEVELOPMENT
The concept of early economic evaluation, which may be defined as "all methods used to inform industry and other stakeholders about the potential value of new medical products in development, including methods to quantify and manage uncertainty," 16 first appeared in the literature in the early the nineties. 10 Published applications of early economic evaluation in drug development are limited 16,17 and appear to be less numerous than those for medical device development. 18 Review articles of early economic evaluation have presented various means of classifying or categorizing the types of early economic evaluation that are possible, in terms of their methodology and the types of research question which they address. 19,20 Linked PMPE modeling is one approach available to enable and enhance early economic evaluation in drug development. We consider three broad categories of drug development problems to which linked PMPE could be applied: (i) applications where the primary focus is on "go/no-go" decisions; (ii) applications whose primary focus is to inform study design and development strategy; and (iii) applications used primarily to inform decisions relating to pricing, reimbursement, or market access strategies ( Table 1). A fourth category is presented for completeness, to include those published PMPE studies that are not focused on drug development questions but may be used to inform resource allocation decisions from a payer perspective.

ECONOMIC EVALUATION OF PHARMACEUTICALS
HTA is a multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its life cycle. The purpose is to inform decision making in order to promote an equitable, efficient, and high-quality health system. 21 This often includes economic evaluation, defined as the process of conducting an analysis of alternative courses of action in terms of both their costs and consequences, taking the perspective of society, a healthcare provider or payer. 22 This process is inherently comparative and is commonly applied when a medicine has received marketing authorization, to inform the decision of whether, and to what extent, it will replace the existing standard of care (SoC). An important source of evidence providing inputs to such evaluations, in particular relating to efficacy, are the pivotal phase III clinical trials. 23 However, other sources of evidence may also be required, and pharmacoeconomic modeling involves methods for linking surrogate and clinical end points, evidence synthesis, generalizing trial findings, extrapolation to appropriate time horizons, and quantifying decision uncertainty. This is represented by "late stage HTA" in Figure 1, whereby costs and outcomes of two treatments, ℞ 1 and ℞ 2 , are informed from pivotal clinical trial(s) and provide inputs to an economic evaluation.
An important outcome measure in the context of HTA is the quality-adjusted life year (QALY), developed in order to facilitate comparisons across different disease areas and interventions. QALYs are based on utilities, which are preference-based valuations of health-related quality of life, and measured on a scale where perfect health = 1 and death = 0. 24 QALYs are calculated as the integral of utility scores over time, often over a patient's lifetime. Aggregated QALYs therefore represent an overall measure of length of life weighted by quality of life. 25 Weights are intended to represent societal preferences for being in a given health state and are obtained via survey methods. The evidence for the utilities associated with different states of health, for use in HTA, is often collected within pivotal clinical trials but may also be taken from external sources and applied to observed health states or clinical events. The ratio of incremental health benefit (e.g., QALYs) to the incremental cost is known as the incremental cost-effectiveness ratio (ICER). Some reimbursement authorities make decisions by relating ICERs to threshold values of willingness to pay, above which health technologies are deemed not cost-effective. 1

LINKED PHARMACOMETRIC-PHARMACOECONOMIC CASE STUDIES
We conducted a literature research to identify studies that had used linked pharmacometric and pharmacoeconomic models in order to address questions relating to drug development or the use of existing medicines. We searched PubMed, Medline, EconLit, Embase, and BIOSYS Previews for articles published between January 2001 and April 2020. Search terms included pharmacometric, pharmacokinetic pharmacodynamic, drug development, and investigational compound. These terms were searched in combination with the terms cost-effectiveness, pharmacoeconomic, health economics and economic evaluation. The relevant articles that this search identified were used to identify additional relevant studies by examining study references. We also used the results of previous reviews on early economic evaluation in medical product development. 16,17 We identified 10 publications implementing linked PMPE models since the first publication by Hughes et al., outlining  the methodology in 2001. 13 Following the study by Poland & Wada, 26 there was a considerable delay until the subsequent studies began to emerge, the first being Pink et al. 27 in 2012. There may, however, have been applications of linked PMPE performed within the pharmaceutical industry that have not been published. The following subsections summarize each of the published studies in turn, grouped according to the type of analysis as outlined in Table 1. The important characteristics of each study including the disease area, the perspective adopted, the approximate timing within, or beyond, drug development to which the approach would be most applicable, the type of decision addressed, and specific research questions are summarized in Table 2. A summary of the types of submodels the studies included, model structures, and specific analyses performed is provided in Table 3.

Drug development applications
Go/no-go decisions. The economic viability of a project should be assessed at every stage of drug development as new data become available. This includes decisions ranging from whether to initiate a drug discovery effort to whether to continue into phase III clinical trials. Economic evaluations performed at very early stages, during drug discovery or in preclinical phases, could help identify target product profiles most likely to achieve the types and scale of benefits that would be required to obtain a price in a range that may justify further R&D investment. Subsequently, as biomarker or early outcome data from clinical studies emerge, pharmacometric models can be used to extrapolate early results to late-stage pharmacoeconomic outcomes to assess whether efficacy and safety thresholds are likely to have been met and justify continued development. An early-stage economic evaluation using linked PMPE was conducted by Hill-McManus et al. 28 to assess the potential for new xanthine oxidase inhibitors to achieve price premiums relative to the existing treatment options. The pharmacology of the comparator, another xanthine oxidase inhibitor, was described using a compartmental pharmacokinetic (PK) model and multicompartment semimechanistic pharmacodynamic (PD) model. This was linked to a pharmacoeconomic model which predicted rates of clinical events and extrapolated to a longer-term time horizon to estimate incremental costs and QALYs. Hypothetical compounds were simulated by modifying the clearance and potency parameters of the existing comparator, to estimate the value-based price of such drugs under conditions of "real-world" drug adherence. Simulations predicted a distribution of possible hypothetical drug prices that would meet a cost-effectiveness threshold of £20,000 (~ US $25,434) per QALY. The results showed only modest price premiums would be acceptable even for much more potent, or slowly cleared (more forgiving 29 ) hypothetical drugs, suggesting that discovery and development of such compounds is unlikely to be commercially viable unless therapeutic benefit in addition to reduction in gout flares could be identified.
Slejko et al. 30 reported a PD model-based meta-analysis (MBMA) linked to a pharmacoeconomic model to assess the cost-effectiveness of a hypothetical drug for chronic obstructive pulmonary disease (COPD). The purpose of the analysis was to predict the threshold of drug efficacy that would be cost-effective. The MBMA was used to predict (as nonlinear functions) the COPD exacerbation rates given changes in forced expiratory volume (FEV 1 ) with placebo or drug treatment, which then served as inputs to a Markov microsimulation pharmacoeconomic model estimating costs and QALYs over a 1-year period. The results indicated that a hypothetical anti-inflammatory drug that increased FEV 1 by 50 mL and decreased exacerbations by 26%, and if priced at €35 (~ US $41 per month), would be associated with an ICER ranging from €13,000 to €207,000 (~ US $15,163 to $241,434) per QALY gained across different patient subgroups, with the lowest ICER in patients with severe COPD. The simulations identified which COPD drug profiles are most likely to be cost-effective and in which patient population this is most likely to be achieved.
Early estimates of cost-effectiveness, or of pricing options, may be useful in planning for product launch and approaches to market access. A linked PMPE model of rituximab for the treatment of follicular lymphoma, undertaken as a proof of concept, was used to predict the trial-based cost-effectiveness of a trial that had yet to report. 27 Using a population PK model and a PD model which linked serum drug concentrations to progression-free survival, the analysis simulated a phase III randomized controlled trial comparing rituximab, fludarabine, and cyclophosphamide with rituximab, cyclophosphamide, vincristine, and prednisolone induction chemotherapies for the treatment of stage II-IV follicular lymphoma. Utility and cost estimates for Markov model states representing disease progression, no progression, or death were used to project lifetime estimates of QALYs and health service costs. Two further analyses of rituximab as first-line and maintenance therapy were compared with economic models based on completed trial results used by the National Institute for Health and Care Excellence (NICE). The predicted ICERs in these analyses were found to differ by 25% or less.
R&D strategy and trial design. Applications that are used to inform R&D strategy or trial design cover a wide range of possible decisions. Examples could include portfolio decisions or compound selection, choice of target patient population, or dose and regimen selection. The influence of trial designs on economic outcomes may be studied by examining the impact of imperfect drug adherence (or interventions to improve adherence), the choice of comparator, or the inclusion/exclusion criteria. It may also be possible to determine optimal sample sizes of clinical trials with respect to the value provided by reduced uncertainty in expected value-based prices, e.g., Brennan & Breeze 31 and Dranitsaris et al., 32 but there are no published examples of this being done using pharmacometric models. Another potential application is to inform decisions regarding clinical trial data collection, beyond sample size, according to the resulting reduced cost of the uncertainty in inputs to economic models, i.e., value of information. 15 Poland & Wada 26 presented a real-world prospective case study of an HIV protease inhibitor drug in late-stage development. The objective was to evaluate alternative development strategies that STATE of the ART differed in terms of the doses and regimens for treatment naïve or experienced patients. Models of patient adherence, PK, PD, and viral dynamics were linked to an economic model in order to compare the development strategies in terms of their predicted net present value. The results highlighted the importance of adherence if using once-a-day dosing and suggested there would be high value to a longer lasting-more forgiving-drug formulation. 33 Many details of the modeling were omitted from the publication in order that the drug and pharmaceutical company involved could not be identified.
Van Hasselt et al. developed a PMPE model to assess the toxicity, dose intensity, disease progression, and cost-effectiveness of eribulin for castration-resistant prostate cancer. 34 The analysis utilized a semiphysiological model for neutropenia, the major dose-limiting toxicity of eribulin, a disease progression / clinical outcome model relating prostate-specific antigen dynamics to overall survival, and an economic model that incorporated quality of life and treatment-related costs. The motivation for the analysis was to illustrate how trial design and decision making in oncology could be further informed using a model-based approach early in clinical drug development. Simulations evaluating alternative treatment regimens, disease progression criteria, dose reduction rules, patient population, and comparators were performed, and QALYs and ICERs were presented for a range of scenarios.
Hill-McManus & Hughes demonstrated how linked PMPE modeling could be used to optimize phase III trial design, focusing primarily on sample size determination. 35 This case study adopted a Bayesian decision theoretic approach to trial design in which the optimal design is that which is predicted to result in the greatest expected utility. The utility function employed was the expected pharmaceutical company return on investment. A prior distribution for the efficacy of a new treatment was obtained via pharmacometric-based clinical trial simulation and used to calculate a posterior distribution given a proposed sample size. Posterior treatment efficacy was converted to a value-based drug price distribution via a pharmacoeconomic model and subsequently to company return on investment. This enabled uncertainty to be propagated from aspects of drug pharmacology through to the predicted return on investment, where drug prices are contingent on the results of phase III testing. Such an approach may have value in supporting trial design decisions and in identifying costly sources of uncertainty, and thus inform future research and development strategies.
Pricing, reimbursement, and market access decisions. Linked PMPE modeling may be able to inform pricing decisions such as price setting to ensure sufficiently high likelihood of reimbursement, or to assess differences in potential prices across different markets and inform market entry strategies. A case study of urate-lowering therapies for treating gout showed how, prior to proceeding into phase III testing, a probability distribution of drug prices could be simulated based on target costeffectiveness thresholds. 36 The simulations used uncertainty in pharmacokinetic-pharmacodynamic (PKPD) model parameters to estimate the likelihood of cost-effectiveness over a range of  drug prices and tested alternative biomarker to clinical outcomes modeling approaches and adherence scenarios. The results highlighted the importance of adherence in clinical trials for these medications in terms of its impact on effectiveness estimates and subsequent economic outcomes.

Other applications
Linked PMPE modeling has been applied to research questions beyond drug development, adopting a societal, regulatory, or payer perspective. This can enable reimbursement authorities or regulators to assess cost-effectiveness of treatments, or treatment strategies, where there is an absence of direct trial evidence. Published case studies using linked PMPE have been applied to assess the cost-effectiveness of pharmacogenetic-guided warfarin dosing algorithms, 37 to compare influenza pandemic treatment scenarios in terms of cost-effectiveness, 38 and to make early predictions of the potential cost-effectiveness of generic versions of the direct oral anticoagulant dabigatran. 39 Pink et al. used pharmacometric-based prediction of time within international normalized ratio (INR) range for warfarin based on a genetic dosing algorithm, to estimate the cost-effectiveness of different oral anticoagulation treatment strategies in patients with nonvalvular atrial fibrillation. 37 Justification for taking this approach was based on there being many possible dosing algorithms-far more than might be conceivably tested in a clinical trial-and the magnitude of the differences in both benefits and costs being small, requiring a prohibitively expensive trial to demonstrate differences in hemorrhagic or stroke outcomes. The authors used a pharmacometric model that described the effects of CYP2C9 and VKORC1 genetic polymorphisms, alongside other clinical parameters, on the relationship between warfarin dose and INR response. Linked to this, the economic model utilized evidence from an indirect comparison of treatment options, a meta-analysis of the association between time in INR range and risk of hemorrhagic events and strokes, and costs and utilities derived from published sources. The principal finding, of an ICER for genotype-guided warfarin vs. clinical algorithm-dosed warfarin of £13,226 (~ US $16,820) per QALY gained compared favorably with a later analysis using trial data. 40 This example provided further evidence of the feasibility of applying a linked PMPE approach to estimate cost-effectiveness.
Kamal et al. 38 developed an integrated modeling framework, including pharmacometric, epidemiology, and economic models, to compare treatments under influenza pandemic scenarios for which it is not practical to conduct clinical trials. This utilized a PKPD model linked to a compartmental epidemiological susceptible, exposed, infected, recovered model, and economic model of costs and health outcomes. The number of infected patients per population of susceptible individuals was simulated for a series of pandemic scenarios, varying oseltamivir dose, basic reproductive number, and drug uptake. The study assumed the perspective of a potential US payer and used QALYs as the measure of benefit and assessed cost-effectiveness using ICERs. The results indicated that across all pandemic scenarios, 75 mg twice-daily oseltamivir was cost-saving and more effective compared with no treatment, with QALY gains of 430 per 100,000 population over 1 year in a scenario of low transmissibility and low severity. The authors concluded that this approach has the potential to inform government planning for an effective and cost-effective response to influenza pandemics. We consider that there is broad scope for linked PMPE in the assessment of value of antimicrobials, where this value depends on a drug's properties as well as where it is positioned in the strategic use of antimicrobials. 41 Wang et al. 39 applied linked PMPE modeling and simulation to assess the effectiveness and cost-effectiveness of potential future generic versions of the direct oral anticoagulant dabigatran. Direct oral anticoagulants have a narrow therapeutic range, and there is concern that future generics at the outer limits of bioequivalence may not be equivalent in terms of clinical effectiveness and might not be cost-effective vs. the branded drug despite the difference in price. The authors used a previously published PKPD model and adapted and reproduced a previously published economic model. The models were used to simulate clinical event rates and to predict QALYs and ICERs for hypothetical generic dabigatran at the typical limits of bioequivalence of 80% and 125% systemic exposure, as well as at the more conservative bioequivalence limits of 90% to 112.5%. The results showed that generics with 80% systemic exposure relative to the branded medication were predicted to result in higher QALYs and lower costs, whereas those with 125% systemic exposure showed worse cost-effectiveness due to increase in bleeding events. The authors acknowledge that these simulations represent the worst-case scenario in terms of bio-inequivalence; however, the work highlights that cost-effectiveness prediction may be useful in assessing the value of generic substitution particularly where there are steep concentration-effect relationships.

CONSIDERATIONS IN LINKING PHARMACOMETRICS AND PHARMACOECONOMICS
In this section we consider five important concepts in the application of linked PMPE models during drug development and how they have been dealt with in the published case studies. These concepts are (i) value outcomes; typically these have been based on QALYs, but this may not always be appropriate; (ii) evidence on comparators, which is an essential input for linked PMPE models; (iii) the approach to linking multiple intermediate end points; (iv) dealing with model uncertainty; and (v) the challenge of model validation.

Value outcomes
All but one of the case studies identified used the QALY as the outcome measure for use in a cost-effectiveness calculation. There may, however, be other treatment effects, beyond those captured in QALYs, which may contribute to how patients or society value new treatments and their opportunity costs. 42,43 These could include convenience (e.g., dose frequency, method/ route of administration or treatment monitoring), as well as measures that incorporate patient preferences and/or risk aversity over the distribution of outcomes, and broader societal considerations (e.g., equity or work productivity). Where reimbursement decisions are known to be based on a range of different criteria, the task of early economic evaluation is likely to be more challenging.
In theory such additional dimensions of treatment effect could be incorporated within early economic evaluations if the necessary studies are performed at an earlier point in development. Methods such as discrete choice experiments 44 are available for assessing the relative weights that patients, or payers, place on different treatment outcomes. The results could then be represented by willingness to pay for product characteristics or translated into health state utility increments for use in cost-effectiveness analysis. 45 Alternatively, reimbursement decision making involving a range of criteria can be modeled using multicriteria decision analysis. 46 Regardless of the approach adopted, an understanding of the decision-making process for the relevant reimbursement authorities is required.

Evidence on comparators
Pharmacoeconomic evaluations are inherently comparative; in HTA this will typically involve assessing the incremental benefit and cost of a new treatment relative to another that is often the current SoC. The relevant comparators are those that the reimbursement authority considers would be most widely used in the clinical context, and it is essential that these are identified. While from a drug development perspective, the only data available on a development candidate may be that which is generated "in-house" through the conduct of clinical studies, evidence on comparators could come from a wide range of sources. Incorporating evidence on comparators in PMPE models may be achieved using point estimates from previous studies, published PKPD models, or meta-analytic approaches for evidence synthesis.
A variety of approaches have been used to incorporate comparator evidence in the published examples of linked PMPE. Van Hasselt, 34 Hill-McManus, 28 Kamal et al., 38 and Wang et al. 39 effectively focused on a single drug and so additional evidence on active comparators was not required. Poland and Wada 26 did not conduct a comparative economic evaluation and, therefore, did not explicitly consider the likely outcomes of comparators. Hill-McManus et al. 36 and Hill-McManus & Hughes 35 used published PK models for the comparators and constructed a PD model based on published phase I trial data and other published models but did not use a meta-analytic framework. Pink et al. 27 used trial results for comparators as well as data from an existing meta-analysis of clinical trials. Both Pink et al. 37 and Slejko et al. 30 employed meta-analyses that linked an intermediate biomarker end point to clinical end points. This latter example used an MBMA and could, therefore, predict economic outcomes for a range of drug efficacies as well as trial design inputs.
Meta-analyses of published data have become the main sources of evidence on comparative effectiveness and safety. 47 Traditional meta-analysis focuses on head-to-head comparisons of drugs administered in the same trials and ignores the (often considerable) data available from other studies involving these drugs. Network meta-analysis (NMA) combines direct and indirect evidence, allows comparisons of multiple treatments, 48 and thereby synthesizes a greater share of the available evidence than a traditional meta-analysis. 49 NMA is often used in HTA for simultaneous indirect comparison of effects of multiple treatments using studied doses and treatment durations, 50 and therefore, has limited potential to extrapolate to different doses and durations.
MBMA is a meta-analysis that incorporates parametric pharmacology models for the effect of treatment and patient population characteristics (e.g., different baseline disease severity) on outcomes. 51 In a drug development setting, MBMA models are useful to predict different doses, timepoints, patient populations, and end points when progressing from one phase of development to the next. MBMA may inform (i) quantitative criteria for the target product profile to position a new drug relative to the SoC and/ or emerging competitors, (ii) probability of success (e.g., being preferred over SoC and/or emerging competitors), (iii) early insights into best-in-class potential, and (iv) probability of technical success in demonstrating superiority or noninferiority in an actual head-to-head trial. Additional examples of strategic integration of MBMA across the drug research-development-utilization continuum are provided by Upreti & Venkatakrishnan. 51 More recently, a framework for dose-response and time-course model-based network meta-analysis has been proposed that combines, often nonlinear, MBMA modeling with the statistically robust properties of NMA. 52,53 Handling of multiple end points Within the context of drug development, the purpose of PMPE modeling is to provide a framework wherein patient-relevant outcomes forming the basis for reimbursement decisions (e.g., survival and quality of life) can be estimated from drug exposure. However, a direct relationship between drug exposure and such outcomes is unlikely to be characterized during drug development, especially in early phases. Efficacy and safety are instead measured using intermediate end points, often biomarkers, which may change during drug development as the scale and nature of clinical testing evolves. Intermediate/biomarker outcomes may also be accepted by regulatory agencies and reimbursement authorities (e.g., in leukemia 54 ) as the basis for decision making and, therefore, used as primary surrogate end points in pivotal phase III clinical trials. 55 The nature and validity of surrogate end points, and whether surrogates are accepted by regulators / reimbursement authorities, may influence how, and whether, PMPE modeling is informative.
The use of biomarker and clinical end points in published PMPE studies is summarized in Table 4, using definitions taken from the Biomarkers Definition Working Group study. 56 A diagrammatic representation of the linking of intermediate and final end points in three of the published case studies in given in Figure 2. Two studies did not use biomarker end points at all, and drug exposure was linked directly to the rates of clinical events. These studies were either assessing the use of a drug in a new indication 27 or as a generic version of an already marketed drug. Therefore, these benefitted from the availability of numerous clinical studies having been reported, including pivotal phase III trials.
Most PMPE case studies used a biomarker as an intermediate end point that was then linked to clinical event rates. Typically, the relationship between biomarker and clinical events was well evidenced and had been used previously for other therapies in the same, or similar, indication. For example, Pink et al. 37 performed a meta-analysis of the relationship between INR and bleeding and thromboembolic events; and, Slejko et al. 30 used MBMA to combine data across many studies to describe the relationship between  Biomarker: a characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention; surrogate: a biomarker that is intended to substitute for a clinical end point. A surrogate end point is expected to predict clinical benefit (or harm or lack of benefit or harm) based on epidemiologic, therapeutic, pathophysiologic, or other scientific evidence; clinical end point: a characteristic or variable that reflects how a patient feels, functions, or survives. 56 FEV 1 and COPD exacerbations. Two case studies in gout used a biomarker, serum uric acid, whose relationship to the relevant clinical event rates is poorly evidenced. 28,36 This biomarker, however, although not validated, 57 was used as a surrogate outcome in the clinical trials. The linking of these end points was thus undertaken as part of the pharmacoeconomic component of the modeling.

Modeling uncertainty
Methods to quantify the extent of uncertainty in modeled outcomes and to examine the relative contribution of model inputs to that uncertainty are used in both pharmacometrics 58 and pharmacoeconomics. 59 In both disciplines, sources of uncertainty can be broadly classified as either relating to the model parameters (usually due to sampling) or to model structural assumptions. That similar approaches to handling these types of uncertainty have emerged in both disciplines suggests these may be readily applied to linked PMPE models. To examine parameter uncertainty, pharmacometrics and pharmacoeconomics both use similar methods of Monte Carlo simulation for sensitivity analyses in which models are evaluated multiple times with parameters varied simultaneously over the entire parameter space. In pharmacometrics this is known as global sensitivity analysis, 60 while in pharmacoeconomics, it is known as probabilistic sensitivity analysis. 61 Approaches to assessing structural uncertainty include transforming to parametric uncertainty 62,63 and model averaging. 64,65 The published PMPE case studies considered model uncertainty using a variety of one-way (or local) sensitivity analyses and probabilistic (or global) sensitivity analyses. The studies using a probabilistic sensitivity analysis approach generated a distribution of outcomes by assigning probability distributions to model inputs and using Monte Carlo simulation to generate model outputs using new sets of input parameters. 27,28,30,36,37 Where the influence of specific parameters was assessed this was only undertaken using one-way (or local) sensitivity analyses, not using methods that account for model outputs being nonlinear in input parameters. More useful for R&D decisions and study design is likely to be the relative contribution of the uncertainty in specific model parameters to the overall uncertainty in modeled outcomes. There are methods from both pharmacometrics / systems pharmacology and pharmacoeconomics for this purpose, 60 for example, pharmacoeconomic models have applied methods taken from Bayesian decision theory. 66 One of the case studies did adopt a value-of-information approach 35 but stopped short of computing the expected value of partial perfect information for model parameters. None of the studies attempted to examine the impact of structural uncertainty beyond some one-way sensitivity analyses testing alternative modeling assumptions.

Model validation
By model validation we are referring to the process of evaluating whether a model is a proper and sufficient representation of the system it is intended to represent in view of an application. This encompasses whether the model is in accordance with what is known about the system and the extent that the results can serve as a solid basis for decision making. 67 Validation is an important element in model development and is relevant to a number of different aspects, such as the conceptual model, the selection and processing of input data, and the outputs of the operational model. 68 Other terms are also used to describe this process, such as model qualification in the disciplines of pharmacometrics and quantitative systems pharmacology. 69,70 Linked PMPE models may be more difficult to validate owing to the additional uncertainty introduced as compared with application of the individual components. 30 The common technique of comparison of model outcomes with external data is often problematic for pharmacoeconomic models where such empirical data may only become available years in the future, if at all. Few of the published linked PMPE case studies have included significant reporting of efforts to validate the models. Here we summarize efforts described to validate model outcomes by comparing these to empirical data. Pink et al. 27 compared simulated ICERs with two actual trial-based economic evaluations, and observed differences of 18% and 25%, although the 95% central ranges of predicted ICERs also included the trial-based results. The authors predicted the cost-effectiveness for a comparison between treatments based on a trial that had yet to begin (NCT01303887), which will provide a valuable opportunity to assess model validity when this trial reports. Pink et al. 37 performed external validation of the PKPD model by comparing simulated vs. actual time in therapeutic range. These authors also showed comparable ICERs when using simulated and trial-based biomarker response data, 40

DISCUSSION
Linked PMPE modeling and simulation has been recognized as a methodology that can enable and enhance economic evaluation during drug development and beyond. There is a small but growing literature of applications to drug development decision problems and to those regarding optimization of existing therapies. We identified ten examples from the published literature which are summarized in this paper. These span a wide range of drugs and disease areas; with applications concerning drug go/no-go; R&D strategy and trial design; and pricing, reimbursement, or market access decisions. Other identified applications include estimating the cost-effectiveness of complex pharmaceutical interventions (such as in the context of pharmacogenomics), generic medicines with narrow therapeutic indices, and treatments of infectious diseases.
During early drug development, compared with conventional pharmacoeconomic evaluations, linked PMPE may potentially enable a far broader range of research questions to be addressed by exploiting pharmacometric models of the exposure-response relationships. As the published case studies show, this can include consideration of the economic outcomes associated with treatment of specific patient subgroups, alternative doses or regimens, or protocol deviations such as nonadherence. The overarching objective is that incorporating such approaches within the decision-making process will contribute to reducing drug development costs by improving decisions regarding project termination and trial design. However, linked PMPE may be of no greater value to decision makers than existing approaches where there are limited patient data, is lack of evidence on comparators, or are poorly characterized relationships between biomarkers and clinical outcomes.
It would seem that there is currently a great deal of interest in early economic evaluation, and the coming years are likely to see a growth in the number of applications, furthering our understanding of the potential of such innovative interdisciplinary methods. The recent article by Grutters et al. 18 and the subsequent commentaries 72 provides a valuable discussion from multiple perspectives on some of the same issues raised in this review, although in the context of nondrug interventions and early economic modeling in general. An article published while ours was under review, aimed towards a health economics audience, also discusses the literature on incorporating pharmacometrics into pharmacoeconomics. 73 These authors' conclusions, in line with our own, point to the potential of this methodology to inform drug development decision making and the need for greater collaboration between the pharmacoeconomics and pharmacometrics communities.
Nearly all the case studies adopted incremental QALYs as the measure of treatment benefit, which yields an ICER as a metric of cost-effectiveness when combined with incremental costs. The application of early economic evaluation will be more complicated in situations where QALYs do not capture all the important treatment benefits, and when payers do not apply ICER-based decision rules or their process and criteria for decision making is not transparent. This may necessitate the use of additional techniques, such as multicriteria decision analysis and other measures of treatment value. Early engagement with payers is clearly important in order that emerging therapies can be assessed against appropriate criteria, and so that future studies are designed to generate evidence that satisfies the requirements of decision makers.
The linked PMPE modeling framework provides a tool to combine and propagate the uncertainty from inputs to an estimate of uncertainty in economic outcomes, such as cost-effectiveness or drug price. We found that most of the published case studies had performed a probabilistic assessment of outcome uncertainty in which input parameters were varied simultaneously over the entire parameter space. The linked PMPE methodology, in which a series of models link across multiple intermediate end points, will typically require a large number of input parameters that may be highly uncertain early in drug development. This would then result in highly uncertain economic outcomes with limited value to decision makers. In this case, value of information analysis 74 may be more useful than the point estimates, enabling the cost associated with sources of uncertainty to be identified and the valuing of study designs according to the extent that they reduce uncertainty. 15 Only one of the case studies used a value-of-information approach, where trial design was optimized by comparing the value of the resulting evidence in terms of its influence on drug prices. 35 This framework could be readily extended to assess the influence of parameters on decision uncertainty (expected value of partial perfect information 66 ) and to direct future research efforts. These methods are well established in healthcare resource allocation decisions from a societal perspective, but there are few examples published that take a pharmaceutical industry perspective. 31,75 We consider that value of information may have considerable potential to add value during drug development and would benefit from further research. 15 Each of the published case studies has the form of a standalone exercise designed to inform a drug development decision or optimize the use of existing therapies. Early economic models, however, could be applied iteratively throughout drug development, incorporating emerging data on treatment efficacy, safety, and pharmacology to provide increasingly robust assessments of the commercial potential of development candidates. This approach has long since been encouraged, 11,12 but, as far as we are aware, economic modeling during drug development is not routinely applied in this way. The pharmacoeconomic model for a specific disease area, target population, and type of treatment may, structurally, not change substantially over the drug development timeline. Safety and efficacy profiles, providing inputs to pharmacoeconomic models, will be estimated with increasing confidence as further studies are conducted and pharmacometric models may then enable applications to a wider range of research questions. This could involve an iterative process of Bayesian updating of model inputs, synthesizing evidence across the development timeline, such as has been demonstrated for medical device development. 76 The concept of a framework in which quantitative methods are applied iteratively to inform decision making during drug development is an established principle of the model-informed drug development (MIDD) paradigm. 77 MIDD embraces the integration of systems pharmacology and pharmacometric modeling approaches: empirical time course analysis including disease progression modeling, empirical to semimechanistic PKPD modeling, and MBMA approaches. The application of these methods in dose/trial optimization, to inform labeling and R&D decision making is well established. [77][78][79] There is a growing appreciation of the value of these methods in enabling more efficient development and regulatory acceptance of novel medicines. 80,81 It is also increasingly being accepted by regulators that MIDD offers an alternative approach to risk-benefit evaluation where there is high unmet medical need and/or there are difficulties in conducting trials. [80][81][82][83] Incorporating early pharmacoeconomic modeling within the MIDD framework would naturally promote linkage between these methods and associated disciplines. 84 This review was limited by only having access to applications of linked PMPE modeling and simulation that were published. This may, in fact, only be a small proportion of all such analyses that have taken place, since the majority may be being performed by the pharmaceutical industry in-house and deemed unpublishable due to being considered too commercially sensitive. Nevertheless, the published literature contains a small number of examples that demonstrate the range of therapies, disease areas, and decisions for which insights can be gained through the use of this interdisciplinary approach.
A potential barrier to the integration of pharmacoeconomic modeling within drug development decision making is the historical separation of clinical development and commercial functions within the pharmaceutical industry, with technical experts and modelers in separate organizational arms linked only by highlevel review and governance. It has been suggested that substantial changes to the pharmaceutical industry operating model, including the breaking down of functional silos, are needed to help reverse the trend of declining R&D productivity. 3,85 Our view is that optimal functioning, with respect to effective integration of predictive modeling within a defined organizational structure, needs to be driven by common and integrated goals that promote cross-functional collaborative end-to-end working. With this in mind, the R&D business process, starting with the desired target product profile, needs to foster greater cross-functional and interdisciplinary alignment to enable innovative interdisciplinary activities such as linked PMPE modeling and simulation to become a more standard component of clinical development plans.
Relevant to both the early engagement with payers to understand what constitutes value and to the bridging of company clinical development and commercial functions are initiatives to foster dialogue between regulators, payer/HTA/reimbursement organizations, and the pharmaceutical industry. These include the US Food and Drug Administration (FDA) paired meeting pilot program for providing regulatory input on MIDD issues 80 and the availability of parallel scientific advice from both the European Medicines Agency (EMA) and HTA bodies. 86 These may naturally encourage cross-functional collaboration and provide information regarding the payer landscape which is essential to early pharmacoeconomic modeling. Early pharmacoeconomics is also relevant to the risk minimization and management efforts and could promote the development of commercial risk management plans. Furthermore, it has been proposed that risk minimization may be operationalized in a way that serves as a bridge between clinical drug development and commercial functions within organisations. 87 Other barriers to the application of linked PMPE modeling and simulation include technical issues such as the appropriate software platforms, and the challenge of communicating novel interdisciplinary methods to decision makers. Widespread application of this methodology may benefit from a common data science platform for managing data and associated expertise. The use of certain software packages within each discipline may also restrict integration, such as the prevailing use of NONMEM (pharmacometrics) and MS Excel (pharmacoeconomics). The use of more flexible (and open source) solutions such as R, Python, or Julia is becoming more widespread, and this would support a collaboration between disciplines. The disciplines also differ to some extent in the standard terminology used to describe the same, or similar, modeling issues, presenting a potential source of confusion. Finally, decision makers across R&D and commercial functions may require education in relation to the potential productivity gains to be achieved through novel or interdisciplinary approaches, such as linked PMPE modeling and simulations, as is occurring in relation to artificial intelligence.
In conclusion, while linked PMPE modeling and simulation remains relatively novel, with few published case studies, it has potential across a range of applications from early drug development to post marketing. It has advantages over conventional pharmacometric studies that do not assess the future value of compounds beyond market authorization; and conversely, it has advantages over standard pharmacoeconomic practices which do not utilize exposure-response relationships. Closer integration and collaboration between the disciplines of clinical pharmacology, pharmacometrics, and health economics is needed in order to realize the potential benefits and promote greater acceptance of this methodology.