Patient‐Focused Drug Development Methods for Benefit–Risk Assessments: A Case Study Using a Discrete Choice Experiment for Antiepileptic Drugs

Regulatory decisions may be enhanced by incorporating patient preferences for drug benefit and harms. This study demonstrates a method of weighting clinical evidence by patients’ benefit–risk preferences. Preference weights, derived from discrete choice experiments, were applied to clinical trial data to estimate the expected utility of alternative drugs. In a case study, the rank ordering of antiepileptic drugs (AEDs), as indicated from clinical studies, was compared with ordering based on weighting clinical evidence by patients’ preferences. A statistically significant change in rank ordering of AEDs was observed for women of childbearing potential who were prescribed monotherapy for generalized or unclassified epilepsy. Rank ordering inferred from trial data, valproate > topiramate > lamotrigine, was reversed. Modeling the expected utility of drugs might address the need to use more systematic, methodologically sound approaches to collect patient input that can further inform regulatory decision making.

Drug development and regulatory decision making require explicit evaluation of benefits and risks. In balancing the benefits and risks of drugs, judgements are required on the maximum acceptable risk (MAR) of harm for an expected health benefit. Traditionally, these judgements have relied on expert clinical opinion of the available evidence. More recently, however, there is increasing acknowledgment that patients' preferences with respect to trade-offs in harms for benefits may differ from those of clinical experts and that patient perspectives need to be considered in benefit-risk assessments. 1,2 The US Food and Drug Administration (FDA) Center for Drug Evaluation and Research currently implements a qualitative approach via the Patient-Focused Drug Development Program under the fifth and sixth authorizations of the Prescription Drug User Fee Act. This involves patients in the approvals process by convening public meetings to discuss the impact of disease on patients' daily lives and patients' perspectives on treatment benefits and adequacy. 3 The European Medicines Agency's Committee for Medicinal Products for Human Use piloted patient involvement in benefit-risk discussions through their participation in expert group meetings and the scientific advice/protocol assistance procedure. 4 The Committee for Medicinal Products for Human Use continues to involve patients in oral explanations when it is believed this could be of benefit. The FDA's Center for Devices and Radiological Health (CDRH), by contrast, has adopted a more quantitative approach and considers evidence relating to patients' perspectives of what constitutes meaningful benefit-risk. 2,5 Moving forward, the 21 st Century Cures Act (section 3002) requires the FDA to develop guidance (by quarter 4 2021) for integrating relevant patient experience data in benefit-risk assessments for new drugs and biological agents. 6 Although specific details are currently unknown, it is worth noting the CDRH's guidance includes 11 recommendations for the conduct of quantitative patient preference studies, 5,7 and a project coordinated by the European Medicines Agency outlined a range of methods. 8 Both organizations identify discrete choice experiments (DCEs) among potential options for use in benefit-risk assessments of medicines.
Stated preference DCEs are a method for quantifying the relative importance of different treatment characteristics (attributes), trade-offs between these attributes, and respondents' total satisfaction (utility) with specified treatments. 9 The method has been used extensively for the evaluation of health services and health technologies, 10,11 and there is increasing application in the determination of benefit-risk trade-off for medicines. 12,13 In DCEs, respondents are asked to choose their preferred option from a set of hypothetical (but realistic) alternatives, on the basis of attributes and their respective levels determined using established qualitative methods. 14 Modeling the expected utility of drugs, on the basis of patientdefined benefit-risk trade-offs, offers a quantitative approach to inform drug development and regulatory decision making. The present study demonstrates the potential for the DCE method in benefit-risk assessment. Patient priorities for drug outcomes are first identified using qualitative interviews and ranking exercises. Patient preferences for the most important outcomes are then valued using a DCE. This generates preference weights for each selected outcome that are applied to observed data from a clinical trial. The expected utilities of alternative drugs are then modeled to generate patient-defined benefit-risk preferences for a selection of alternative drugs.
The case study is based on antiepileptic drugs (AEDs), selected because of the particular need to balance benefits against the risks of harm. 15 Although two thirds of patients treated with AEDs will achieve remission from seizures within 5 years of diagnosis, at least 40% of patients will experience treatment-related adverse effects. 16,17 Patient-determined levels of acceptable harm for improved seizure control are rarely considered in clinical trials. 18 However, they are essential considerations, especially for women of childbearing potential with generalized epilepsy, in whom choices need to be made between the most effective yet teratogenic treatment, valproate, and less teratogenic (but less effective) alternatives. 17,19 In this case study, we aimed to compare patient-defined benefit-risk preferences with the results of a clinical trial of AEDs 16,17 and to assess differences between patient subgroups.

Benefit and risk attribute selection
Fifty-four patients and nine physicians (34% and 90% of those invited, respectively) participated in the qualitative study to identify the most important benefit and risk outcomes for use in the DCE. Reduction in seizure frequency was the most important treatment outcome (benefit) across all groups (Table S1), and discussions surrounding this outcome focused on seizures stopping and patients achieving remission. The importance of adverse effects (risks) varied by patient group: recently diagnosed patients were most concerned about feelings of aggression and depression, whereas those with an established diagnosis were most concerned about memory problems. Women of childbearing potential also ranked memory problems highest, followed by the risk of fetal abnormality if they became pregnant while taking the drug. Physicians ranked memory problems and depression highly. Patients and physicians valued life impacts (e.g., reduced independence), but in the context of the decision to initiate or switch an AED, physicians considered them to be consequences of epilepsy, or the adverse events (AEs) of treatment, rather than independent treatment outcomes. As such, they were excluded from the DCE. Attributes selected for the DCE are presented in Table 1. Two versions of the DCE were developed to represent the most important attributes selected by patients with a recent or established diagnosis (DCE-1) and women of childbearing potential (DCE-2) ( Figure S1).

Discrete choice experiment
Among those who consented to the survey, 4 withdrew before randomization and 29 did not start their DCE, leaving 280 patients for this analysis ( Table 2).
All patients preferred the AED offering the greatest benefits and the lowest risk of harm, as indicated by direction of the signs on the coefficients (+/−) ( Table 3). Patients had stronger preferences for reductions in the risk of AEs than improvements in 12-month seizure remission, as shown by the greater magnitude of the harm coefficients. Experiencing fewer seizures did not have a significant influence on AED preferences of women of childbearing potential (P = 0.685).
The maximum acceptable increase in risks of adverse effects for an AED that increases the chance of 12-month remission by 10% (in DCE-1) were as follows: 3.1% for depression, 3.0% for memory problems, and 2.5% for aggression. For women of childbearing potential, the MAR of fetal abnormality was 2.0% for a 10% increase in probability of remission, compared with 5.6% for depression and 3.4% for memory problems.

Patient utility associated with selected AEDs
The preference-weighted outcomes (remission, reduction, memory problems, depression, aggression/fetal abnormality) for four alternative drugs for focal epilepsy (carbamazepine, lamotrigine, gabapentin, and topiramate) and three alternative drugs for generalized and unclassified epilepsy (valproate, lamotrigine, and topiramate) 16,17 are displayed in Figure 1. On the basis of total utility (sum of the weighted outcomes), respondents to DCE-1 indicated carbamazepine would be the most preferred AED for focal epilepsy, and topiramate the least. The disutility associated with topiramate (−0.67) suggests these patients would prefer to avoid it. For generalized or unclassified epilepsy, valproate yielded higher utility than both lamotrigine and topiramate. In both cases, ranking based on utility differed from ranking based on time to treatment failure, as observed in clinical trials ( Figure 1a); however, this did not reach statistical significance.
Women of childbearing potential responding to DCE-2 favored lamotrigine over carbamazepine for focal epilepsy, in agreement with trial-based ranking; however, gabapentin ranked second when the trial outcomes were weighted in the utility model. Topiramate was associated with the lowest utility. Lamotrigine was also favored for generalized or unclassified epilepsy, with valproate being the least preferred; thus, weighting by patient preferences for benefit and harm outcomes resulted in a statistically significant reversal of rankings. 20 The combined risk and preference weighting for fetal abnormality heavily influenced the disutility of valproate ( Figure 1b).
The improvement in benefit associated with a switch to valproate did not outweigh the increase in risk for women of childbearing potential. A switch from topiramate to valproate for generalized or unclassified epilepsy, for instance, increased the risk of fetal abnormality by 6.65%, 19 which exceeds the 1.41% MAR that corresponds to the 7% improvement in 12-month remission seen with valproate 17 (Table S2).

DISCUSSION
DCEs represent a valid and reliable method to quantify patientfocused outcomes. [10][11][12][13] The utility model demonstrated how quantitative data on patient preferences can be integrated with clinical evidence to provide a patient-focused benefit-risk analysis that could be used to inform regulatory decision making. The case study illustrates how DCEs can provide an explicit estimate of patient-defined MAR, which may differ from value judgements based on clinical evidence alone.
The FDA implemented changes after the fifth authorization of the Prescription Drug User Fee Act, to increase the clarity, transparency, and consistency of its benefit-risk assessments. With the passage of the 21st Century Cures Act, the FDA now has an imperative to consider how relevant patient experience data and related information can be incorporated into the structured benefit-risk assessment framework to inform regulatory decision making. 6 Our proposed method is one approach that might address the FDA's recognized need to use more systematic, methodologically sound approaches to collect patient input so that it becomes data that can further inform regulatory decision making 21 (Figure 2). Plausible estimates based on: • Seizure frequency 16,17 • Clinical expert opinion (research team, scientific advisory group, and group discussion meeting with prescribing physicians) We would like you to imagine you have the choice between two medications: medication A and medication B. We will give you the same information about each medication. The chance of responding well: -Seizures stop -Fewer seizures Finally, we will also give you information on the risk of harm to the fetus if you get pregnant while taking this medication. This may cause problems, such as spina bifida, a hole in the heart, and a cleft palate (where the roof of the mouth is not correctly joined). This may also cause neurodevelopment problems, such as poor memory, poor language and social skills, and low IQ. AED, antiepileptic drug; DCE, discrete choice experiment; IQ, intelligence quotient; SANAD, Standard vs. New Antiepileptic Drugs. a As described in each of the eight choice questions. b Only in DCE for patients with a recent or established diagnosis. c Term "adverse effects" used to describe adverse events, as per findings of our qualitative study. d Only in DCE for women of childbearing potential.
Patients have an important role alongside all other stakeholders in determining relevant outcomes and priorities, acceptable uncertainty, as well as benefit-risk and value of a medicine. 1,2 However, patients believe that decision makers, particularly regulators, do not always have a complete understanding of the risks that patients with some illnesses are willing to accept, and that this benefit-risk assessment will vary by disease. 22 Our DCE-based approach linked with clinical trial evidence addresses this, by generating patientdefined benefit-risk thresholds for drugs for specific clinical indications.
There are several examples of what could be interpreted as different levels of acceptance of risk among some patients, regulators, or other stakeholders-most notably, in the context of HIV, where "patient experts" successfully challenged the drug development and licensing paradigms, leading up to the FDA's Accelerated Approval pathway. The antiparkinsonian drug, tolcapone, was marketed in the European Union in 1997, but cases of fatal hepatotoxicity led to its marketing authorization being suspended the following year. 23 Patients and physicians argued that some patients experience improved quality of life and pleaded to gain access to the drug while explicitly accepting the risk of hepatotoxicity. Their lobbying led to the lifting of the suspension in 2004. Both natalizumab (for patients with relapsing forms of multiple sclerosis) and alosetron (for irritable bowel syndrome) were voluntarily withdrawn by their manufacturers because of safety concerns. 23 However, the FDA recommended their reintroduction to the market at the request of patients and family members. The use of methods to support regulatory benefit-risk assessments that consider patient preferences explicitly might have lessened the likelihood of misaligned preferences between the various stakeholders. Benefit-risk assessments are a qualitative exercise grounded in quantitative evidence, but just because patient perspectives are subjective does not mean that they cannot be quantified and analyzed accordingly.
Our case study showed that although seizure freedom was ranked the most important treatment outcome by patients during qualitative interviews, patients also prioritized reduction in the risk of adverse effects, and the importance of these effects differed by patient subgroup. The results of the DCE, however, suggest that patients are willing to accept a reduction in the chance of remission in exchange for a reduction in the risk of adverse effects-and that they attach a higher value on improving risk reduction than benefit. Women of childbearing potential would accept 5% reduction in the probability of 12-month remission to reduce the risk of fetal abnormality by 1%. The potential for our method to identify both MAR, and willingness to forego benefit, is particularly important with respect to prescribing valproate, which is considered the most effective treatment for certain epilepsy syndromes, such as juvenile myoclonic epilepsy, but is associated with the highest risk of teratogenicity. 24 This study provides quantitative evidence on women's strength of preference to avoid valproate, which aligns with recent advice from regulatory authorities. [25][26][27] Understanding the balance between benefit and harm, and the extent to which women are willing to forego benefit for this reduction in harm, could assist in the development of recommendations of alternative treatments to valproate and inform parameters of equivalence for new drugs.
We are aware of previous use of DCE to elicit preferences for AEDs. [28][29][30][31] Lloyd et al. 28 found that UK patients were willing to forego 4.85% seizure control for a 1% reduction in risk of hair loss, 4.45% seizure control for a 1% reduction in risk of skin rash, and 1.37% seizure control for a 1-lb (0.45-kg) reduction in weight gain. Manjunath et al. 29 measured US patient preferences for add-on AEDs in terms of seizure frequency and AEs; and they found that seizure reduction was the top priority when ranked against the reduction or elimination of AEs. Powell et al. 30 compared patients' and neurologists' preferences for a carbamazepine pharmacogenetic testing service. Ettinger et al. 31 also conducted a DCE in patients and neurologists, and they found that, although both ranked seizure control most important, neurologists' preferences were more influenced by seizure reduction compared with adverse effects for patients. None of these studies, however, assessed preferences in the context of trial data pertaining to actual AEDs; in fact, we are unaware of any other studies that integrate benefit-risk preference weights, estimated using DCE methods, and trial data.   The key strengths of our study are in meeting the recommended qualities of patient preference studies specified by the CDRH 5 : (i) Patient centeredness: the DCE measured patient preferences for outcomes. Furthermore, the study encompassed the views of both patients and physicians at appropriate stages of developing the DCE, to ensure the hypothetical task was relevant to the context of clinical decisions. (ii) Representativeness of the sample and generalizability of results: the study used an appropriate sample size in each clinically well-defined group; however, there may be limitations in generalizability because of the sampling. (iii) Capturing heterogeneity of patients' preferences: we identified clinically important subgroups a priori and tailored individual DCEs in response to heterogeneity in preferences at the qualitative phase. (iv) Established good research practices by recognized professional organizations: we adhered to good practice guidelines. 14,32 (v) Effective communication of benefit, harm, risk, and uncertainty: attribute levels were displayed as natural frequencies with pictographs, as recommended. 5 (vi) Minimal cognitive bias: cognitive interviewing was used to minimize framing effects and choices ordered randomly. (vii) Logical soundness: we demonstrated a high level of internal validity with a dominance test. (viii) Relevance: potential attributes identified from validated outcome measures, reduced systematically using a qualitative study and defined by clinically meaningful thresholds. (ix) Robustness of analysis of results: analysis adhered to good practice guidelines 9,32 with uncertainty represented by bootstrapped confidence intervals. (x) Study conduct: trained and experienced researchers interviewed patients, and the DCE questionnaire was verified by patients and healthcare professionals. (xi) Comprehension by study participants: robust application of cognitive interviews and questionnaire piloting to ensure face validity and optimal comprehension choice tasks. 14 There were, however, limitations to our approach. Bias arising from patients self-selecting to complete the questionnaire will limit the generalizability of the results. We also acknowledge that although DCE is a recommended approach to benefit-risk assessment, other methods exist, including multicriteria decision analysis 33 and health outcome modeling. 8,34 Our methods highlighted differences in responses linked to the methods used. The ranking of single outcomes identified seizure remission as the most important, whereas the DCE identified that reductions in the risk of AEs were more highly valued than improvements in seizure control. Although ranking exercises do not consider trade-offs explicitly, DCEs benefit from capturing this to assess the MAR for a given level of benefit. DCEs are inherently limited by the potential for patients' stated preferences not reflecting their revealed preferences accurately and the cognitive burden placed on respondents, which increases rapidly with the number of attributes. Consequently, differences between studies in benefit and risk attributes are inevitable, [28][29][30][31] although in mitigation, our study used a robust approach to attribute selection. 5,14,32 In conclusion, our case study findings indicate that accounting for patient preferences, in addition to clinical variables, could lead to different treatment choices or regulatory decisions. The most substantive results for AEDs are that women of childbearing potential show a clear preference for lamotrigine over valproate, principally driven by a preference to avoid the risk of teratogenicity (major malformation rate, ≈11% 19 ). This is despite clinical trial results showing valproate to be significantly more effective than lamotrigine in generalized onset seizures, 17 and aligns with recommendations from regulators. 25,26 The study demonstrates how patients' preference for treatment benefits and risks can be incorporated into benefit-risk assessments by obtaining quantitative evidence using a DCE. Our method represents a transparent and valid approach to examining a patient-oriented perspective in support of regulatory assessments of medicines. 1 It also builds on previous research that highlighted that, although studies quantifying preferences may allow for formal evidence-based appraisal of benefit-risk values, more is required to ensure best practice and to develop methods that combined stated

Analysis of condiƟon
IdenƟfying and measuring what maƩers most to paƟents (e.g. findings of qualitaƟve interviews, ranking exercises)

Current treatment opƟons
Defining therapeuƟc context for benefit-risk assessments (e.g. idenƟfying important harms and benefits)

Benefit
IntegraƟng paƟent-reported outcomes and elicited preferences into benefit-risk assessments (e.g. DCE linked with clinical data)

Risk and risk management
Understanding and miƟgaƟng risks from paƟents' perspecƟves (e.g. by understanding the trade-off between benefits and risks) Stage 1 preferences and clinical data. 35 Certainly, drug regulators will need to be cognizant of the evolving methods of preference-based benefit-risk, including DCEs. 36

METHODS
The study comprised three stages ( Table 4): (i) identification of the most important outcomes of AED treatment using qualitative interviews and ranking exercises with patients and a focus discussion group meeting of physicians; (ii) elicitation of patient preferences for these outcomes using a DCE; and (iii) estimation of the expected utility associated with alternative AEDs by combining preference weights with data from a clinical trial. Ethical approval was granted by the UK National Health Service Research (reference number: 11/NW/0191).

Stage 1: benefit-risk outcome selection
The first stage in the method was to identify patients' most important outcomes of AED treatment for inclusion in the DCE. A qualitative study was designed according to good practice guidelines.14,32,37 All interviews were conducted in patients' own homes, and consent was requested to audiotape record for subsequent transcription. Patients were asked to consider predefined treatment outcomes, which were categorized according to whether they were treatment benefits, adverse effects, or life impacts, and selected from clinical trials16,17 and validated outcome measures.38-41 Patients were first asked to nominate any additional outcomes, on the basis of their own experiences, that they considered were missing from each category. Next, they were asked to select the outcomes that were most important to them from each category. Finally, considering all the outcomes they had selected, across the three categories, they were asked to choose their top four, and rank them in order of importance.
The face validity and plausibility of the 10 most important attributes overall were examined using cognitive interviews with patients and a meeting with prescribing physicians. The meeting of physicians responsible for prescribing AEDs to adults with epilepsy was convened at the Walton Centre National Health Service Foundation Trust. Participants were first asked to rank the 10 treatment outcomes that patients had considered the most important. They then participated in semistructured discussions to share their practical experience of discussing treatment outcomes with patients and, in particular, their distinction between "adverse effects" and "life impacts" relating to AEDs. They were asked to independently record  the frequency and seriousness at which an AE becomes a "clinically important" AE that required a change in treatment. For example, clinically important depression was described as "low mood which often affects activities of daily life." The purpose of this exercise was to ensure parity between DCE descriptions of treatment outcomes and clinically important AE data recorded in clinical trials,16,17 to ensure optimal integration of preferences with clinical evidence. Prescribers were also asked to draw on their experience of communicating benefit-risk to patients and to provide feedback on the formatting of the patient DCE questionnaire. Discussions were audiotape recorded, and ranking results were noted in workbooks that were self-completed during the session.

Stage 2: DCE
The DCE was conducted, analyzed, and reported according to good practice guidelines.32,42 The five treatment outcomes ranked highest by patients, and which were considered clinically plausible, were included as attributes in the DCE. Each attribute was assigned two levels on the basis of data from a clinical trial16,17 and information on the risk of AED-related fetal abnormality available to patients on the Epilepsy Action charity website at the time of the survey. Levels represent the frequency of events observed in clinical trials and were presented in the questionnaire as a "1 in X people experience…", supplemented with pictograms that used a traffic light color coding for positive and negative effects.
The DCE consisted of eight binary choice scenarios ( Figure S1), which asked: Which medication would you prefer? A partially dominant choice was used to test the internal validity of the DCE, in which "medication A" had a higher chance of remission and lower risk of AEs; therefore, it was assumed respondents would prefer this option.

Patient DCE questionnaire
Adults self-reporting as being aged ≥18 years and diagnosed as having epilepsy by a physician were eligible to complete the survey. Respondents were required to consent to participate in the study before they accessed the questionnaire, and there was no compensation for their time. Recruitment was via publicity from the charity, Epilepsy Action, an advertisement in local press, and posters in 113 National Health Service outpatient clinics across England and Wales. The survey was implemented online (Snap Surveys, London, UK) between June and October 2013 and hosted by the Epilepsy Action website. Estimated completion time was 30 minutes. Target sample size was a minimum of 63 respondents per DCE, based on each main effect level of interest being represented across the design at least 500 times.43 A random sample of 25% was directed to a DCE designed to compare patients' with physicians' preferences for pharmacogenetic testing prior treatment with carbamazepine, which is reported elsewhere.30 Remaining respondents were directed to one of the two DCEs reported herein via a series of filter questions (DCE-2 for women of childbearing potential; DCE-1 for everyone else).
The questionnaire was piloted in a convenience sample of Epilepsy Action staff and volunteers, clinical and academic research staff, physicians who agreed to be contacted after the focus group, and members of our scientific advisory group. These individuals provided feedback on aspects such as the phrasing of the attributes and the selection criteria for women of childbearing potential, which was changed from being defined by age to a single question that asked, "Is there any chance, however remote, that you may become pregnant in the future?"

Statistical analysis
Responses to the DCE were analyzed in STATA, version 13 (StataCorp LP, College Station, TX) using a random-effects logit model that allowed for multiple observations (eight binary choices) from the same respondent. 44 The regression model estimated preference weights for each attribute that indicate the importance of attributes and the direction of effect. The coefficients (β) from the regression were used to calculate the MAR of an AE that respondents were willing to accept in exchange for a percentage point improvement in benefit (12-month remission) ( Figure S2).
Stage 3: estimating the utility for a given AED The expected utility model was parameterized using the preference weights elicited in the DCE and data on the number of events for each outcome assessed in the DCE, from the Standard vs. New Antiepileptic Drugs (SANAD) clinical trial of AEDs16,17 and from a meta-analysis of the effects of prenatal exposure to commonly prescribed AEDs on the prevalence of congenital malformations in the child19 ( Table 5). The β coefficients for each outcome (derived in stage 2) were multiplied by the corresponding clinical trial event rate and summed to estimate the total utility for each AED42 ( Figure S2). The ranking of AEDs by total utility was compared with ranking by time to treatment failure (because of inadequate seizure control or AEs), chosen to reflect the benefit-harm trade-off observed in the clinical trial.16,17

SUPPORTING INFORMATION
Supplementary information accompanies this paper on the Clinical Pharmacology & Therapeutics website (www.cpt-journal.com).