Additional Resources & FAQ’s about Conducting Patient Preference Studies

Additional Resources

In addition to the Framework and Overview, these additional resources may help you as you consider whether you might want to pursue a patient preference study.

Guidances
CDRH Guidance: Patient Preference Information – Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling

Consultants and Experts in Patient Preference Studies
Click here to view a list of patient preference experts and consultants.

First steps for sponsors initiating a patient preference study
Click here to view a helpful guide on the first qualitative steps for sponsors initiating a patient preference study

Frequently Asked Questions about Conducting Patient Preference Studies

Preferences are qualitative or quantitative statements of the relative desirability or acceptability of attributes that differ among alternative health interventions (See MDIC Framework section II, p. 19-20). “Attributes” of a medical intervention or treatment are features of the treatment such as effectiveness, safety, tolerability, means of implantation/use, duration of the effect, duration of use, frequency of use, lifestyle aspects of use, and other characteristics that impact benefit-risk considerations. While “preference” in conventional use indicates desired features, in the patient preference context, preferences can be for benefits that are desired or harms that are accepted. Preferences can be qualitative (e.g. Migraine patients care more about being able to function than relieving pain) or quantitative (e.g. For migraine patients, the preference changes associated with completely restoring function from having to stay in bed all day to being able to work or go to school is three times as large as the preference change associated with complete removal of severe pain). Preferences may also be assessed of caretakers, physicians, regulators or other stakeholders.

Patient preference studies are conducted to measure preferences. Typically, preference studies take place via facilitated small-group discussions or via on-line surveys.

Patient preference studies and PROs are both patient-centered measures; however, they are fundamentally different. Patient reported outcomes are measures of outcomes experienced by patients. For example, pain or the ability to do specific tasks are realized outcomes that can only be measured based on patient report. Preferences, in contrast, reflect what patients want and what they are willing to accept to get it. For example, patients may be willing to accept significant side effect risks for a treatment that reduces pain or improves their ability to do certain daily activities. Preferences are a measure of the extent to which patients are willing to trade off risks for treatment benefits. Some PRO instruments include preference questions in addition to measuring outcomes.

Coming soon.

Patient preference methodologies have been around for a long time. Different patient preference methods have evolved in different disciplines and have been used in different applications for decades. For example, health-state utility methods have been used in cost-utility analysis. Discrete-choice experiments have been used in transportation studies and environmental economics. Structured weighting methods have been used in multiple applications of decision analysis. The application of preference methods to heath, while not completely new, has begun to accelerate in recent years. These methods provide a means for engaging patients and providing systematic information from the patient perspective.

Patient preferences are statements made by individuals regarding the relative desirability or acceptability of a range of health experiences, treatment options, or health states (See MDIC Framework, section II). A greater understanding of patient preference of treatments is central to current models of shared patient-doctor decision making, treatment adherence, and improved outcomes. They are not, however, the only tool available to gain insight into patient perspectives. “Patient perspectives” refer to a type of patient input, or patient point of view, and includes information relating to patients’ experiences with a disease or condition and its management (See CDRH Guidance on Patient Preference). While patient preference information is a specific type of patient perspective, there is a broad range of approaches to soliciting patients’ points of view, including questionnaires, Quality of Life scales, focus groups, interviews and surveys. While useful and utilized, all of them provide different pieces of information with regards to the patient perspective.

Information from patients regarding their preferences for benefits and tolerance for risks in medical technologies can be useful across the medical device development lifecycle. Broadly, there are three areas where patient preference information can be particularly helpful to device companies and regulators.

Patient preference information can be useful throughout the medical device development lifecycle.

During ideation stage, medical device companies could use patient preference information to ascertain the potential usability of the device and to identify the unmet needs in device design.

  • Patient preference information can reveal the desirability of some specific device features over the alternatives.
  • Patient preference information can quantify how a patient is willing to uptake a probable risk for a stated benefit.
  • Patient preference information can estimate patients’ willingness-to-pay for a stated benefit or willingness-to-avoid for a probable risk.

See section IV in MDIC Framework for more information.

During the regulatory process, medical device companies can include patient preference information on device labeling, in addition to the totality of other evidences.

  • Information on device labeling can be helpful to decision makers such as health care professionals and/or patients when making decisions that involve difficult benefit-risk tradeoffs or treatment choices. Medical device companies should communicate patient information clearly, accurately and instructively in order to help health care professionals and/or patients to understand the potential benefits and risks of the device (See section IV in MDIC Framework)
  • For standard labeling, characteristics of patients who considered the device’s probable benefits to outweigh its probable risks should be included, hence, include a clear indication of the population for whom the device is appropriate. For instance, for an implantable device to treat knee pain and to improve knee function, patients with the highest pain and functional limitation may benefit more than the overall study population without any increase in adverse events.
  • For health care professional labeling, a summary of the patient preference study that describes the population under study, the method used to derive patient preferences such as attributes and levels, and the results of the study, should be included.
  • See the CDRH Guidance on Patient Preference for additional information.

During post-market monitoring, medical device companies could use patient preference information to evaluate whether patient stated preferences align with patient revealed preferences, hence, to understand patient communicated outcomes of the device.

Other potential uses for patient preference information include:

  • Marketing: Patient preference information can be used to identify market opportunities, target patient populations for a technology, identify key attributes that are important to patients and develop patient-facing marketing materials to encourage adoption of the technology.
  • Reimbursement: Patient preference information may be useful in reimbursement, but at present, such information is not explicitly sought or considered. As preference information is developed for regulatory purposes, it may be useful for that information to be published in the literature that supports the use of the technology. Published studies may be especially helpful if preference studies identify a subset of patients that prefer a less expensive technology with a different benefit-risk profile than the average patient prefers.

For more information about uses for patient preference information, check out section VII in the MDIC Framework.

Patient preference studies can provide a wide range of outcome-related data. At a fundamental level, patient-preference studies can provide three type of information: what is important to patients, how important different treatment features are to patients, and the tradeoffs patients are willing to make among treatment features. This type of information can be used to inform the selection of clinical-study endpoints and the necessary effect sizes of clinical-study endpoints. This type of information can also be used to understand the extent to which patients are willing to tolerate the risks associated with alternative treatments. Finally, patient-preference studies can be used to understand the diversity of patient preferences or identify segments of the patient population with different benefit-risk preferences.

While the techniques for preference assessment have a long history, their application to treatments for health care is relatively new. Consequently, there has been limited opportunity for patient preferences to have an impact on regulatory activities. In early 2015, FDA CDRH approved a device to treat obesity that had missed one its co-primary efficacy endpoints. The approval was in part based on an obesity device patient preference study CDRH had conducted (See Ho et al in Surgical Endoscopy and FDA Press Release). To date, while preference studies have been included in a very few submissions to FDA CDER, there is little information on the role these studies may have had.

With the release of FDA CDRH’s guidance on the use of patient preferences, sponsors have begun discussions with CDRH about potential preference studies. Additionally, FDA CDER discussions since the release of the PDUFA VI draft goals have suggested an interest in the role of patient preference studies in determining which endpoints are of greatest importance to patients and how patients trade off benefits and risks.

Patient preferences are not the same as a benefit-risk assessment. Patient preference studies provide information that may inform a benefit-risk decision, though not all benefit-risk decisions benefit from preference information (See Question 6 for a description of the conditions for which a preference study may be of value).

Assessing whether benefits outweigh risks depends on both the frequency of benefit and risk events and the severity, or clinical impact, of these events. Typically, clinical or epidemiological data indicate how many benefit and harm events are caused by different treatments, or they give the change in other measures of efficacy and safety. In many cases, the benefit-risk decision is straightforward. In preference sensitive decisions (See MDIC Framework, section II, p. 22), clinical judgment is often sufficient to assess the severity of these events and render a defensible benefit-risk decision. In cases where clinical judgment is insufficient or challenging to defend, a preference study can provide the patient perspective needed for a defensible and transparent assessment.

There are numerous means to apply patient preferences to benefit-risk. They can help determine which endpoints are most important to patients and which particular ranges for values of these endpoints are most important to patients, which can help in designing target product profiles and clinical trials. Preference studies can give the maximum acceptable risk for a given benefit or the minimum required benefit for a given risk (See MDIC Framework, section II, p 25-27), providing a straightforward means to determine whether benefits outweigh harms. Preference studies can also measure the heterogeneity of preferences in a population, potentially showing the existence of subgroups with different takes on benefit-risk. Finally, comparing preferences between different stakeholders, such as patients vs. physicians, can provide insight into why they may make different treatment decisions.

The MDIC Patient Centered Benefit-Risk Resources page has a number of useful resources for initiating a patient preference study. An MDICx webinar on practical considerations for measuring and integrating patient preference information into your regulatory submission offers useful information for sponsors looking to conduct a patient preference study.

Some key questions to consider when getting started:

  • How much is currently known about the benefits and harms of the technology?
  • Knowing these benefits and harms, is there preference-sensitive decision that patients face regarding the use of the technology?
  • What type of patients should be surveyed?
  • Have prior preference studies been done in the same or similar indication?
  • How would the results of a patient preference study help better frame the benefit-risk assessment of the technology in a specific patient population?
  • What are the budget and time available for the study?

In doing an initial assessment of these questions, it might be helpful to involve people with different perspectives on the potential value of patient preference information on regulatory benefit-risk assessment, including:

  • Internal expertise:
    • Clinical
    • Patient Reported Outcomes
    • Health Economics
    • Regulatory
    • Medical Affairs
    • Commercial
  • External expertise
    • Experienced research groups
    • Patient panel companies
    • Patient advocacy groups
    • Regulatory agencies (e.g. CDRH, EMA)

Experienced research groups can be particularly helpful in scoping out the requirements and budgets for a patient preference study.

Should the patients in the study reflect the intended indicated population?

Yes, where reasonable, participants enrolled in a preference study should reflect the intended indicated population. Patients’ preference may vary with the severity of the disease or condition, acuteness or chronicity of the disease, or availability or lack of alternative options. It is important to account for these variations when recruiting. As recommended by the FDA in their guidance document, the selected participants should be representative and well-informed so that results obtained can be generalized to the population of interest.Sometimes this may require oversampling underrepresented populations to ensure that there is sufficient diversity in the sample to estimate whether patients with different characteristics have different preferences.

Sometimes it may be difficult to achieve a representative sample. For instance, this may require clinical data to confirm diagnosis. In that situation, it may be necessary to work with clinical sites to ensure the eligibility of patients. In other instance, the intended population may be too ill to obtain an adequate sample size, or it may be too costly or time consuming to find patients with a clinician diagnosis of the condition of interest. In cases such as these, it will be important to minimize the sample size required for the study by, for instance, designing a study or adopting methods that can provide reliable preference information from smaller patient samples. Nevertheless, it may be necessary to expand the inclusion/exclusion criteria for participation resulting in a larger pool of participants from which to choose. In this instance, it will be important to be transparent about and test the impact of the uncertainty introduced by a lack of participant specificity on the results of any treatment evaluation.

Are the patients in the clinical trial the right patients for the patient preference study?

When deciding if one should elicit preferences from clinical trial patients, researchers should consider how similar the sample enrolled in the trial is to the population of interest. Researchers should also consider whether there is sufficient diversity in the characteristics of patients enrolled in the clinical trial to facilitating the effect of patients’ characteristics on preferences. Capturing preferences from patients enrolled in a clinical trial may not elicit results which are generalizable to the larger population of interest because clinical trials enroll participants using strict inclusion and exclusion criteria, resulting in a more narrowly defined group of participants than might be of interest. Other considerations when assessing the relevance of the trial population include: the size of the trial sample – if the sample is too small the preference study may not generate precise parameter estimates.

Sponsors should select a patient preference assessment method based on the needs of their patient preference study and the information sponsors hope to gain from the study. One resource for considering and selecting a patient preference assessment method is the MDIC Catalog of Methods for Assessing Patient Preferences for Benefit and Harms of Medical Technologies, the primary appendix to the MDIC report “A Framework for Incorporating Information on Patient Preferences Regarding Benefit and Risk into Regulatory Assessments of New Medical Technology.” If you partner with an experienced research group on your patient preference study, they will make recommendations on a method based on your study goals.

It is important to partner with patient groups when gathering patient-provided information. While a company could design and conduct a preference study without direct patient group input, there is considerable advantage to having a wide perspective on design elements for the survey that patient groups can provide. Other stakeholders reviewing the study results may also look more favorably on a study conducted with patient group involvement.

To start, research and identify which patient group(s) for a particular disease is best to work with. Which group is most active within a community, has a respected reputation, and expertise in your particular area? The Clinical Trials Transformation Initiative (CTTI) has developed a tool to help sponsors identify organizational expertise and assets that can be useful in targeting which organizations might be best equipped to assist with specific projects. Faster Cures has also recently launch an online directory of patient groups. Engage as early as possible and maintain open lines of communication. Clearly state your goals and why this partnership is important. Ensure patient partner input is sought and used at key steps in the survey design and deployment process.
The planned and final contractual relationship between the sponsor and patient groups should be made clear and transparent in discussions with the patient group, patient group members and presentations/publications on the work.

Note that while patient groups and the people they serve understand medical product development is a complicated and rigorous process, there will be varying levels of familiarity with specific stages and steps. It may be necessary to help the patient group learn what they need to know and stay engaged throughout the process and how their input is being used. If available, provide patient groups with training and easy to understand materials with adequate time to review. Explain why patient community perspectives are being sought and how they are being incorporated throughout the process.

Patient groups can help you capture the breadth of patient views, preferences and experiences. The goal of the partnership should be meaningful patient engagement. Meaningful means direct relationships and partnerships that are bidirectional, reciprocal, and continuous. Communications are open, honest, and clear. Engagement goals, participants, methods, desired impacts, and actual impacts are clearly outlined and transparent. Transparency is of critical importance to establish trust. A true partnership is created by engaging, informing, and actively listening to the patient group at every point of development – from conception to post marketing surveillance.

  • Regulatory
    • There is increasing attention on and rising demand for patient preference data, both from the FDA, and from departments and individuals within medical device companies (e.g., reimbursement and market access). There is a realization that patient preference data will not only help with the HTA regulatory submission for any new technology, but also will be a powerful mechanism to generate evidence from the end user of the medical device product.
  • Health Economics and Outcomes Research (HEOR)
    • Researchers from HEOR departments are interested in patient preference information to better understand the preferences and needs of patients. HEOR departments will also be part of the education and awareness process for generating and disseminating the patient preference data, in responding to rising demand and unmet needs.
  • Medical Affairs
    • Medical Affairs teams will ask benefit-risk and patient preference questions similar to clinical teams, but generally in the context of post-approval benefit-risk evaluation or pre/post-approval health technology assessment. There is increasing interest in using preference data in both applications.
  • Commercial and Marketing groups
    • Preference studies are often used to rough out medical need and potential market shares of a prospective treatment. However, these studies are not always conducted in a manner that will permit the use of the results when answering clinical benefit-risk questions.

Preference studies are publishable in peer-reviewed journals. They have appeared in both clinical journals as well as those focused on patient preference research, such as Value in Health, The Patient, Patient Preference and Adherence, PharmacoEconomics, and Patient Preference and Adherence. Value in Health published a special themed section titled “Incorporating Patient Preferences Into Regulatory Decision Making”, which included several preference studies (Value in Health, September 2016–October 2016, Volume 19, Issue 6, p699-908).

Nevertheless, there can be challenges in publishing these studies. Clinical journal reviewers are likely to be unfamiliar with preference elicitation methods and therefore unwilling to consider such papers for publication, or they may be skeptical of certain methods, such as the hypothetical nature of the elicitation method used in conjoint/DCE studies, questioning the validity or usefulness of the data. On the other hand, patient-preference-focused journals are likely to involve reviewers who are experts in preference research with high standards and expectations regarding design, analysis and reporting, in this quickly evolving field of research.

Studies that have greater difficulty getting published are those that omit important attributes or appear biased in terms of attribute or level selection, are narrowly focused in terms of treatment or study population, or appear to be commercially rather than scientifically driven. Additionally, papers that do not adequately describe important aspects of the study (e.g., research question, rationale for selected preference elicitation method, methods for survey design, attribute selection, and analysis, implications of findings, etc.) have more difficulty being accepted. The ISPOR Task Force has published a checklist for conjoint/DCE studies outlining good research practices for study design and analysis as well as reporting in publications. MDIC also has developed a catalogue of preference research methods that can inform optimal preference study design and FDA CDRH guidance “Patient Preference Information – Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling” provides suggestions and recommended qualities of a patient preference study. Additionally, sponsors may meet with FDA CDRH to get advice on prospective preference studies. These resources can be used to guide researchers as they design their studies and prepare their papers for publication, ensuring adherence to good practice for study implementation and reporting, and increasing the likelihood of acceptance.

The cost of patient preference study is commonly in the range of $150-400K, depending on the target population, number of subgroups of interest, potential need for qualitative research and target countries. Subject recruitment is a key cost driver, with harder to reach populations requiring more resources. If there is interest in examining preferences by specific subgroups, sample size will necessarily increase, as will the associated costs for recruitment, data collection and analysis. Global studies also are more labor intensive and have the added cost of translation of various study materials, including patient interview transcripts developed during the survey design phase as well as the preference survey itself. Additional qualitative research may be needed for novel indications or rare diseases, which can add time and cost to the preliminary survey design steps.

As the FDA, in particular CDRH, captures the headlines with their Patient Preference Initiative, other efforts, such as those in Europe, go relatively unnoticed. Below is an overview of these efforts. Developments are currently being made in the use of patient preferences for the evaluation of pharmaceuticals, which we have summarised. A more systematic effort to identify and describe the use of preferences in the EU is currently being undertaken by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR).

Approval
EMA efforts to incorporate quantitative preference data into approval decisions was initiated with the Benefit-Risk Assessment (BRA) Methodology project. This reviewed methods for undertaking quantitative BRA and piloted the most promising. It recommended that where the benefit-risk balance of a drug is marginal, multi-criteria decision analysis (MCDA) could support the BRA. However, efforts to collect preferences focused mainly on members of the decision making committee (the Committee for Medicinal Products for Human Use). More recently, the EMA has piloted efforts to collect patient preference data, eliciting preferences using a variant of the swing weighting method (EMA pilot). Beyond quantitative patient preferences, the EMA has recently implemented a range of efforts to facilitate patient engagement in the decision making process. For a history of this effort, see here.

Reimbursement
Health technology assessment (HTA) in the EU is undertaken at a member state level. A number of member states are using methods to incorporate quantitative preference data into decision making; however, these efforts do not always focus on the patient.

The country with the greatest emphasis on patient preferences is Germany. The collection of quantitative data on patient preferences is part of the Institute for Quality and Efficiency in Healthcare (IQWiG)’s General Methods guide, and is required when a drug undergoes an economic evaluation. Preference data are used to combine multiple outcomes into an overall assessment of drugs in a therapy area, which in turn is used to construct an efficiency frontier. To inform the development of this guidance, IQWiG piloted two methods for eliciting patient preferences, the analytical hierarchy process (AHP) and discrete choice experiment (DCE). More details on the pilots are available here: AHP studyDCE study.

Other efforts by HTA bodies to incorporate preferences into decision making have focused on the preference of a broader set of stakeholders. These include:

  • Belgium: An MCDA has been developed to support HTA, with preferences collected from the general population using a DCE. More details can be found here.
  • Lombary, Italy: An MCDA is used to support HTA, with preferences collected from committee members using direct elicitation methods. More details can be found here.
  • Hungary: An MCDA is used to support HTA, with preferences collected from committee members using direct elicitation methods. More detail can be found here.

Future developments
The agenda for the future use of patient preference data in both approval and reimbursement decisions in the EU will be strongly influenced by PREFER. PREFER is a research project carried out as a public-private collaboration under the Innovative Medicine Initiative (IMI). It was kicked off in October 2016, and its objective is to pilot patient preference studies in diverse settings to gain insight into how such studies can add value to policy decisions. The findings from these studies will form the basis of recommendations for the development of guidelines on how and when to use patient perspectives in BRA and HTA. More details about PREFER can be found here.

Efforts to consider how to involve patient preferences in decisions are also ongoing at a member state level. For instance, in the UK, the National Institute for Health and Care Excellence (NICE) has partnered with Myeloma UK to award a 2-year grant to explore best practice in capturing information about patient preferences. The aim of the study is to evaluate how to improve how patients’ views are used to determine which myeloma treatments are made available to patients. More details can be found here.

Videos

In addition to the Framework and Overview, these additional resources may help you as you consider whether you might want to pursue a patient preference study.

Guidances
CDRH Guidance: Patient Preference Information – Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling

Consultants and Experts in Patient Preference Studies
Click here to view a list of patient preference experts and consultants.

First steps for sponsors initiating a patient preference study
Click here to view a helpful guide on the first qualitative steps for sponsors initiating a patient preference study