Advertisement
Mayo Clinic Proceedings Home

Patient Reported Outcomes Have Arrived: A Practical Overview for Clinicians in Using Patient Reported Outcomes in Oncology

Published:September 25, 2019DOI:https://doi.org/10.1016/j.mayocp.2019.04.005

      Abstract

      Ensuring that the patient’s voice is routinely incorporated in all aspects of health care in oncology is essential to provide quality care. Patient reported outcomes (PROs) are standardized measures that are used to obtain the patient’s perspective and are increasingly used in all aspects of health care to ensure optimal delivery of patient-centered care. The US Food and Drug Administration encourages that PROs be used in studies for label indications. There are no uniform standardized methods to use PROs nor is there consensus on which PROs are best for regulatory approval, comparative effectiveness research, toxicity assessment, health-related quality of life, or symptom monitoring. For this review, we conducted a literature search using PubMed and Google Scholar, and herein summarize the evidence related to the use of PROs in clinic care and research. Using valid, reliable, and easily interpretable PROs developed in comparable populations will provide the most useful results. Various ways that PROs can be used successfully in oncology have been exemplified in this overview to provide clinicians and researchers practical guidance.

      Abbreviations and Acronyms:

      CDER (Center for Drug Evaluation and Research), CER (comparative effectiveness research), CTCAE (Common Terminology Criteria for Adverse Events), EORTC (European Organization for Research and Treatment of Cancer), FDA (Food and Drug Administration), HRQOL (health-related quality of life), PRO (patient reported outcome), PRO-CTCAE (Patient Reported Outcomes Common Terminology Criteria for Adverse Events), PROMIS (Patient Reported Outcomes Measurement Information System), PS (performance status), RCT (randomized clinical trials), SF-36 (short-form 36 questionnaire)
      Article Highlights
      • Patient reported outcomes (PROs) can be used in oncology toward designing clinical trials, for US Food and Drug Administration (FDA) drug approval and label claims, comparative effectiveness research, and to improve clinical care.
      • More directly, the use of PROs can improve physician-patient communication, patient quality of life, and survival.
      • PROs used for drug approval claims should be very specific and working with the FDA to ensure that a label claim would be possible once results are available.
      • PROs used in clinical studies can go beyond health-related quality of life assessment and be used for stratification, performance status evaluation, and toxicity assessments.
      • The infrastructure required for implementation and lack of consensus on PROs are major barriers in its use at the current time at individual centers.

      What Is a Patient Reported Outcome?

      A patient reported outcome (PRO) describes the impact of health-related conditions and/or treatments provided by patient self-report without introduction of bias by any third party.
      • Patrick D.L.
      • Burke L.B.
      • Powers J.H.
      • et al.
      Patient-reported outcomes to support medical product labeling claims: FDA perspective.
      PROs have been included in the practice of medicine long before there were instruments to measure them, simply by asking patients to report on their illness and its effect on their lives. However the value of PROs were not well understood until the late 1970s and 1980s with the Rand Health Insurance Experiment and the Medical Outcomes Study showing that interpreting traditional clinical outcomes within the context of provider and patient reports could improve patient care.
      • Brook R.H.
      • Ware J.E.J.
      • Davies-Avery A.
      • et al.
      Overview of adult health measures fielded in Rand’s health insurance study.
      Since then, there has been a greater focus on PROs and a wide array of measurements developed to collect PROs, from patient diaries to multidimensional multi-item questionnaires. Unfortunately, the availability of numerous standardized questionnaires that capture PROs for symptom status, health-related quality of life (HRQOL), physical function, social function, and well-being, and the vast heterogeneity of PRO measures have only added complexity to the successful use and/or implementation in clinical research and health care settings. This has created a practical barrier to accepting the value and routine use of PROs waning enthusiasm for PROs in the medical community.
      A resurgence of focus on PROs occurred in the early 2000s when The Institute of Medicine’s publication of the Quality Chasm highlighted patient-centeredness as one of the six aims for quality health care.
      Institute of Medicine
      Crossing the quality chasm: a new health system for the 21st century.
      This publication documented a clear need to incorporate the patient back into the central focus of care and spurned national changes in health care.
      Institute of Medicine
      Crossing the quality chasm: a new health system for the 21st century.
      Similarly, a growing body of evidence in the cancer literature showed that PROs may provide stronger information than physician-reported outcomes, such as performance status, a well-known prognostic factor used to stratify patients and affect eligibility in clinical studies.
      • Schnadig I.D.
      • Fromme E.K.
      • Loprinzi C.L.
      • et al.
      Patient-physician disagreement regarding performance status is associated with worse survivorship in patients with advanced cancer.
      • Warsame R.
      • Kumar S.K.
      • Gertz M.A.
      • et al.
      Hematology patient reported symptom screen to assess quality of life for AL amyloidosis.
      Moreover, a meta-analysis of cancer clinical trials where PROs were assessed showed that the majority of studies using PROs showed an association between PROs and survival providing powerful prognostic information.
      • Gotay C.C.
      • Kawamoto C.T.
      • Bottomley A.
      • Efficace F.
      The prognostic significance of patient-reported outcomes in cancer clinical trials.
      Another meta-analysis of 30 randomized clinical trials (RCTs) from the European Organisation for Research and Treatment of Cancer (EORTC) between 1986 and 2004 showed that HRQOL provided prognostic value for survival in patients with cancer even after adjusting for various sociodemographic and clinical variables including performance status (PS).
      • Quinten C.
      • Coens C.
      • Mauer M.
      • et al.
      Baseline quality of life as a prognostic indicator of survival: a meta-analysis of individual patient data from EORTC clinical trials.
      The routine collection of PROs and emphasis on incorporating the patient’s voice into health care is ideal, but several challenges exist for clinicians and researchers to achieve this goal. There are a myriad of ways PROs can be used, both in practice and for research; however, the lack of consensus on the best instruments, and confusion about how best to use and interpret measures exist. This review provides high-level guidance to clinicians about the various ways to use PROs practically and successfully in the clinical setting with a focus in oncology, but the guiding principles are the same for any specialty.
      We conducted a systematic literature review using PubMed and Google Scholar limited to articles published in English between January 1, 1980, and June 30, 2018. The following key words and their combination were searched: patient-reported outcomes, clinical trial, oncology, hematology, clinical practice, quality of life, health-related quality of life, health outcomes, progression-free survival, overall survival, toxicity, response, and comparative effectiveness research.

      How to Use Pro for Us Food and Drug Administration Drug Approval and Label Claims?

      To fully understand how to gain drug approval using PROs, it is important to understand the process by which drugs are approved by the US Food and Drug Administration (FDA). Firstly a panel of scientists, physicians, chemists, pharmacologists, and biostatisticians review a potential therapeutic agent by the FDA Center for Drug Evaluation and Research (CDER). Pharmaceutical companies work closely with the FDA CDER from the time there is promising in vitro evidence of a drug’s potential to align their studies for the best chances of a successful review if primary endpoints are met. An abbreviated summary of the steps of the FDA drug review process are as follows:
      • 1.
        Submit an investigational new drug application to CDER.
      • 2.
        Establish criteria for drug primary endpoints for upcoming protocol of clinical trial.
      • 3.
        Proceed with clinical trial (phase I-III).
      • 4.
        If primary endpoints are met, then submit new drug approval application for drug approval and label claims.
      • 5.
        If primary endpoints not met, withdraw application.
      • 6.
        A review meeting of FDA is held and application is approved or denied.
      • 7.
        If application is denied, and investigators/pharmaceutical industry do not agree with decision, the application is brought up to the drug advisory committee for review, to assess if risk outweighs the benefit. A final decision is made. (Breakthrough designation is an accelerated approval process for a drug that is in early phases generally that has had significant response rates with limited toxicity; it will allow for patients to get access to a drug earlier. Usually this will require confirmatory randomized phase III study in the future.)
      Clinical trials are increasingly expected to incorporate the patient’s perspective for study outcomes.
      • Scoggins J.F.
      • Patrick D.L.
      The use of patient-reported outcomes instruments in registered clinical trials: evidence from ClinicalTrials.gov.
      This is reflected in the increase in use of PRO measures in clinical trials from 14% between 2004 and 2007 to 27% in 2013.
      • Vodicka E.
      • Kim K.
      • Devine E.B.
      • Gnanasakthy A.
      • Scoggins J.F.
      • Patrick D.L.
      Inclusion of patient-reported outcome measures in registered clinical trials: Evidence from ClinicalTrials.gov (2007-2013).
      PROs collected during clinical trials are expected to be standardized and assess a specific concept such as a symptom, function, or overall HRQOL. However, there is no standardization on PRO measures or consensus on which PRO measures are appropriate for clinical trial use for various treatments/trials. Adding to that challenge is the fact that there are numerous validated measures for any given PRO assessment.
      • Fitzpatrick R.
      • Davey C.
      • Buxton M.J.
      • Jones D.R.
      Evaluating patient-based outcome measures for use in clinical trials.
      Selecting the appropriate specific PRO as an endpoint and the appropriate instrument for drug approval is a daunting task. For some classes of drugs or diseases, PROs may be the only source of information in which efficacy can be based on, for example, the symptomatic management of headaches or pain. Whereas for other therapies, PROs may add to the body of evidence to support its approval, label claim, marketing, and/or understanding of the effect on patients’ wellbeing, for example, oncologic therapeutics. The first major report of a PRO as an endpoint in a clinical trial was reported in the New England Journal of Medicine in 1986.
      • Croog S.H.
      • Levine S.
      • Testa M.A.
      • et al.
      The effects of antihypertensive therapy on the quality of life.
      This study involved a double-blind RCT of differences between antihypertensive medication impacts on HRQOL. This article had significant economic impact and galvanized the pharmaceutical industry to use PROs to support drug approval and use HRQOL in labeling and marketing.
      Since then, the FDA has released guidance for use of PROs in clinical trials in 2009 titled “Guidance for Industry: Patient Reported Outcome Measures Use in Medical Product Development to Support Labeling Claims.”
      • U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research
      • U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research
      • U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health
      Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance.
      This guidance was created to act as a blueprint for using PROs in clinical trials, so that appropriate, validated, and reliable measures were used to increase the likelihood of a favorable review by the FDA. In addition, it would avoid broad claims of improved HRQOL based on inadequate measures.
      • U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research
      • U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research
      • U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health
      Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance.
      The FDA also provides the clinical outcomes assessment qualification to help determine if the measure used is an adequate assessment for use in the study. Although these were created to aid industry and clinician investigators on what to incorporate in the protocol as far as PROs are concerned, its rigid expectations has resulted in less usage.
      • Demuro C.
      • Clark M.
      • Doward L.
      • Evans E.
      • Mordin M.
      • Gnanasakthy A.
      Assessment of PRO label claims granted by the FDA as compared to the EMA (2006-2010).
      • Fehnel S.
      • DeMuro C.
      • McLeod L.
      • Coon C.
      • Gnanasakthy A.
      US FDA patient-reported outcome guidance: great expectations and unintended consequences.
      There has been significant work on how many new drugs have been approved that have at least one indication based on a PRO by the FDA in the last 2 decades by Willke
      • Willke R.J.
      Measuring the value of treatment to patients: patient-reported outcomes in drug development.
      and Gnanasakthy et al.
      • Gnanasakthy A.
      • Mordin M.
      • Clark M.
      • DeMuro C.
      • Fehnel S.
      • Copley-Merriman C.
      A review of patient-reported outcome labels in the United States: 2006 to 2010.
      • Gnanasakthy A.
      • Mordin M.
      • Evans E.
      • Doward L.
      • DeMuro C.
      A review of patient-reported outcome labeling in the United States (2011–2015).
      Their work showed that in the following timeframes PRO use in new drug approval has steadily declined: between 1997 and 2002 there was 30% PRO labeling; between 2006 and 2010 there was 24% PRO labeling; and most recently between 2011 and 2015 only 16.5% of new drugs had a PRO included in labeling. The trend is similar in oncology: specifically, there were only 3 of 40 (7.5%) oncologic drugs approved between 2010 and 2014 that had a PRO-related label (abiraterone acetate, ruxolitinib phosphate, and crizotinib).
      • Gnanasakthy A.
      • DeMuro C.
      • Clark M.
      • Haydysch E.
      • Ma E.
      • Bonthapally V.
      Patient-reported outcomes labeling for products approved by the office of hematology and oncology products of the US Food and Drug Administration (2010-2014).
      • Basch E.
      • Geoghegan C.
      • Coons S.J.
      • et al.
      Patient-Reported outcomes in cancer drug development and US regulatory review: perspectives from industry, the Food and Drug Administration, and the patient.
      Thus, the evidence would suggest that the stringent criteria posed by the FDA have been challenging to implement and uphold due to a wide variety of opinion and practice of the degree to which the guidance is taken literally.
      To be successful for regulatory drug approval, it will be important to be critical at the time of protocol development. Before including a PRO measure in a clinical trial, one should consider if it is appropriate to use a PRO endpoint at all; that is, there should be a scientific hypothesis that the inclusion of that PRO will help answer as with any other endpoint. Including PROs for the sake of completion is not effective and often does not result in adding to the likelihood of approval or label claims. Defining and selecting the appropriate tool is difficult; therefore, a multidisciplinary taskforce was established including scientists, regulatory agencies, and industry called the Harmonization Group to address this particular issue. When is it appropriate to include a PRO in a trial? This group outlined the times when inclusion of PROs would add value to the drug development or clinical trial process.
      • Acquadro C.
      • Berzon R.
      • Dubois D.
      • et al.
      Incorporating the patient’s perspective into drug development and communication: an ad hoc task force report of the patient-reported outcomes (PRO) Harmonization Group meeting at the food and drug administration, February 16, 2001.
      They were described as follows: (1) if the patient’s self-report is the primary or sole indicator of disease activity; (2) if the treatment may have small impact on survival but significant impact positive or negative on HRQOL; (3) if the treatment may adversely affect patient functionality or well-being; (4) if the treatment arms offer equal efficacy but differential PRO benefits; or (5) if treatment-related decisions are based on a combination of parameters.
      • Acquadro C.
      • Berzon R.
      • Dubois D.
      • et al.
      Incorporating the patient’s perspective into drug development and communication: an ad hoc task force report of the patient-reported outcomes (PRO) Harmonization Group meeting at the food and drug administration, February 16, 2001.
      Using these guidelines can help in selection of the right instrument for the right question for the right clinical study. There are numerous PROs that have been studied in oncology, both general and disease-specific, and selecting the PROs that best apply to the population, disease, needs of the trial, and that are ideally comparable to other studies will increase the successful use of data.
      An example of using PRO for a trial is shown by the first oncologic drug to be approved with a PRO solely as the endpoint supporting label indication; ruxolitinib in myelofibrosis from the Controlled Myelofibrosis Study With Oral JAK Inhibitor Treatment-I (COMFORT) trial.
      • Mesa R.A.
      • Gotlib J.
      • Gupta V.
      • et al.
      Effect of ruxolitinib therapy on myelofibrosis-related symptoms and other patient-reported outcomes in COMFORT-I: a randomized, double-blind, placebo-controlled trial.
      The constitutional symptoms of myelofibrosis have a huge impact on HRQOL in this disease, and are associated with shortened survival. In this study, multiple validated PRO tools were used (Myelofibrosis Symptom Assessment Form version 2.0, EORTC Quality of Life Questionnaire Core 30 (EORTC QLQ-C30), and the Patient Reported Outcomes Measurement Information System [PROMIS] Fatigue Scale) for assessing symptoms in myelofibrosis (fatigue, night sweats, pruritus, abdominal discomfort, early satiety, bone/muscle pain, inactivity) and compared symptoms between patients who received ruxolitinib to placebo. There was a dramatic improvement in clinically meaningful symptoms in patients who received the drug, and a worsening of myelofibrosis-related symptoms such as night sweats, abdominal pain, and HRQOL for patients on placebo.
      • Mesa R.A.
      • Gotlib J.
      • Gupta V.
      • et al.
      Effect of ruxolitinib therapy on myelofibrosis-related symptoms and other patient-reported outcomes in COMFORT-I: a randomized, double-blind, placebo-controlled trial.
      This study worked with the FDA to ensure that these endpoints would be appropriate for their label approval, and could be a framework for successful use of PRO in drug approval for other researchers. The PROMIS Fatigue Scale became FDA-approved as a clinical outcomes assessment in myelofibrosis based on this trial. Another consideration is that PROs can be combined with other meaningful endpoints as a composite for approval. For instance, Burris et al
      • Burris 3rd, H.A.
      • Moore M.J.
      • Andersen J.
      • et al.
      Improvements in survival and clinical benefit with gemcitabine as first-line therapy for patients with advanced pancreas cancer: a randomized trial.
      used the composite of patient reported pain and analgesic consumption, PS, and weight loss for successful approval of gemcitabine in pancreatic cancer.

      How to Design Better Clinical Trials Using Pros?

      Current classical clinical trial endpoints in oncology include overall survival, progression-free survival, disease-free survival, and response rates.
      • Wildiers H.
      • Mauer M.
      • Pallis A.
      • et al.
      End points and trial design in geriatric oncology research: a joint European Organisation for Research and Treatment of Cancer-Alliance for clinical trials in oncology–International Society of Geriatric Oncology position article.
      However, these endpoints may not always be the best reflection of the true risks, benefits, quality of life (QOL), and cost for patients. Using PROs may be a unique opportunity to complement these classical endpoints and better stratify patients, assess toxicity, predict potential outcomes (function and quality of life), and symptom monitoring for clinical trials. Many studies in hematology and oncology have already incorporated PRO assessments for QOL or HRQOL, but reporting is infrequent and variable. The most commonly used PROs for HRQOL in oncology include the Functional Assessment of Cancer Therapy-General (FACT-G), the EORTC QLQ, and the National Institutes of Health (NIH)–developed PROMIS. There are also numerous disease-specific cancer-related HRQOL assessments that are available and validated. These measurements are usually only reported at the end of studies as aggregate and may benefit from real-time assessment because patients may report significant declines that may or may not be seen by clinicians in a timely manner. There are numerous other questionnaires used that focus on domains that contribute to HRQOL such as physical functioning, social functioning, emotional well-being, and overall management of their health. Studies may benefit from changing the way PROs are used to more than HRQOL by focusing on use for stratification, symptom monitoring, and subjective adverse event (AE) reporting.
      PROs have shown to be prognostic in multiple tumor types.
      • Gotay C.C.
      • Kawamoto C.T.
      • Bottomley A.
      • Efficace F.
      The prognostic significance of patient-reported outcomes in cancer clinical trials.
      • Quinten C.
      • Coens C.
      • Mauer M.
      • et al.
      Baseline quality of life as a prognostic indicator of survival: a meta-analysis of individual patient data from EORTC clinical trials.
      Efficace et al
      • Efficace F.
      • Gaidano G.
      • Breccia M.
      • et al.
      Prognostic value of self-reported fatigue on overall survival in patients with myelodysplastic syndromes: a multicentre, prospective, observational, cohort study.
      showed that self-reported fatigue was an independent prognostic factor beyond the gold standard of the International Prognostic Scoring Symptom in myelodysplastic syndrome that includes objective clinical and laboratory factors. This is also true in allogenic stem cell transplantation wherein the pre-transplant physical component summary (PCS) score of the short-form 36 questionnaire (SF-36) was able to predict 1-year post-transplant mortality and transplant-related death. Moreover, the SF-36 PCS was also able to predict for post-transplant early mortality rate, and this remained prognostic in multivariate analysis when physician-assigned Karnofsky PS did not.
      • Wood W.
      • Le-Rademacher J.
      • Syrjala K.L.
      • et al.
      Patient-reported physiical functioning predicts the success of hematopoietic cell transplantation (BMT CTN 0902).
      These examples highlight the ability of PROs to get an accurate assessment of patient risk thus allowing for improved stratification of patients in studies who are likely to have different outcomes. Another way to use PROs in trials is to incorporate self-reported PS into clinical trials. Historically, physician-assigned PS has been considered the gold standard.
      • Brook R.H.
      • Ware J.E.J.
      • Davies-Avery A.
      • et al.
      Overview of adult health measures fielded in Rand’s health insurance study.
      The PS can determine if patients are eligible for studies as well as be used for stratification purposes. However, there is increasing evidence that there may be discordance between patient- and physician-assigned PS. In a study of prostate cancer, patients’ discordance between patient- and physician-PS was observed, and when there was discordance it was frequently towards physicians rating patients better.
      • Schnadig I.D.
      • Fromme E.K.
      • Loprinzi C.L.
      • et al.
      Patient-physician disagreement regarding performance status is associated with worse survivorship in patients with advanced cancer.
      In a retrospective study of amyloidosis patients, patient reported PS was found to remain prognostic for survival whereas physician-reported PS was not.
      • Warsame R.
      • Kumar S.K.
      • Gertz M.A.
      • et al.
      Hematology patient reported symptom screen to assess quality of life for AL amyloidosis.
      A large meta-analysis of 30 RCTs from the EORTC with a completed baseline EORTC-QLQ-C30 questionnaire showed that on stratified multivariate modeling, patient reported parameters, but not physician-reported PS, provided significant prognostic information.
      • Quinten C.
      • Coens C.
      • Mauer M.
      • et al.
      Baseline quality of life as a prognostic indicator of survival: a meta-analysis of individual patient data from EORTC clinical trials.
      These studies call to question whether only physician PS should be used for clinical studies, especially if they often err favorably and put patients who are unwell into clinical studies potentially putting them at increased risk.
      Traditionally, toxicity assessment in oncology clinical trials has predominantly been reported using the National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE).
      US Department of Health and Human Services
      This library of AEs includes approximately 10% symptoms such as nausea and pain, and these have traditionally been assigned by the health care team.
      US Department of Health and Human Services
      There is evidence to suggest that clinician-derived or investigator-obtained symptomatic AEs may be unreliable or frequently missing.
      • Trotti A.
      • Colevas A.D.
      • Setser A.
      • Basch E.
      Patient-reported outcomes and the evolution of adverse event reporting in oncology.
      • Fromme E.K.
      • Eilers K.M.
      • Mori M.
      • Hsieh Y.C.
      • Beer T.M.
      How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30.
      • Atkinson T.M.
      • Li Y.
      • Coffey C.W.
      • et al.
      Reliability of adverse symptom event reporting by clinicians.
      In the advent of increasing use of maintenance chemotherapies (eg, multiple myeloma), targeted therapies (eg, various lymphomas and chronic myeloid leukemia), or chronic indefinite treatments (eg, chronic myeloid leukemia), ensuring the tolerability and functional impact of these treatments would benefit from patient reporting. The NCI developed a library of 124 PRO items, the PRO-CTCAE, to capture 78 symptomatic AEs. It is written in plain language and can describe the attribution, frequency, severity, and impact to daily lives.
      • Basch E.
      • Reeve B.B.
      • Mitchell S.A.
      • et al.
      Development of the national cancer institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE).
      The PRO-CTCAE is to be used in conjunction with the CTCAE and is freely available to incorporate specific AEs of interest into any given clinical trial. It has been tested and found to be valid and reliable in a large, heterogeneous population of cancer patients in the United States.
      • Dueck A.C.
      • Mendoza T.R.
      • Mitchell S.A.
      • et al.
      Validity and reliability of the us national cancer institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE).
      The benefit of the PRO-CTCAE is that it documents the interference with activity in addition to severity, which may provide insights into true tolerability of treatments. The MD Anderson Symptom Inventory is another valid, reliable, and frequently used option for PRO assessment in clinical trials and provides a list of commonly experienced symptoms for patients to choose from and report that can be specified to expected outcomes for the study treatment.
      • Fishman B.
      • Pasternak S.
      • Wallenstein S.L.
      • Houde R.W.
      • Holland J.C.
      • Foley K.M.
      The Memorial Pain Assessment Card. A valid instrument for the evaluation of cancer pain.
      Thus, clinical trials would benefit by the thoughtful incorporation of PROs in the stratification of patients toxicity assessment, and as important trial endpoints either as HRQOL or as part of a composite endpoint.

      How to Use Pros in Comparative Effectiveness Research?

      PROs can be used to assess patient-centered outcomes as part of comparative effectiveness research (CER). CER can inform health care decisions by providing evidence on the effectiveness, benefits, and harms of different treatment options and aims to provide practical evidence to support real-world decision-making. Results of CER research can assist clinicians, patients, and other stakeholders in making informed decisions to improve health care, validate an intervention, and identify which treatments best meet a certain population’s needs. An essential part of CER is the integration of patients’ perspectives about their health with clinical and biological data to evaluate the safety and effectiveness of interventions.
      • Reeve B.B.
      • Wyrwich K.W.
      • Wu A.W.
      • et al.
      ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research.
      Capturing the patient-subjective experience is essential in prospective clinical CER to examine real-world outcomes related to existing treatments or process interventions.
      • Basch E.
      • Abernethy A.P.
      • Mullins C.D.
      • et al.
      Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology.
      As argued by Basch et al,
      • Basch E.
      • Abernethy A.P.
      • Mullins C.D.
      • et al.
      Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology.
      without direct evidence reflecting the patient experience, stakeholders including patients, clinicians, payers, investigators, and regulators have incomplete information for decision-making. Information obtained or reported by the health care team does not always accurately reflect patients’ experiences with care and generally cannot substitute for direct patient reporting.
      • Trotti A.
      • Colevas A.D.
      • Setser A.
      • Basch E.
      Patient-reported outcomes and the evolution of adverse event reporting in oncology.
      Implementing PROs in CER can help to develop a patient-centered research agenda by collecting and using PROs for a comprehensive understanding of the patient experience during care, as well as using PROs to obtain treatment outcomes. This ensures an alignment between outcomes of interest for patients and clinicians.
      Although standardized validated tools exist to gather data on PROs for clinical care and research, significant barriers persist in the incorporation of these measures into routine clinical care, making them unavailable for later CER or pragmatic clinical trials. Based on a white paper published in 2012 on an effectiveness guidance document, recommendations for the appropriate inclusion of PROs in the design and implementation of prospective CER in adult oncology include 15 recommendations including guidance on selection of measures, implementation methods, and data analysis and reporting.
      • Basch E.
      • Abernethy A.P.
      • Mullins C.D.
      • et al.
      Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology.
      This guidance document recommends that all prospective clinical CER studies in adult oncology should include PRO measures. Further, inclusion of suitable patient-reported symptoms should be appropriate to the study population, intervention, context, objectives, and setting. Measures must include an assessment of HRQOL and a measure that enables cost-utility analysis. Lastly, the selected measures must show validity, reliability, sensitivity in a comparable patient population, ability to detect changes over time, as well as an appropriate recall period. The implementation of PROs in this setting should limit data collection so that patients can complete as quickly as possible (not exceeding 15 to 20 minutes), collect PRO data as frequently as needed to meet research objectives without overburdening patients, use electronic data capture technologies whenever possible, consider whether measurement equivalence has been established when mixing modes of PRO data collection, and use methods to minimize missing PRO data. For data analysis and reporting, this guidance document recommended conducting a power calculation for the key PRO endpoints, including a plan for analyzing and reporting missing PRO data, reporting the proportion of patients experiencing a change from baseline shown as being meaningful for each measure as well as mean group changes, evaluating the cumulative distribution of responses, and publishing results of PRO data simultaneously with other clinical outcomes. Two excellent papers include the Patient-Centered Outcomes Research Institute guidance for methodological standards and patient-centeredness in CER along with the International Society for Quality of Life Research recommendations for the minimum standards for PRO measures to be used in CER.
      • Reeve B.B.
      • Wyrwich K.W.
      • Wu A.W.
      • et al.
      ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research.
      • Methodology Committee of the Patient-Centered Outcomes Research Institute
      Methodological standards and patient-centeredness in comparative effectiveness research: the PCORI perspective.
      The Consolidated Standards of Reporting Trials (CONSORT) has a CONSORT-PRO extension has suggested including the following five checklist items while reporting PROs in RCTs: (1) PROs must be identified as a primary or secondary outcome in the abstract; (2) a description of the hypothesis of the PROs and relevant domains must be provided; (3) evidence of the PRO instrument's validity and reliability must be provided or cited; (4) statistical approaches for dealing with missing data must be explicitly stated; and (5) PRO-specific limitations of study findings and generalizability of results to other populations and clinical practice must be discussed.
      • Calvert M.
      • Blazeby J.
      • Altman D.G.
      • et al.
      Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension.
      Meta-analyses in multiple tumor types highlight the need for including research questions or hypotheses about HRQOL, rationale for using a particular assessment, baseline assessment, management of missing data, and a need for better adherence to existing HRQOL reporting guidelines.
      • Zikos E.
      • Ghislain I.
      • Coens C.
      • et al.
      Health-related quality of life in small-cell lung cancer: a systematic review on reporting of methods and clinical issues in randomised controlled trials.
      • Ghislain I.
      • Zikos E.
      • Coens C.
      • et al.
      Health-related quality of life in locally advanced and metastatic breast cancer: methodological and clinical issues in randomised controlled trials.
      • Efficace F.
      • Bottomley A.
      • Vanvoorden V.
      • Blazeby J.M.
      Methodological issues in assessing health-related quality of life of colorectal cancer patients in randomised controlled trials.
      • Efficace F.
      • Bottomley A.
      • van Andel G.
      Health related quality of life in prostate carcinoma patients: a systematic review of randomized controlled trials.
      • Efficace F.
      • Bottomley A.
      Health related quality of life assessment methodology and reported outcomes in randomised controlled trials of primary brain cancer patients.

      PROs in Cancer Registries

      Large clinical databases of prospectively maintained data are invaluable in oncology for observational and epidemiologic research. PROs can complement these databases. The Center for International Blood and Marrow Transplant Research (CIBMTR) is a prospectively maintained transplant registry that has successfully incorporated centralized PRO data collection in the blood and marrow transplant population. Using a hybrid model of local consent from transplant centers followed by patients completing the Functional Assessment of Cancer therapy–Bone Marrow Transplant (FACT-BMT) and SF-36 measures and returning to the CIBMTR at pre-transplant, 6 months, 1 year, and 2 years post-transplant showed feasibility of longitudinal PRO collection, high retention rates, as well as high patient satisfaction.
      • Shaw B.E.
      • Brazauskas R.
      • Millard H.
      • et al.
      Centralized patient-reported outcome data collection in transplantation is feasible and clinically meaningful.

      How to Use Pros to Improve Clinical Oncology Care?

      Symptom management is a cornerstone of clinical care. However, symptoms, subjective toxicities, and impaired functioning can go undetected by clinicians as much as 50% to 74% of the time.
      • Trotti A.
      • Colevas A.D.
      • Setser A.
      • Basch E.
      Patient-reported outcomes and the evolution of adverse event reporting in oncology.
      • Di Maio M.
      • Gallo C.
      • Leighl N.B.
      • et al.
      Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials.
      A review by Kluetz et al
      • Kluetz P.G.
      • Slagle A.
      • Papadopoulos E.J.
      • et al.
      Focusing on core patient-reported outcomes in cancer clinical trials: symptomatic adverse events, physical function, and disease-related symptoms.
      proposed disease symptoms, physical function, and symptomatic AEs as the core concepts that are the key contributors to HRQOL and should be used to assess the impact of cancer therapies. Two large systematic reviews provide evidence supporting the utility of PROs in oncologic care.
      • Chen J.
      • Ou L.
      • Hollis S.J.
      A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting.
      • Kotronoulas G.
      • Kearney N.
      • Maguire R.
      • et al.
      What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials.
      Kotronoulas et al,
      • Kotronoulas G.
      • Kearney N.
      • Maguire R.
      • et al.
      What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials.
      in a systematic review of five electronic databases from inception to 2012, identified 24 controlled trials of adults receiving active anticancer treatment or supportive care, and showed improved patient health care team communication but tentative evidence on improving quality of life. Chen et al
      • Chen J.
      • Ou L.
      • Hollis S.J.
      A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting.
      conducted a systematic review of 27 oncology studies from 2000-2011, including 16 randomized controlled trials that supported the routine collection of PROs with feedback. This study showed that collection of PROs with feedback was associated with a strong evidence of enhancing patient-provider communication and improved patient satisfaction, moderate strength of evidence for monitoring treatment response and detecting unrecognized problems, and weak to no positive effect on changes to patient management and improved health outcomes.
      • Chen J.
      • Ou L.
      • Hollis S.J.
      A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting.
      A more recent RCT of electronic patient reported symptom monitoring versus usual care based on follow-up showed a 5.2-month overall survival improvement in the PRO group compared with the usual care group.
      • Kotronoulas G.
      • Kearney N.
      • Maguire R.
      • et al.
      What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials.
      PROs can be used serially to monitor PROs over time to improve HRQOL, improve communication, and aid clinical management.
      • Basch E.
      • Deal A.M.
      • Dueck A.C.
      • et al.
      Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment.
      • Velikova G.
      • Booth L.
      • Smith A.B.
      • et al.
      Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial.
      • Detmar S.B.
      • Muller M.J.
      • Schornagel J.H.
      • et al.
      Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial.
      • Snyder C.F.
      • Aaronson N.K.
      Use of patient-reported outcomes in clinical practice.
      PRO monitoring can help patients articulate their problems, identify problems patients might not have raised, may help patients feel cared for, and may improve patients’ knowledge about outcomes, collectively enhancing communication between patients and clinicians.
      • Snyder C.F.
      • Aaronson N.K.
      Use of patient-reported outcomes in clinical practice.
      • Dobrozsi S.
      • Panepinto J.
      Patient-reported outcomes in clinical practice.
      Additionally, PROs can be used in clinical care to tailor supportive care for patients as well as toward quality improvement and performance measurement.
      • Dobrozsi S.
      • Panepinto J.
      Patient-reported outcomes in clinical practice.
      The Alliance Foundation Trials-39 Electronic Patient Reporting of Symptoms During Cancer Treatment trial is a cluster RCT at multiple US sites through the Alliance Foundation Trials that is currently ongoing (NCT03249090). This study will compare weekly patient symptom reporting via Web or automated telephone system compared with usual care delivery. The primary outcomes for this study will be physical functioning measured via the QLQ-C30 at 3 months and overall survival at 24 months.
      PROs can also be used to optimize clinical care among cancer survivors. As oncology care improves, patients are living longer. Cancer survivors can have short- and long-term effects in physical function and mental health domains as a result of the cancer diagnosis and its treatment. Whereas many of the available PRO instruments such as FACT, EORTC, and PROMIS can be used, these focus on symptoms and QOL experienced. There are tools developed to assess survivors’ unmet supportive care needs including the Cancer Survivors’ Unmet Needs (CaSUN), Cancer Survivors’ Partners Unmet Needs (CaSPUN), Impact of Cancer v2 (IOCv2), Quality of Life in Adult Cancer Survivors (QLACS), and Social Difficulties Inventory (SDI-21).
      • Gordon B.E.
      • Chen R.C.
      Patient-reported outcomes in cancer survivorship.
      Some of the unique domains these instruments provide include focus on comprehensive cancer care, information, relationships, partner impact, body changes, life interferences, health awareness, employment, and financial toxicity.
      The Figure shows the key factors determining PROs and the key outcomes that can be affected by PROs.
      Figure thumbnail gr1
      FigureFramework for utility of patient reported outcomes (PROs) in clinical care.

      Limitations and Barriers to Using Pros in Practice

      Similar to research, choosing the right instrument is critical, but choosing among the numerous available options is a challenge and determining what the goal of use will be essential. Considerations to be included are whether an instrument should be disease-specific versus general, screening versus diagnostic, and burden for patients to complete and providers to interpret. The clinician and the patient must both benefit in tangible ways from the PRO. The International Society for Quality of Life Research group has published a user’s guide for PRO in clinical practice that gives a summary of considerations.
      • Snyder C.F.
      • Aaronson N.K.
      • Choucair A.K.
      • et al.
      Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations.
      Implementation of PRO initiatives at individual centers is a major barrier. Although there has been significant interest in using PROs in management of individual patients, many clinics and centers that already use screening questionnaires (eg, National Comprehensive Cancer Network distress thermometer to screen for need of psychosocial support, screening for depression, etc) are hindered by the lack of a systematic, accessible, and uniform way of collecting PROs. Among excellent examples of PRO initiatives implemented to date, clinician champions with support from institutional leadership is paramount.
      • Basch E.
      Patient-reported outcomes - harnessing patients' voices to improve clinical care.
      At the 2018 American Society of Clinical Oncology Annual Meeting, a session dedicated to the implementation of PROs in clinical practice reviewed several case studies of institutions which have successfully incorporated PROs in routine clinical practice.
      • Basch E.
      • Barbera L.
      • Kerrigan C.L.
      • Velikova G.
      Implementation of patient-reported outcomes in routine medical care.
      Ideally, the electronic platform will provide ease of use as well as data storage when implementing PRO collection. Systematization and automation of data collection systems are crucial in facilitating this process which will be enhanced by the widespread availability of mobile and portable technologies. It will be essential to have a process in place to synthesize and manage the potential deluge of information, and ensure any high-risk issues result in appropriate evaluation in a timely manner. Real-time PRO data collection will become more common, and possibly even be used as an additional vital sign in the clinic for patient well-being assessments and guide subsequent therapy. Multiple examples of this already exist in the orthopedic literature.
      • Baumhauer J.
      Patient-reported outcomes — are they living up to their potential?.
      With the field of oncology rapidly growing in newer therapies (eg, immuno-oncology, targeted therapies, chimeric antigen receptor T cell, and other cellular therapies with different toxicities such as immune-related AEs, cytokine release syndrome, and neurotoxicity), it is important for clinicians to work with measurement experts to develop proper qualitative studies to identify key outcomes salient to these treatment modalities which would then inform the use of appropriate PRO domains.

      Conclusion

      PROs have a significant potential to improve patient care in clinical oncology by improving patient-provider communication, patient QOL, and survival. Despite the current challenges of heterogeneity, length, and interpretation, PROs will become standard practice in research and clinic in oncology.

      Acknowledgments

      The authors thank Angela Dispenzieri, MD, and Kathryn Flynn, PhD, for their input in developing this manuscript.

      Supplemental Online Material

      References

        • Patrick D.L.
        • Burke L.B.
        • Powers J.H.
        • et al.
        Patient-reported outcomes to support medical product labeling claims: FDA perspective.
        Value Health. 2007; 10: S125-S137
        • Brook R.H.
        • Ware J.E.J.
        • Davies-Avery A.
        • et al.
        Overview of adult health measures fielded in Rand’s health insurance study.
        Med Care. 1979; 17 (1-131): iii-x
        • Institute of Medicine
        Crossing the quality chasm: a new health system for the 21st century.
        National Academies Press, Washington, DC2001: 1-8
        • Schnadig I.D.
        • Fromme E.K.
        • Loprinzi C.L.
        • et al.
        Patient-physician disagreement regarding performance status is associated with worse survivorship in patients with advanced cancer.
        Cancer. 2008; 113: 2205-2214
        • Warsame R.
        • Kumar S.K.
        • Gertz M.A.
        • et al.
        Hematology patient reported symptom screen to assess quality of life for AL amyloidosis.
        Am J Hematol. 2017; 92: 435-440
        • Gotay C.C.
        • Kawamoto C.T.
        • Bottomley A.
        • Efficace F.
        The prognostic significance of patient-reported outcomes in cancer clinical trials.
        J Clin Oncol. 2008; 26: 1338-1345
        • Quinten C.
        • Coens C.
        • Mauer M.
        • et al.
        Baseline quality of life as a prognostic indicator of survival: a meta-analysis of individual patient data from EORTC clinical trials.
        Lancet Oncol. 2009; 10: 865-871
        • Scoggins J.F.
        • Patrick D.L.
        The use of patient-reported outcomes instruments in registered clinical trials: evidence from ClinicalTrials.gov.
        Contemp Clin Trials. 2009; 30: 289-292
        • Vodicka E.
        • Kim K.
        • Devine E.B.
        • Gnanasakthy A.
        • Scoggins J.F.
        • Patrick D.L.
        Inclusion of patient-reported outcome measures in registered clinical trials: Evidence from ClinicalTrials.gov (2007-2013).
        Contemp Clin Trials. 2015; 43: 1-9
        • Fitzpatrick R.
        • Davey C.
        • Buxton M.J.
        • Jones D.R.
        Evaluating patient-based outcome measures for use in clinical trials.
        Health Technol Assess. 1998; 2 (1-74): i-iv
        • Croog S.H.
        • Levine S.
        • Testa M.A.
        • et al.
        The effects of antihypertensive therapy on the quality of life.
        N Engl J Med. 1986; 314: 1657-1664
        • U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research
        • U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research
        • U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health
        Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance.
        Health Qual Life Outcomes. 2006; 4: 79
        • Demuro C.
        • Clark M.
        • Doward L.
        • Evans E.
        • Mordin M.
        • Gnanasakthy A.
        Assessment of PRO label claims granted by the FDA as compared to the EMA (2006-2010).
        Value Health. 2013; 16: 1150-1155
        • Fehnel S.
        • DeMuro C.
        • McLeod L.
        • Coon C.
        • Gnanasakthy A.
        US FDA patient-reported outcome guidance: great expectations and unintended consequences.
        Expert Rev Pharmacoecon Outcomes Res. 2013; 13: 441-446
        • Willke R.J.
        Measuring the value of treatment to patients: patient-reported outcomes in drug development.
        Am Health Drug Benefits. 2008; 1: 34-40
        • Gnanasakthy A.
        • Mordin M.
        • Clark M.
        • DeMuro C.
        • Fehnel S.
        • Copley-Merriman C.
        A review of patient-reported outcome labels in the United States: 2006 to 2010.
        Value Health. 2012; 15: 437-442
        • Gnanasakthy A.
        • Mordin M.
        • Evans E.
        • Doward L.
        • DeMuro C.
        A review of patient-reported outcome labeling in the United States (2011–2015).
        Value Health. 2017; 20: 420-429
        • Gnanasakthy A.
        • DeMuro C.
        • Clark M.
        • Haydysch E.
        • Ma E.
        • Bonthapally V.
        Patient-reported outcomes labeling for products approved by the office of hematology and oncology products of the US Food and Drug Administration (2010-2014).
        J Clin Oncol. 2016; 34: 1928-1934
        • Basch E.
        • Geoghegan C.
        • Coons S.J.
        • et al.
        Patient-Reported outcomes in cancer drug development and US regulatory review: perspectives from industry, the Food and Drug Administration, and the patient.
        JAMA Oncol. 2015; 1: 375-379
        • Acquadro C.
        • Berzon R.
        • Dubois D.
        • et al.
        Incorporating the patient’s perspective into drug development and communication: an ad hoc task force report of the patient-reported outcomes (PRO) Harmonization Group meeting at the food and drug administration, February 16, 2001.
        Value Health. 2003; 6: 522-531
        • Mesa R.A.
        • Gotlib J.
        • Gupta V.
        • et al.
        Effect of ruxolitinib therapy on myelofibrosis-related symptoms and other patient-reported outcomes in COMFORT-I: a randomized, double-blind, placebo-controlled trial.
        J Clin Oncol. 2013; 31: 1285-1292
        • Burris 3rd, H.A.
        • Moore M.J.
        • Andersen J.
        • et al.
        Improvements in survival and clinical benefit with gemcitabine as first-line therapy for patients with advanced pancreas cancer: a randomized trial.
        J Clin Oncol. 1997; 15: 2403-2413
        • Wildiers H.
        • Mauer M.
        • Pallis A.
        • et al.
        End points and trial design in geriatric oncology research: a joint European Organisation for Research and Treatment of Cancer-Alliance for clinical trials in oncology–International Society of Geriatric Oncology position article.
        J Clin Oncol. 2013; 31: 3711-3718
        • Efficace F.
        • Gaidano G.
        • Breccia M.
        • et al.
        Prognostic value of self-reported fatigue on overall survival in patients with myelodysplastic syndromes: a multicentre, prospective, observational, cohort study.
        Lancet Oncol. 2015; 16: 1506-1514
        • Wood W.
        • Le-Rademacher J.
        • Syrjala K.L.
        • et al.
        Patient-reported physiical functioning predicts the success of hematopoietic cell transplantation (BMT CTN 0902).
        Cancer. 2016; 122: 91-98
        • US Department of Health and Human Services
        Common Terminology Criteria for Adverse Events (CTCAE). National Institutes of Health, Bethesda, MD2010
        • Trotti A.
        • Colevas A.D.
        • Setser A.
        • Basch E.
        Patient-reported outcomes and the evolution of adverse event reporting in oncology.
        J Clin Oncol. 2007; 25: 5121-5127
        • Fromme E.K.
        • Eilers K.M.
        • Mori M.
        • Hsieh Y.C.
        • Beer T.M.
        How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30.
        J Clin Oncol. 2004; 22: 3485-3490
        • Atkinson T.M.
        • Li Y.
        • Coffey C.W.
        • et al.
        Reliability of adverse symptom event reporting by clinicians.
        Qual Life Res. 2012; 21: 1159-1164
        • Basch E.
        • Reeve B.B.
        • Mitchell S.A.
        • et al.
        Development of the national cancer institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE).
        J Natl Cancer Inst. 2014; 106: 1-11
        • Dueck A.C.
        • Mendoza T.R.
        • Mitchell S.A.
        • et al.
        Validity and reliability of the us national cancer institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE).
        JAMA Oncol. 2015; 1: 1051-1059
        • Fishman B.
        • Pasternak S.
        • Wallenstein S.L.
        • Houde R.W.
        • Holland J.C.
        • Foley K.M.
        The Memorial Pain Assessment Card. A valid instrument for the evaluation of cancer pain.
        Cancer. 1987; 60: 1151-1158
        • Reeve B.B.
        • Wyrwich K.W.
        • Wu A.W.
        • et al.
        ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research.
        Qual Life Res. 2013; 22: 1889-1905
        • Basch E.
        • Abernethy A.P.
        • Mullins C.D.
        • et al.
        Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology.
        J Clin Oncol. 2012; 30: 4249-4255
        • Methodology Committee of the Patient-Centered Outcomes Research Institute
        Methodological standards and patient-centeredness in comparative effectiveness research: the PCORI perspective.
        JAMA. 2012; 307: 1636-1640
        • Calvert M.
        • Blazeby J.
        • Altman D.G.
        • et al.
        Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension.
        JAMA. 2013; 309: 814-822
        • Zikos E.
        • Ghislain I.
        • Coens C.
        • et al.
        Health-related quality of life in small-cell lung cancer: a systematic review on reporting of methods and clinical issues in randomised controlled trials.
        Lancet Oncol. 2014; 15: e78-e89
        • Ghislain I.
        • Zikos E.
        • Coens C.
        • et al.
        Health-related quality of life in locally advanced and metastatic breast cancer: methodological and clinical issues in randomised controlled trials.
        Lancet Oncol. 2016; 17: e294-e304
        • Efficace F.
        • Bottomley A.
        • Vanvoorden V.
        • Blazeby J.M.
        Methodological issues in assessing health-related quality of life of colorectal cancer patients in randomised controlled trials.
        Eur J Cancer. 2004; 40: 187-197
        • Efficace F.
        • Bottomley A.
        • van Andel G.
        Health related quality of life in prostate carcinoma patients: a systematic review of randomized controlled trials.
        Cancer. 2003; 97: 377-388
        • Efficace F.
        • Bottomley A.
        Health related quality of life assessment methodology and reported outcomes in randomised controlled trials of primary brain cancer patients.
        Eur J Cancer. 2002; 38: 1824-1831
        • Shaw B.E.
        • Brazauskas R.
        • Millard H.
        • et al.
        Centralized patient-reported outcome data collection in transplantation is feasible and clinically meaningful.
        Cancer. 2017; 123: 4687-4700
        • Di Maio M.
        • Gallo C.
        • Leighl N.B.
        • et al.
        Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials.
        J Clin Oncol. 2015; 33: 910-915
        • Kluetz P.G.
        • Slagle A.
        • Papadopoulos E.J.
        • et al.
        Focusing on core patient-reported outcomes in cancer clinical trials: symptomatic adverse events, physical function, and disease-related symptoms.
        Clin Cancer Res. 2016; 22: 1553-1558
        • Chen J.
        • Ou L.
        • Hollis S.J.
        A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting.
        BMC Health Serv Res. 2013; 13: 211
        • Kotronoulas G.
        • Kearney N.
        • Maguire R.
        • et al.
        What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials.
        J Clin Oncol. 2014; 32: 1480-1501
        • Basch E.
        • Deal A.M.
        • Dueck A.C.
        • et al.
        Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment.
        JAMA. 2017; 318: 197-198
        • Velikova G.
        • Booth L.
        • Smith A.B.
        • et al.
        Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial.
        J Clin Oncol. 2004; 22: 714-724
        • Detmar S.B.
        • Muller M.J.
        • Schornagel J.H.
        • et al.
        Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial.
        JAMA. 2002; 288: 3027-3034
        • Snyder C.F.
        • Aaronson N.K.
        Use of patient-reported outcomes in clinical practice.
        Lancet. 2009; 374: 369-370
        • Dobrozsi S.
        • Panepinto J.
        Patient-reported outcomes in clinical practice.
        Hematology Am Soc Hematol Educ Program. 2015; 2015: 501-506
        • Gordon B.E.
        • Chen R.C.
        Patient-reported outcomes in cancer survivorship.
        Acta Oncol. 2017; 56: 166-173
        • Snyder C.F.
        • Aaronson N.K.
        • Choucair A.K.
        • et al.
        Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations.
        Qual Life Res. 2012; 21: 1305-1314
        • Basch E.
        Patient-reported outcomes - harnessing patients' voices to improve clinical care.
        N Engl J Med. 2017; 376: 105-108
        • Basch E.
        • Barbera L.
        • Kerrigan C.L.
        • Velikova G.
        Implementation of patient-reported outcomes in routine medical care.
        Am Soc Clin Oncol Educ Book. 2018; 38: 122-134
        • Baumhauer J.
        Patient-reported outcomes — are they living up to their potential?.
        N Engl J Med. 2017; 377: 6-9