VOLUME 25 䡠 NUMBER 32 䡠 NOVEMBER 10 2007 JOURNAL OF CLINICAL ONCOLOGY R E V I E W A R T I C L E Lessons Learned in the Assessment of Health-Related Quality of Life: Selected Examples From the National Cancer Institute of Canada Clinical Trials Group Michael Brundage, David Osoba, Andrea Bezjak, Dongsheng Tu, Michael Palmer, and Joseph Pater From the Departments of Oncology and Community Health and Epidemiology, and National Cancer Institute of Canada Clinical Trials Group Queen’s Cancer Research Institute, Queens University, Kingston; Department of Radiation Oncology, Princess Margaret Hospital; University Health Network, University of Toronto, Toronto, Ontario; and QOL Consulting, West Vancouver, British Columbia, Canada. Submitted February 26, 2007; accepted June 15, 2007. Supported in part by the National Cancer Institute of Canada. Authors’ disclosures of potential conflicts of interest and author contributions are found at the end of this article. Address reprint requests to Michael Brundage, MD, MSc, Cancer Clinic of Southeastern Ontario at Kingston General Hospital, 25 King St W, Kingston, Ontario, Canada, K7L 5P9; e-mail: firstname.lastname@example.org. © 2007 by American Society of Clinical Oncology 0732-183X/07/2532-5078/$20.00 DOI: 10.1200/JCO.2007.11.4645 A B S T R A C T In this article, we provide a brief historical review of the development of patient-reported outcome measurement, analysis, and reporting in clinical trials of the National Cancer Institute of Canada Clinical Trials Group (NCIC CTG). In doing so, we examine selected lessons learned in furthering the quality of these data and their application to clinical practice. We conclude that sequential institution of key policies within the NCIC CTG and the development of a collective philosophy within the group has enabled the routine incorporation of health-related quality of life into clinical trial protocols according to robust scientific principles; that collection of quality data is possible in a variety of circumstances (although not universally so); that patient-reported data on subjective experiences is likely to be more reliable and valid than conventional toxicity information; and that simple analyses that report group trends as well as individual patient response rates are preferred. J Clin Oncol 25:5078-5081. © 2007 by American Society of Clinical Oncology INTRODUCTION The purpose of this article is to provide a Canadian cooperative clinical trials group perspective on issues central to the application of patient-reported outcome (PRO) assessment in cancer clinical trials. The National Cancer Institute of Canada Clinical Trials Group (NCIC CTG) is Canada’s largest cancer clinical trials group, and it is from the perspective of the NCIC CTG that we provide experiences and comments relevant to the Patient-Reported Outcomes Assessment in Cancer Trials (PROACT) themes.Inthesixsectionsofthisarticle,wefirstprovide abriefhistoryanddescriptionoftheorganizationofthe NCIC CTG and then provide historical and current perspectives pertaining to the main PROACT themes addressing the decision to collect PRO data, data collectionandplannedanalyses,fieldoperations,andfinal data analysis and interpretation. Finally, we provide examples of ancillary or complementary research undertaken by researchers at the NCIC CTG that has contributed to the group’s current strategies for incorporating PROs in clinical trials. QUALITY OF LIFE COMMITTEE OF THE NCIC CTG The NCIC CTG was established in 1980 as a clinical research program of the National Cancer Institute of Canada. The group’s mission is to develop and conduct clinical trials aimed at improving the treatment and prevention of cancer with the ultimate goal of reducing cancer morbidity and mortality from this disease. Under direction of Dr J. Pater, the NCIC CTG has grown into a cooperative effort of more than 80 Canadian centers with full membership participation. The Quality of Life (QOL) Committee has been an NCIC CTG standing committee since 1987, born out of recognition of the potential value of using validated measures of multidimensional PROs in clinical trials of cancer therapies. Subsequently, health-related QOL (HRQOL) assessment in the CTG has evolved, in keeping with developing understanding within the discipline. The main mandate of the QOL Committee is to provide methodologic and practical support for collection of HRQOL data in clinical trials conducted by NCIC CTG. The QOL Committee presently consists of 17 members, who were invited as a result of their expertise in QOL research, with representation of the diversity of professional, cancer site–specific, and geographic backgrounds of NCIC CTG investigators. The members include medical, radiation, and surgical oncologists; social scientists; a senior biostatistician; a clinical research associate (CRA) representative; and representatives of the NCIC CTG central office. In the sections that follow, we describe further details of the committee’s activities. 5078 Information downloaded from jco.ascopubs.org and provided by at NIH LIBRARY on December 3, 2010 from 126.96.36.199 Copyright © 2007 American Society of Clinical Oncology. All rights reserved. NCIC CTG Perspective DECISION TO COLLECT PRO DATA Members of the QOL Committee influence the decision to collect PRO data in two related ways, as contributing members of the disease site executive and as clinical trial QOL coordinators for specific trials. In general, the decision to collect PRO data in a specific trial is made by the trial principle investigator in consultation with the QOL Committee liaison for that disease site and with appropriate input from the trial physician coordinator (a member of the CTG central office faculty). Should there be consensus that the trial will include a QOL end point, the QOL Committee member then becomes the QOL coordinator for that study. By QOL end point, we refer to PRO assessments that have been guided by the QOL Committee and that include measures provided by validated QOL instruments and, where relevant, trial-specific symptom checklists. In addition, the committee helps oversee the use of patient symptom diaries and measures of psychological distress (the latter more recently in conjunction with a psychosocial behavioral working group). Over the evolution of our experience with QOL outcome measurement, the CTG has implemented several policies that have influenced this general decision process. For example, in keeping with our belief that QOL is an integral end point in most randomized studies, the CTG executive approved a policy in 1989 stating that “there should be a statement about the anticipated impact on quality of life with every proposed phase III clinical trial and whether or not quality of life measures will be incorporated in the protocol.”1 As described in more detail by Osoba et al,2 this policy was successful in bringing QOL to the attention of all investigators considering clinical trial protocol proposals. Furthermore, implementation of the policy required investigators to address potential adverse or beneficial consequences of the study interventions that might be best assessed by direct reporting by patients and, thus, enhance the interpretation of the overall study results. Consequently, 52 (92%) of 57 phase III trials led by the CTG since 1990 have included an HRQOL end point. A second example of effective CTG policy implementation is the past strategic decision to include a QOL Committee liaison member on the executive of each disease site committee. As a result, QOL Committee members had the opportunity to educate the clinician investigator leaders regarding opportunities to incorporate QOL measurement into clinical trials under development or under consideration by that site group. This process resulted in QOL elements of study protocols being integrated earlier into the trial development process than had been previously experienced. These and other policies, taken together, have shaped and defined expectations of clinical investigators and QOL Committee members alike, allowing the integration of HRQOL measurement as a fundamental element of clinical trial development. Several other strategic decisions have complemented the implementation of these policies. For example, the CTG focus on using the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire C30 instrument3 and collaborating with its developers to contribute to its validation4 has increased familiarity of the CTG members with the instrument’s administration, analysis, and interpretation. The organizational infrastructure of the group has allowed for effective integration of the QOL Committee members with the disease site groups, and the QOL Committee has had the opportunity to set its own strategic plan in the context of the overall CTG direction. This evolution of the CTG members’ general attitudes toward the value of PRO data has facilitated ongoing decisions to use, or not use, PRO approaches in a given clinical trial setting. PLANNING DATA COLLECTION AND ANALYSIS In an effort to develop standards for the incorporation of PROs across clinical trials within the CTG, the QOL Committee provided guidelines for CTG investigators regarding the inclusion of HRQOL assessments.1 For example, the guidelines stated that the HRQOL components should be a part of the main protocol and not an add-on study. They further provided explicit instructions for the sections of the protocol dealing with the study rationale, HRQOL research hypothesis, the clinical objectives of HRQOL assessment, and research elements such as sample size requirements, instrument description and rationale, instructions for administration of the instrument(s), timing of assessments, planned analysis, and the wording for the consent form. The guidelines, while undergoing revision and modification over time, have provided the committee with a consistent framework for debating appropriate standards of HRQOL research practice within the CTG. Interested readers may find the original guidelines published by Osoba1 in 1992 and discussed further in 1998.5 FIELD OPERATIONS As with other cooperative clinical trials groups, field operations for the NCIC CTG have evolved substantially over time. The first clinical trial to collect HRQOL information (BR.56) was designed to compare chemotherapy for advanced non–small-cell lung cancer with best supportive care without chemotherapy. Despite good intentions of investigators who agreed to including the PRO in this study, compliance was poor (⬍ 25% of patients). Accordingly, a scientific session was held at a subsequent annual meeting of NCIC CTG investigators to stimulate interest in HRQOL and to re-examine the best approach to incorporating PROs into CTG study protocols. Since then, several educational activities for both clinicians and CRAs in the field have helped to move forward the PRO agenda of the group.7 These activities have included plenary scientific sessions at annual group meetings, poster sessions, a dedicated section in the CTG newsletter devoted to HRQOL, and trial-specific educational sessions designed to address perceived or anticipated operational issues before protocol activation.7 Examples of changes implemented by the CTG as a result of these deliberations include the following: a policy to integrate a complete HRQOL assessment as part of baseline documentation required before random assignment; a policy to make this assessment required for all patients able to read English or French but to allow patients not able to complete these forms (as a result of language barriers) to still participate in study protocols; and routine educational sessions with CRAs to review the rationale for PRO end point inclusion, PRO instrument characteristics, questionnaire administration details, timing of assessments, and special considerations at the time of activating each clinical trial protocol including HRQOL end points.7 As such, we experienced a paradigm shift over time where early concerns regarding patient burden with completing PRO questionnaires were 5079 www.jco.org Information downloaded from jco.ascopubs.org and provided by at NIH LIBRARY on December 3, 2010 from 188.8.131.52 Copyright © 2007 American Society of Clinical Oncology. All rights reserved. Brundage et al replaced by an understanding that patients were willing to, able to, and generally enthusiastic about participating in these assessments.2 In parallel with the implementation of these field operations, the CTG implemented central office quality assurance measures that further enhanced the collection of quality PRO data. A rigorous central office process for ensuring that new protocols were complete in terms of PRO measures was developed. Furthermore, participating center CRAs were routinely queried regarding late or missing expected forms becase these were considered equivalent in importance to the conventional clinical trial data expected at each assessment.8 The CTG Central Office, in collaboration with the QOL Committee, developed processes for monitoring the implementation of these policies, such as systematically reviewing the completion rate for baseline questionnaires over 16 clinical trials in a variety of disease sites, demonstrating a compliance of 97% for 2,556 patients at baseline.9 Finally, more recently, the Central Office has developed an electronic process to regularly inform each QOL coordinator about the compliance (expected v received PRO data forms at all study evaluation time points) for each active trial in that coordinator’s portfolio. This process enables the early identification of patient assessment time points where PRO submission compliance rates are lower than expected, thereby providing the opportunity for intervention on operational issues at the earliest possible time. The NCIC CTG recommends a relatively simple, clinically practical analysis that consists of four steps.14 After calculating the questionnaire completion rates and the baseline scores, the individual change scores from baseline are determined, and the treatment groups are statistically compared under the null hypothesis that change scores are no different between groups. In addition, the proportions of patients who have reported a predetermined clinically significant change score (using a 10-point threshold or other appropriate cut point) are calculated, and the differences in the number of patients who have benefited in each treatment group are tested for statistical significance. An advantage of this approach of calculating the proportion of patients with an HRQOL response is that patients with missing data may still be included in an intent-to-treat analysis by considering them as nonresponders. Moreover, criteria for duration of response can be applied on the basis of individual patient change scores. These approaches have been compared on summary statistical measures,15 and further research is required to explore how these approaches compare with more complex statistical analysis such as linear mixed regression models, growth curve analyses, or newer strategies such as patient assessment using item response theory coupled with computer adaptive testing. DATA ANALYSIS AND INTERPRETATION Research supervised or led by NCIC CTG physician coordinators and members of the QOL Committee has furthered the group’s collective efforts in PRO research in several ways. Although we do not attempt to comprehensively review these research efforts here, we provide examples of how research regarding QOL methodology and/or clinical application has been relevant to the overall NCIC CTG research agenda. Early in the experience of the group with HRQOL measurement, we were able to demonstrate that non–laboratory-based conventional toxicity measurement was subject to significant interobserver variation in its recording, even by experienced nurse CRAs.16 These efforts were useful in highlighting for clinicians the potential advantages of patients directly reporting their own subjective toxicity events. Early demonstrations of high compliance in patients completing QOL assessments and the success of educational interventions helped to sustain feasibility.7,17 The demonstrations of HRQOL scores as an independent prognostic factor for survival18 and as sensitive to specified time frame of consideration in the questionnaire19 helped to underscore the independent validity of these measures for clinicians. The added value of HRQOL information to conventional clinical trial end points was further demonstrated by HRQOL data defining clinical benefit of study interventions (eg, in trials of antiemetics in emetogenic chemotherapeutic regimens)10; by the demonstrated convergence of HRQOL-related benefit to improvement demonstrated in survival end points (eg, improved symptom control or patient wellbeing in palliative treatments yielding survival advantages)20,21; and by detecting and characterizing adverse effects of adjuvant treatment.22 Further research demonstrated that, within a given clinical trial, conventional toxicity information often yielded findings that differed from those reported by patients.23,24 Our qualitative25 and quantitative26 research into communicating the results of the NCIC CTG trials to patient or lay consumers of HRQOL information provided a clinically feasible strategy for reporting these data to patients in decision As operational issues within the NCIC CTG and methodologic understanding within the discipline of HRQOL measurement have evolved, so too has the approach of the CTG to HRQOL analysis and reporting of HRQOL data. Initially, analyses focused on between-group (intervention allocation) mean scores for each QOL domain. With recognition of the influence of missing data not at random (survivor bias), our focus shifted to between-group mean change scores using repeatedmeasures approaches.10 We recognized that such analyses are limited to those patients reporting both baseline and follow-up data and, thus, are limited to a subset of trial participants, risking bias that would influence the generalizability of results. More recently, the CTG has had interest in describing the proportions of patient participants who are clinically improved or deteriorated with respect to HRQOL domains at particular assessment time points compared with baseline scores. As has been described in detail elsewhere, there exists a variety of approaches to determining the magnitude of change in score that constitutes a valid cut point for defining a specific patient as clinically changed for either the better or worse.11-13 Clearly cut points that are too low risk false-positive identification of responders as a result of chance variance in scores, whereas cut points that are too high run the risk of being insensitive to clinically meaningful responses. Because no gold standard exists, the current CTG pragmatic approach is to use, for 100-point HRQOL scales, a cut point of 10 points in improvement or deterioration in defining response scores.14 We selected this 10-point threshold as appropriate for initial analyses based on our desire to minimize false-positive responses, given data from several sources, including patients’ reporting of significant subjective differences in their status,11 as well as the work of others regarding non–anchorbased assessment methods.13 We recognize that specific clinical contexts and other instruments may call for a modified cut point. 5080 ANCILLARY RESEARCH OF THE QOL COMMITTEE MEMBERS JOURNAL OF CLINICAL ONCOLOGY Information downloaded from jco.ascopubs.org and provided by at NIH LIBRARY on December 3, 2010 from 184.108.40.206 Copyright © 2007 American Society of Clinical Oncology. All rights reserved. NCIC CTG Perspective aids or educational materials. Finally, our contributions to determining minimally important clinical changes in patients’ status were critical to the development of anchor-based methods for data analysis and to clinician understanding of the data generated by these trials.11 universally so); that patient-reported data on subjective experiences are likely to be more reliable and valid than conventional toxicity information; and that simple analyses that report group trends as well as individual patient response rates are preferred. AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST CONCLUSION We have provided a brief historical review of the development of PRO measurement, analysis, and reporting in clinical trials of the NCIC CTG. In doing so, we have provided lessons learned in furthering the quality of these data and their application to clinical practice. We conclude that institution of group policy and collective philosophy has enabled the routine incorporation of HRQOL into clinical trial protocols according to robust scientific principles; that collection of quality data is possible in a variety of circumstances (although not REFERENCES 1. Osoba D: The Quality of Life Committee of the Clinical Trials Group of the National Cancer Institute of Canada: Organization and functions. Qual Life Res 1:211-218, 1992 2. Osoba D, Bezjak A, Brundage M, et al: Evaluating health-related quality of life in cancer clinical trials: The National Cancer Institute of Canada Clinical Trials Group experience. Mayo Clinic Proc (in press) 3. Aaronson NK, Ahmedzai S, Bergman B, et al: The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 85:365-376, 1993 4. Osoba D, Aaronson N, Zee B, et al: Modification of the EORTC QLQ-C30 (version 2.0) based on content validity and reliability testing in large samples of patients with cancer: The Study Group on Quality of Life of the EORTC and the Symptom Control and Quality of Life Committees of the NCI of Canada Clinical Trials Group. Qual Life Res 6:103108, 1997 5. Osoba D: Guidelines for measuring healthrelated quality of life in clinical trials, in Staquet M, Hays RD, Fayers PM (eds): Quality of Life Assessment in Clinical Trials. Oxford, United Kingdom, Oxford University Press, 1998, pp 20-35 6. Rapp E, Pater JL, Willan A, et al: Chemotherapy can prolong survival of patients with advanced non-small-cell lung cancer: Report of a Canadian multicentre randomized trial. J Clin Oncol 6:633-641, 1988 7. Sadura A, Pater J, Osoba D, et al: Quality-oflife assessment: Patient compliance with questionnaire completion. J Natl Cancer Inst 84:1023-1026, 1992 8. Osoba D, Zee B: Completion rates in healthrelated quality-of-life assessment: Approach of the National Cancer Institute of Canada Clinical Trials Group. Stat Med 17:603-612, 1998 The author(s) indicated no potential conflicts of interest. AUTHOR CONTRIBUTIONS Conception and design: Michael Brundage, David Osoba, Andrea Bezjak, Dongsheng Tu, Michael Palmer, Joseph Pater Manuscript writing: Michael Brundage, David Osoba, Andrea Bezjak Final approval of manuscript: Michael Brundage, David Osoba, Andrea Bezjak, Dongsheng Tu, Michael Palmer, Joseph Pater 9. Palmer M, Zee B, Bezjak A, et al: The timing of baseline quality of life in clinical trials. Qual Life Res 9:337, 2006 (abstr) 10. Osoba D, Zee B, Pater J, et al: Determinants of postchemotherapy nausea and vomiting in patients with cancer: Quality of Life and Symptom Control Committees of the National Cancer Institute of Canada Clinical Trials Group. J Clin Oncol 15:116123, 1997 11. Osoba D, Rodrigues G, Myles J, et al: Interpreting the significance of changes in health-related quality-of-life scores. J Clin Oncol 16:139-144, 1998 12. Osoba D: Interpreting the meaningfulness of changes in health-related quality of life scores: Lessons from studies in adults. Int J Cancer Suppl 12:132-137, 1999 13. Norman GR, Sloan JA, Wyrwich KW: Interpretation of changes in health-related quality of life: The remarkable universality of half a standard deviation. Med Care 41:582-592, 2003 14. Osoba D, Bezjak A, Brundage M, et al: Analysis and interpretation of health-related quality-oflife data from clinical trials: Basic approach of The National Cancer Institute of Canada Clinical Trials Group. Eur J Cancer 41:280-287, 2005 15. Liu J, Tu D, Dancey J, et al: Quality of life analyses in a clinical trial of DPPE (tesmilifene) plus doxorubicin versus doxorubicin in patients with advanced or metastatic breast cancer: NCIC CTG Trial MA 19. Breast Cancer Res Treat 100:263-271, 2006 16. Brundage MD, Pater JL, Zee B: Assessing the reliability of two toxicity scales: Implications for interpreting toxicity data. J Natl Cancer Inst 85:11381148, 1993 17. Paul N, Pater J, Whitehead M, et al: Methods of toxicity data collection and evaluation of the relative effectiveness of the case resort flow sheet, the patient symptom diary, and the quality of life questionnaire. Control Clin Trials 12:648, 1991 (abstr) 18. Dancey J, Zee B, Osoba D, et al: Quality of life scores: An independent prognostic variable in a general population of cancer patients receiving chem- otherapy—The National Cancer Institute of Canada Clinical Trials Group. Qual Life Res 6:151-158, 1997 19. Pater J, Osoba D, Zee B, et al: Effects of altering the time of administration and the time frame of quality of life assessments in clinical trials: An example using the EORTC QLQ-C30 in a large anti-emetic trial. Qual Life Res 7:273-278, 1998 20. Bezjak A, Dixon P, Brundage M, et al: Randomized phase III trial of single versus fractionated thoracic radiation in the palliation of patients with lung cancer. Int J Radiat Oncol Biol Phys 54:719728, 2002 21. Bezjak A, Tu D, Seymour L, et al: Symptom improvement in lung cancer patients treated with erlotinib: Quality of life analysis of the National Cancer Institute of Canada Clinical Trials Group Study BR 21. J Clin Oncol 24:3831-3837, 2006 22. Levine MN, Pritchard KI, Bramwell VHC, et al: Randomized trial comparing cyclophosphamide, epirubicin, and fluorouracil with cyclophosphamide, methotrexate, and fluorouracil in premenopausal women with node-positive breast cancer: Update of National Cancer Institute of Canada Clinical Trials Group Trial MA5. J Clin Oncol 23:5166-5170, 2005 23. Savage C, Pater J, Tu D, et al: He said/she said: How much agreement is there on symptoms between common toxicity criteria and quality of life? Proc Am Assoc Cancer Res 21:386a, 2002 (abstr 1540) 24. Butler L, Bacon M, Carey M, et al: Determining the relationship between toxicity and quality of life in an ovarian cancer chemotherapy clinical trial. J Clin Oncol 22:2461-2468, 2004 25. Brundage M, Leis A, Bezjak A, et al: Cancer patients’ preferences for communicating clinical trial quality of life information: A qualitative study. Qual Life Res 12:395-404, 2003 26. Brundage M, Feldman-Stewart D, Leis A, et al: Communicating quality of life information to cancer patients: A study of six presentation formats. J Clin Oncol 23:6949-6956, 2005 ■ ■ ■ 5081 www.jco.org Information downloaded from jco.ascopubs.org and provided by at NIH LIBRARY on December 3, 2010 from 220.127.116.11 Copyright © 2007 American Society of Clinical Oncology. All rights reserved.