Download Identification of performance indicators across a

PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER
PROGRAMS,
Khare et al.
ORIGINAL
ARTICLE
Identification of performance indicators
across a network of clinical cancer programs
S.R. Khare mscn mba,* G. Batist md,† and G. Bartlett phd*
ABSTRACT
Background Cancer quality indicators have previously been described for a single tumour site or a single treatment
modality, or according to distinct data sources. Our objective was to identify cancer quality indicators across all
treatment modalities specific to breast, prostate, colorectal, and lung cancer.
Methods Candidate indicators for each tumour site were extracted from the relevant literature and rated in a
modified Delphi approach by multidisciplinary groups of expert clinicians from 3 clinical cancer programs. All
rating rounds were conducted by e-mail, except for one that was conducted as a face-to-face expert panel meeting,
thus modifying the original Delphi technique. Four high-level indicators were chosen for immediate data collection.
A list of confounding variables was also constructed in a separate literature review.
Results A total of 156 candidate indicators were identified for breast cancer, 68 for colorectal cancer, 40 for lung
cancer, and 43 for prostate cancer. Iterative rounds of ratings led to a final list of 20 evidence- and consensus-based
indicators each for colorectal and lung cancer, and 19 each for breast and prostate cancer. Approximately 30 clinicians
participated in the selection of the breast, lung, and prostate indicators; approximately 50 clinicians participated in
the selection of the colorectal indicators.
Conclusions The modified Delphi approach that incorporates an in-person meeting of expert clinicians is an
effective and efficient method for performance indicator selection and offers the added benefit of optimal clinician
engagement. The finalized indicator lists for each tumour site, together with salient confounding variables, can be
directly adopted (or adapted) for deployment within a performance improvement program.
Key Words Performance measures, quality indicators, health care quality assessment, quality improvement,
quality of cancer care
Curr Oncol. 2016 Apr;23(2):81-90
INTRODUCTION
Instituting quality indicators for the purpose of continuous
performance monitoring and subsequent improvement
has become a requisite component of health care delivery
in most institutions around the world. Quality of care
can be defined as “the degree to which health services for
individuals and populations increase the likelihood of
desired health outcomes and are consistent with current
professional knowledge”1. Given the emphasis on quality,
systematic measurement and reporting processes have
been implemented in many health care systems. After
significant investment in the supporting infrastructure
necessary for reliable data collection, several national
www.current-oncology.com
initiatives are now focused on close monitoring of quality
performance. Examples include the Dutch Health Care
Performance Reports, Danish hospital sector reports, the
English National Health Service Quality Accounts, U.S.
national reports on quality and disparities, national health
care quality reports by Belgium and Sweden, and provincial
performance reports in Canada 2.
National initiatives in the area of oncology are well
described, usually in the form of national cancer plans. England, for example, has instituted a cancer National Service
Framework with national directorship, a collaborative for
hospital-level service organization, and 34 cancer networks
for service coordination between hospitals3. Efforts in the
United States date back to 1922, with the formation of the
Correspondence to: Satya Rashi Khare, 5858 Côte-des-Neiges, Suite 300, Montreal, Quebec H3S 1Z1.
E-mail: [email protected] n DOI: http://dx.doi.org/10.3747/co.23.2789
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.
81
PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER PROGRAMS, Khare et al.
Commission on Cancer, a division of the American College
of Surgeons, tasked with standardizing and improving the
quality of cancer care4.
In Canada, the Canadian Partnership Against Cancer,
in collaboration with national and provincial partners, issues annual performance reports with specific metrics that
cover the continuum of cancer care and control5. Yet despite
the national collaboration, functions related to cancer care
delivery are independently planned and funded by each
province. The BC Cancer Agency, for example, operates
6 regional cancer centres and 4 specialized networks to
ensure collaboration and consistency of care across British
Columbia6. In Ontario, a Cancer Quality Council was established in 2002 in response to the lack of central oversight for
system-level cancer care performance. Together with Cancer
Care Ontario, the Council devised and implemented a performance monitoring scorecard balanced against a strategy-​
based framework7. Similar cancer-specific programs and
networks have been established in the provinces of Alberta,
Saskatchewan, and Manitoba8–10.
The province of Quebec is making progress toward a
province-wide cancer control strategy and improved capture of cancer-related data in a central registry. Advancements come mainly through the efforts of the Direction
québécoise de cancérologie and of community-based
organizations such as the Coalition Priorité Cancer au Québec. Until recently, no available system had the capacity for
adequate performance monitoring of the wide breadth of
cancer services—a situation that is perhaps most evident
in the annual Canadian Partnership Against Cancer reports, in which Quebec data were often excluded from the
analyses because of variations in definitions, unreliable
vital status of cancer cases, or simply unavailable data 5.
To address the need for improvement in the quality of
cancer care, the Rossy Cancer Network (rcn) was formed
in Montreal. The rcn is a network of 3 McGill University–
affiliated hospital cancer programs that aims to harmonize
cancer care activities and to provide a superior quality of
care. In 2012, the rcn collectively served approximately
11,500 new cancer patients11, representing roughly 24% of
all incident cases in Quebec12.
One of the missions of the rcn is to create a network-w ide standardized performance improvement
framework11. Collaboration between hospital leaders, who
are integral to the rcn’s governance, is facilitating a move
to common measurement and reporting activities that are
expected to set the stage for more uniform care delivery
and synergistic quality improvement initiatives. At the
core of all rcn activities is the objective to improve survival
times, reduce mortality, and increase patient satisfaction.
To accomplish those goals, relevant quality metrics have
to be identified and tracked over time.
The objective of the present study was to systematically
identify common quality indicators for continuous performance improvement across the network, with a focus on
the 4 major cancers: breast, prostate, colorectal, and lung.
outcomes. Each phase was separately deployed for each
of the tumour sites. Figure 1 presents a simplified view of
the process.
The Delphi technique involves iterative rounds with
controlled feedback to reach consensus by a group of experts in a systematic manner13; it is widely used in the selection of quality metrics across various clinical domains14–17.
The modified Delphi approach includes the addition of a
face-to-face meeting of expert participants18.
Phase I: Evidence and First Rating
Candidate performance measures in each tumour category were identified through reviews of published indicators
used by other cancer agencies, networks, or national
organizations. Examples include the BC Cancer Agency,
Cancer Care Ontario, the American Society of Clinical
Oncology, and the National Surgical Adjuvant Breast and
Bowel Project. Those reviews were supplemented by a
comprehensive review of the scholarly literature specific
to each tumour site. The name of the tumour site together
with the term “quality indicators” was used to identify
major papers, after which a snowball strategy was used
to identify secondary references.
All candidate indicators, organized by tumour site,
were tabulated under one of two network strategic categories: treatment (including outcomes of treatment) and
access to care (including patient flow). For indicators that
could have been appropriately placed in either category, the
“best fit” category was chosen. For example, indicators related to referrals, treatment on a clinical trial, presentation
at a tumour board, and clinical documentation were all included under the “access” category even though they were
also associated with treatment. Each tumour-specific list
was then sent in an e-mail message to all clinicians across
the network who had relevant involvement in care delivery for the respective tumour site (30 breast, 29 prostate,
37 lung, and 49 colorectal clinicians). The clinicians were
asked to use a 5-point Likert scale (0–4: strongly oppose
indicator inclusion to strongly support indicator inclusion)
to rate each indicator according to these guiding criteria:
■■
■■
Relevance to cancer care quality (a measure of indicator validity)
Importance to cancer care quality (a measure of indicator value)
METHODS
Using a 3-phase modified Delphi approach, quality metrics
were developed for treatment, access and patient flow, and
82
FIGURE 1 Modified Delphi approach for indicator selection.
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.
PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER PROGRAMS, Khare et al.
■■
■■
■■
Ability to benchmark results (a measure of the ability
to set targets)
Representative of an emerging practice
Applicable across network cancer programs (a measure of appropriateness for the rcn)
Raters were also encouraged to add free-text modifications or comments.
No time limit was set for responses, and weekly e-mail
reminders were sent to maximize participation during approximately 1 month. Results from the completed ratings
were manually collated by a member of the research team
(SRK). Indicators with an average rating of 2 or less (0, 1,
2) were eliminated from the list, and a second version containing the remaining indicators was compiled for phase ii
of the process.
Phase II: Face-to-Face Discussion and Iterative Ratings
Clinicians on the e-mail list for the first rating opportunity
were invited to participate in a face-to-face meeting to
discuss the remaining indicators. Four meetings were held,
one for each tumour site, with some panellists receiving
multiple requests if their area of expertise or care delivery
pertained to more than one tumour type.
The remaining indicators for each tumour site were
individually discussed for verbal rating and consensus
resolution to include or delete the indicator. Panellists
also discussed similar indicators that were redundant
and came to consensus on which indicator to keep and the
appropriate wording for the indicator. The guiding criteria
from phase i were also used in phase ii. Indicators that
passed the expert panel discussion, and any newly added
indicators, were collated into a version 3 list.
The version 3 list was sent by e-mail to members
who participated in either the first rating round or the
face-to-face discussion for their respective tumour site.
Panellists—still representing a multidisciplinary, multi-​
institutional panel—were asked to use the same 5-point
Likert scale to rate the indicators on the same criteria used
in phase i.
The results were collated, and indicators were again
eliminated if their average rating was 2 or less. This
rating process continued iteratively until the final lists
of indicators were achieved. The number of iterations
varied depending on the tumour site. Nonresponders to
preceding rounds were not excluded from responding to
subsequent rounds.
Phase III: High-Level Indicator Selection
To select high-level indicators for immediate data collection, the final indicator lists from each tumour site were
merged into one amalgamated list and sent to two members
of the research team (GBatist, SRK). Both members independently used predefined criteria (Is the indicator applicable to all four tumour sites? Reflective of the academic
nature of the rcn? Representative of best practices associated with clinical care?) to select 4 high-level indicators
from the amalgamated list. Criteria related to the validity
and appropriateness of each indicator were not included
here because those factors had already been confirmed
during the earlier consensus process. After the independent
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.
selections had been made, the choices were compared.
Once the two team members had achieved consensus, the
high-level indicator list, together with the amalgamated list
that represented all the indicators, was sent by e-mail to
all panellists who had attended the face-to-face meetings.
Panellists were asked to indicate whether they agreed or
disagreed with the selections, and to provide alternative
choices if they disagreed. Weekly e-mail reminders were
sent to maximize feedback during approximately 1 month.
After the 1-month period, panellist feedback was reviewed,
and the high-level indicators were finalized.
Identification of Confounding Variables
During the indicator selection process, concurrent work
was completed to identify confounding factors necessary
for stratified reporting and risk-adjusted analysis. A comprehensive literature review used the key phrases “cancer
and comorbidity interaction,” “cancer confounders,” “comorbidity indices and cancer,” “cancer outcomes and prognostic factors,” “prognostic indices and cancer,” “cancer
survival prediction,” “predicting cancer progression and
mortality,” and “comorbidity considerations in oncology”
to search electronic databases accessible through McGill
University, including medline, PubMed, the Cochrane Library, and articles found through Google Scholar. Additionally, patient and tumour characteristics generally included
in clinical research were reviewed through the Web sites
of various organizations and through published studies.
RESULTS
The literature review and environmental scan identified
156 potential indicators for the breast group, 68 for the
colorectal group, 40 for the lung group, and 43 for the
prostate group.
The first e-mail rating round eliminated 36 breast
indicators, 12 colorectal indicators, 2 lung indicators, and
15 prostate indicators, leaving 120 breast, 56 colorectal, 38
lung, and 28 prostate indicators for discussion at the expert
panel meeting.
On average, the length of the face-to-face discussion
was 3 hours. The research team moderated the discussions.
The breast and prostate meetings were attended by 10 clinicians; the lung meeting, by 13; and the colorectal meeting,
by 9. All groups included at least one surgical oncologist,
medical oncologist, radiation oncologist, and pathologist.
The directors of Oncology at each of the 3 network hospitals
attended the meeting that most closely matched their area
of expertise. Nurses and allied health professionals were
invited to all sessions, but were in attendance only at the
breast and lung meetings.
Although most indicators reflected evidence-based
guidelines for clinical care (grade 1 or 2 measures), additional empirically supported indicators not otherwise
included, or indicators based on a high degree of consensus in the expert provider group (grade 3 measures)
were added to the lists during the discussions. After the
verbal ratings and consensus concerning the inclusion
or exclusion of the remaining indicators, the candidate
list was further cut to 60 breast, 29 colorectal, 36 lung,
and 31 prostate indicators. Of those indicators, 4 lung, 4
83
PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER PROGRAMS, Khare et al.
prostate, and 2 colorectal indicators were newly added (1
of which had previously been eliminated). Indicators were
also reworded for improved clarification and accuracy.
After the face-to-face discussion, the breast group went
through 1 e-mail rating cycle to eliminate 41 indicators, the
colorectal group went through 2 e-mail cycles to eliminate
9 indicators, the lung group went through 1 e-mail cycle
to eliminate 16 indicators, and the prostate group went
through 2 e-mail cycles to eliminate 12 indicators. The final
list of evidence- and consensus-based indicators numbered
20 for the lung and colorectal sites and 19 for the breast and
prostate sites. Considering all the e-mail rating opportunities combined, approximately 30 clinicians participated
in the selection of the breast, prostate, and lung indicators,
and approximately 50 clinicians, in the selection of the
colorectal indicators. Figure 2 summarizes the progressive
reduction of indicators.
The final indicator lists were a mix of process and
outcome measures, all of which are shown in Tables i, ii,
iii, and iv. Measures related to structure were ref lected
in other categories not discussed here. Process measures
accord with best practice and cut across disciplines
(surgery, radiation therapy, chemotherapy, psychosocial
oncology, palliative care, pathology, and genetics). Measures related to access addressed wait times for diagnostics and treatment, clinical trial enrolment, presentation
to a tumour board, and referral to genetic counselling.
Outcome measures were focused on treatment toxicities,
surgical complications, relapse or recurrence, survival,
and mortality.
Table v lists the 4 high-level indicators slated for immediate data collection. They are relevant to all 4 tumour sites
of interest, reflective of the academic nature of the rcn, and
reflective of best practices associated with clinical care. The
two members of the research team who had independently
selected the indicators from the amalgamated list had identical choices, and consensus by the expert panellists was
achieved in 1 e-mail round, with no disagreement about the
selections. The high-level indicators address time to initial
treatment, treatment toxicity, use of multidisciplinary tumour boards, and participation in clinical trials.
Patient and tumour characteristics that could act as
confounding factors in the analysis phase were reduced
from a wide selection of prognostic factors, including comorbidities, to a base model that would lend itself to feasibility in data collection. Table vi lists those characteristics.
DISCUSSION
Quality measures for breast, prostate, colorectal, and lung
cancer were extracted from the national and international
literature, with final lists compiled through expert consensus achieved using a modified Delphi approach. Each set
of measures is rooted in evidence-based clinical practice
and includes both process and outcome metrics relevant
to quality measurement. The lists, and the process used
to create them, could easily be transferred to similar performance improvement initiatives in other jurisdictions.
Furthermore, confounding factors that could be useful in
the analysis phase are provided. Our work expands on previous studies that have focused on a single tumour site or a
84
FIGURE 2 Phased reduction of indicators.
single treatment modality, or on measures that correspond
with local existing datasets3,16,19.
High-level indicators were chosen so that data collection efforts would prioritize indicators pertinent to all
tumour sites; incremental incorporation of all other indicators is planned. The chosen indicators address access
to treatment and the incidence of serious toxicities. They
also address frequency of patient presentation at a multidisciplinary tumour board, which has been associated with
better outcomes20, and participation in clinical trials. Both
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.
PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER PROGRAMS, Khare et al.
TABLE I Treatment and access and patient flow performance indicators for breast cancer
Indicator
Description
Treatment
1
Percentage of patients with early-stage breast cancer (stage I or II) and clinically negative axillary nodes who receive sentinel node biopsy
2
Complete synoptic pathology report according to the Canadian Association of Pathologists or Rossy Cancer Network guidelines
3
Percentage of patients with involvement of axillary lymph nodes (1–3 nodes or more) who received adjuvant radiation
4
Percentage of patients with estrogen receptor–negative invasive carcinoma (tumour > 1 cm or node-positive) who received adjuvant
chemotherapy within 8 weeks of surgical resection
5
Percentage of patients with inflammatory breast cancer or locally advanced nonresectable estrogen receptor–negative carcinoma who
received neoadjuvant chemotherapy
6
Percentage of patients with stage III breast cancer who underwent baseline staging imaging, including bone scan, liver ultrasonography,
and chest radiography
7
Percentage of patients who received systemic-relapse post-adjuvant therapy within 5 years of diagnosis
8
Percentage of patients with primary operable breast cancer who developed first recurrence to ipsilateral breast or skin or chest wall (or
both) within 5 years after mastectomy or breast-conserving surgery
9
Percentage of biopsies performed at first site of metastasis (stage IV patients)
10
Percentage of patients receiving chemotherapy with grade 4 toxicity
Access and patient flow
11
Time from abnormal mammogram to diagnostic biopsy
12
Time from diagnostic biopsy to initial breast cancer surgery
13
Percentage of breast cancer patients treated on a clinical trial
14
Percentage of breast cancer patients offered referral to genetics for evaluation and counselling
15
Percentage of breast cancer patients presented to the multidisciplinary tumour conference (tumour board) at any time after diagnosis
16
Wait time for adjuvant radiation therapy from the final pathology report
17
Wait time for systemic adjuvant therapy from the final pathology report
18
Wait time for first-line chemotherapy for metastatic disease, from medical oncology visit that decides on chemotherapy
19
Wait time for computed tomography or magnetic resonance imaging from doctor’s requisition
of the foregoing indicators also reflect the academic nature
of the university-affiliated cancer programs.
As an initial starting point, the chosen indicators
constitute the first rcn Quality Report; they also serve to
reveal data quality issues that have to be corrected as work
proceeds. The end goal of the process was to report on
standardized quality metrics for comparison and improvement of care delivery, including efficient use of services as
reflected in selected measures that address the underuse
of effective testing for patients who could benefit and the
overuse of unnecessary testing for patients in whom the
potential for benefit is limited4.
The final indicator lists represent a wide range of measurement points that should promote a balanced perspective for improving health care quality. Improvement in one
area at the expense of another would contradict the concept
of high-quality care, a concept that was an important consideration in the U.K. Quality Outcomes Framework, which
involved financial incentives for achieving predetermined
quality thresholds2,21.
The selected confounding factors should help to determine whether observed differences in outcomes across the
network are indeed attributable to differences in the quality
of care as opposed to differences in the case or hospital
mix. In the absence of data stratification and risk-adjusted
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.
analysis, outcome measures are as ineffectual as process
measures that have no scientifically sound link to improved
outcomes22,23. Compliance with evidence-based processes
requires that statistical parallels to improved patient outcomes remain valid. Similarly, to safeguard the validity
of results, factors that can influence the interpretation
of outcome measures must be assessed and statistically
accounted for15,22–25.
Most of the selected measures have been implemented
in health care systems operating outside of Quebec, thus
presenting questions about validity in the local context.
Although true validity will be expected to be discovered
through responsiveness to subsequent improvement initiatives, review and selection by an expert multidisciplinary
panel of clinicians with vast experience and familiarity
with local care delivery processes added to the validity of
the indicators in the local context.
A further advantage of inclusivity among the participating local clinicians is the creation of a bottom-up
approach that serves to maximize buy-in because of frontline decision-making concerning what defines quality in
cancer care26. The final indicator lists were achieved in
a systematic elimination process using iterative e-mail
correspondence and a face-to-face meeting with a subset of the original expert panel to engage in an in-depth
85
PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER PROGRAMS, Khare et al.
TABLE II Treatment and access and patient flow performance indicators for prostate cancer
Indicator
Description
Treatment
1
Number of needle cores per biopsy
2
Complete synoptic pathology report according to the Canadian Association of Pathologists or Rossy Cancer Network guidelines
3
Percentage of patients with metastatic disease treated with first-line systemic therapy
4
Percentage of patients with bone metastases receiving bone-targeted therapy (for example, bisphosphonates or RANK ligand inhibitor)
5
Percentage of patients with positive surgical margins, by stage
6
Percentage of patients with positive margins and prostate-specific antigen (PSA) between 0.2 ng/mL and 0.5 ng/mL who receive radiation
therapy
7
Percentage of patients receiving radiotherapy who have Radiation Therapy Oncology Group grade 3 or higher rectal or bladder toxicity
during the treatment period
8
Percentage of patients with acute surgical complication within 30 days (blood loss of 2.0 L or more; rectal injury; cardiovascular
complications such as arrhythmias, myocardial infarction, heart failure, or pulmonary edema; proximal deep-vein thrombosis or pulmonary
embolism; infection; or placed on long-term anticoagulant therapy)
9
Blood transfusion rate from the surgical start time, to and including 72 hours postoperatively
10
Hospitalization rate within 30 days of treatment, and diagnosis code at time of admission
11
Percentage of patients with significant urinary incontinence (>2–3 pads daily) at 1 year after surgery
12
Biochemical disease-free and overall survival at 5, 10, and 15 years after primary treatment by radical prostatectomy or radiation therapy,
by stage of disease
Access and patient flow
13
Time between positive biopsy showing high-risk disease (clinical stage T3-4, or Gleason score 8–10, or PSA > 20 ng/mL at diagnosis) and
initiation of one or more of these treatments: radiation therapy, systemic therapy, surgery
14
Percentage of low-risk patients (clinical stage T1–2a, and Gleason score ≤ 6, and PSA < 10 ng/mL at diagnosis) with documentation of
discussion about treatment options and adverse effects
15
Percentage of patients having undergone definitive therapy for prostate cancer who are followed at least twice in the first year and at
least annually thereafter
16
Percentage of patients with high-risk disease (clinical stage T3-4, or Gleason score 8–10, or PSA > 20 ng/mL at diagnosis) who undergo
general staging tests (pelvic computed tomography, magnetic resonance imaging, and bone scan)
17
Percentage of castration-resistant metastatic patients referred to a medical oncologist or multidisciplinary tumour board
18
Median length of stay after radical prostatectomy
19
Percentage of prostate cancer patients treated on a clinical trial
discussion and debate about further eliminations. The
opportunity for face-to-face discussion was a modification
of the Delphi technique that proved valuable not only in
the context of achieving consensus, but also in building a
sense of ownership of the process and its results. Consultation with those who would be responsible for uptake of
the results and action in the form of care improvements
has been shown to be an effective approach in other quality improvement programs. For example, an important
component of the success of the U.S. National Surgical
Quality Improvement Program is surgeon ownership of
the selection and definition of common quality measures
used for continuous tracking 24.
Alternatively, a top-down approach was used in the
selection of a strategy-based quality framework. Several
frameworks have been proposed to guide quality monitoring and improvement, including the well-known U.S.
Institute of Medicine paradigm that emphasizes effective,
safe, timely, efficient, patient-centred, and equitable care1.
Other common frameworks are organized either using the
86
cancer care continuum (from prevention to end-of-life) or
institutional goals and objectives—and sometimes both5,7.
Choosing an appropriate framework depends largely on
the internal environment and the stage of organizational
development. During nascent stages, a strategy-based
framework that aligns with overarching goals and objectives offers two distinct advantages: communication of
goals to all key stakeholders, and effective measurement
of progress toward goal attainment (Brown A, Institute
of Health Policy, Management and Evaluation. Personal
communication, 2012). At later stages in development, it
might be perfectly reasonable to integrate a framework
that reflects the patient journey or the quality dimensions
mentioned earlier.
Selection of performance indicators, followed by
measurement and eventual improvements, is a process
that is successful when all pertinent players are involved
and engaged. An integral component of our work was the
up-front engagement of clinicians and hospital administration across the 3 clinical cancer programs. Soliciting
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.
PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER PROGRAMS, Khare et al.
TABLE III Treatment and access and patient flow performance indicators for lung cancer
Indicator
Description
Treatment
1
Complete synoptic pathology report according to the Canadian Association of Pathologists or Rossy Cancer Network guidelines
2
Percentage of patients undergoing curative localized therapy (either surgery or chemoradiation) who receive positron-emission tomography
before treatment
3
Percentage of patients diagnosed with nonsquamous and non-small-cell disease with assigned EGFR and ALK status, by stage
4
Percentage of patients with metastatic lung cancer treated with cytotoxic chemotherapy during the last 2 weeks of life
5
Overall survival by stage at initial therapy
6
Percentage of patients with validated biomarker who receive appropriate targeted therapy
7
Percentage of lobectomies performed by video-assisted thoracoscopic surgery
8
Number of lymph nodes retrieved during lobectomy
9
Percentage of patients who die within 30 days of surgery
10
Percentage of patients receiving systemic therapy experiencing grade 3 or 4 toxicity
Access and patient flow
11
Clinical stage at diagnosis in any of the network hospitals
12
Time from first abnormal chest radiograph to pathology diagnosis
13
Wait time for final pathology (histologic assignment and genotyping)
14
Wait time for diagnostic imaging
15
Percentage of lung cancer patients presented at a multidisciplinary tumour conference (tumour board)
16
Wait time from booking curative thoracic surgery to procedure
17
Wait time from referral for curative radiation therapy to treatment
18
Wait time to systemic therapy for metastatic disease
19
Percentage of patients with metastatic lung cancer referred to outpatient palliative care services
20
Percentage of lung cancer patients treated on a clinical trial
input from all key players early in the process is expected
to help establish a sustainable performance improvement program coupled with a complementary culture of
patient-focused practice.
Limitations
Several process limitations deserve consideration. Some
of the selected measures require clinical information not
routinely collected by local registries, thus threatening
the feasibility of data collection. The selection process did
not focus on feasibility as a selection criterion, although
feasibility was certainly taken into account after the lists
had been created. In the absence of a properly coded electronic medical record, a more focused a priori approach to
anticipated data collection challenges would perhaps have
changed the finalized lists. However, the goal during the
process was to identify metrics truly reflective of quality
of care, per consensus by those who deliver the care and
would inevitably be accountable for performance results.
Contrary to work published by others15,17,25, the focus of the
network was not on what our systems could provide, but
rather what we needed our systems to provide. A second-tier review of the selected measures by a methods panel to
prioritize them according to feasibility and validity considerations27 could be a valuable addition to the modified
Delphi technique.
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.
On a related note, some of the clinical information
required to complete abstraction of a measure might not
be consistently documented, with adverse effects on data
reliability. Conversely, feedback about underperformance
of measures that are known to be poorly documented could
lead to better documentation efforts, as was shown in a
study that reported results for chemotherapy administration in stage iii colon cancer28.
Another process-related limitation was one in which
a solution is not readily apparent. In the effort to elicit
multiple viewpoints, invitations to the multidisciplinary
rating panels were extended to all relevant disciplines, including nurses, physiotherapists, dietitians, psychologists,
and social workers. Despite that effort, panel involvement
was, for the most part, limited to physicians and (for some
tumour sites) nurses. Considering the candidate indicators
presented for consensus elimination, the strength of the
final indicator lists was most likely not affected; however,
the opportunity for wider buy-in might have been reduced.
The adoption (or adaptation) of the performance
indicator lists described here might be limited by a few
factors. One that was already mentioned is the feasibility
of data collection, which can vary between institutions.
Another is the fact that the indicators were selected within
a strategic framework and thus reflect the goals of the
network in which they were developed. That being said,
87
PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER PROGRAMS, Khare et al.
TABLE IV Treatment and access and patient flow performance indicators for colorectal cancer
Indicator
Description
Treatment
1
Complete synoptic pathology report according to the Canadian Association of Pathologists or Rossy Cancer Network guidelines
2
Percentage of patients with rectal cancer undergoing surgery with a positive distal or radial margin
3
Percentage of patients undergoing surgery or radiation therapy for rectal cancer who receive pre-treatment imaging of the pelvis
with magnetic resonance imaging (MRI) within the preceding 1 month
4
Percentage of patients undergoing surgery for colon or rectal cancer who receive preoperative chest, abdominal, or pelvic computed
tomography and MRI for rectal cancer only
5
Percentage of patients with rectal cancer undergoing sphincter-saving resection
6
Percentage of patients with stage III colon cancer who commence adjuvant chemotherapy within 8 weeks of surgery
7
Percentage of patients receiving chemotherapy who experience grade 3 or 4 toxicity
8
Percentage of patients undergoing surgery for rectal cancer in whom continuity is re-established and who experience an anastomotic
leak
9
Percentage of patients having undergone colon or rectal cancer surgery who experience an unplanned return to the operating room
within 28 days
10
Percentage of patients who die within 28 days of non-emergent colon or rectal cancer surgery, excluding multivisceral surgery
11
Rate of local recurrence within 5 years for patients who have had rectal cancer surgery, by stage
12
Percentage of colon or rectal cancer patients with systemic relapse within 5 years after adjuvant therapy, by initial stage
13
5-Year and adjusted 5-year overall survival rates for colon or rectal cancer patients, by stage
Access and patient flow
14
Percentage of patients with a family history of colorectal cancer offered referral to genetics
15
Time between diagnostic biopsy and initial colorectal local therapy
16
Wait time for computed tomography or MRI for staging
17
Percentage of colorectal cancer patients treated on a clinical trial
18
Percentage of patients with colon or rectal cancer, not treated with preoperative chemotherapy or radiotherapy, admitted for surgery
within 8 weeks from the time of first surgical consultation
19
Percentage of patients with known or suspected stage II or III rectal cancer who see a radiation oncologist or are presented to a
multidisciplinary tumour board preoperatively or within 4 weeks postoperatively
20
Percentage of patients with stage II colon cancer whose case is reviewed by the tumour board or medical oncologist within 4 weeks
TABLE V High-level performance indicators for all cancers
Indicator
Description
1
Time between diagnosis and initial treatment, with specification
of treatment modality
2
Percentage of patients with high-grade (3 or 4) acute toxicity
with cytotoxic chemotherapy
3
Percentage of patients presented to a multidisciplinary tumour
board at any time after diagnosis
4
Percentage of patients treated on a clinical trial at any time
after diagnosis
TABLE VI Base model of patient- and tumour-related characteristics
Confounding variables
Demographic
Tumour-related
Age at diagnosis
Primary cancer site
Sex
Date of diagnosis
Postal code
Stage of disease
Grade of disease
88
operationalization of the indicators in any evidence-based
oncology practice would still be applicable, but might
not cover the spectrum of care services within the local
framework. For example, literature related to prevention
and screening was not applicable in the rcn context and
was thus not reflected in the indicator lists. Additionally,
other local contexts might vary widely from the rcn in care
delivery practices and patterns, and thus would require
validity of the indicators within their unique context.
Lastly, as is the case with all performance indicators in
any mode of care, constant re-evaluation and modification
will be required as new research is published and synthesized, with subsequent changes to best practice.
CONCLUSIONS
Performance improvement programs aimed at ensuring
high quality-of-care standards are a fundamental component of health care delivery. Health care systems around
the world have instituted national systems of quality
monitoring, including those focused on cancer. Although
Canada, and more specifically Quebec, holds efficient
and effective cancer care as a priority, central oversight
of the broad range of cancer services remains elusive.
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.
PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER PROGRAMS, Khare et al.
In response, small-scale interinstitutional networks can
build common platforms of quality monitoring to ensure
high-performance cancer care and measurement of its
effect on patient outcomes.
The rcn initiative, through a common performance
improvement methodology, aims to ensure care of equal
standard regardless of where within the network the care
is delivered. To that end, standard performance indicators
for use across the network were developed for the top 4
tumour sites. The next phase of work will focus on the
data abstraction process, including the identification of
data sources, reliability of the information, and feasibility
of data collection. That information will be used to plan
the building of a data warehouse and to transform the
supporting technological infrastructure.
In the context of cancer care, whereby patients are
increasingly forced to seek a full range of services from
multiple providers who might practice across several institutions, integrated performance monitoring and improvement are necessary. When efforts are fragmented between
institutions, the chances of inconsistent results increase,
potentially translating into varying quality of care, with
poor patient experiences, ineffective and inefficient care,
and worst of all, care that is unsafe.
ACKNOWLEDGMENTS
This work was conducted under the rcn mandate, an initiative
made possible by a generous financial contribution from the
Larry and Cookie Rossy Family Foundation. We thank the rcn
leadership, the directors of Oncology at the 3 cancer programs,
and the health care providers who dedicated significant time and
expertise during the completion of this work.
CONFLICT OF INTEREST DISCLOSURES
We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare that we have none.
AUTHOR AFFILIATIONS
*Department of Family Medicine, McGill University, Montreal,
QC; †Segal Cancer Centre, Jewish General Hospital, and Rossy
Cancer Network, McGill University, Montreal, QC.
REFERENCES
1. United States, The National Academies, Institute of Medicine,
Committee on Quality of Health Care in America. Crossing
the Quality Chasm: A New Health System for the 21st Century.
Washington, DC: National Academies Press; 2001.
2. Organisation for Economic Co-operation and Development.
Improving Value in Health Care: Measuring Quality. Paris,
France: Forum on Quality of Care; 2010.
3. McCarthy M, Gonzalez-Izquierdo A, Sherlaw-Johnson C,
Khachatryan A, Coleman MP, Rachet B. Comparative indicators for cancer network management in England: availability, characteristics and presentation. BMC Health Serv
Res 2008;8:45.
4. Schneider EC, Malin JL, Kahn KL, Emanuel EJ, Epstein AM.
Developing a system to assess the quality of cancer care:
asco’s national initiative on cancer care quality. J Clin Oncol
2004;22:2985–91.
5. Canadian Partnership Against Cancer (cpac). The 2012 Cancer
System Performance Report. Toronto, ON: cpac; 2012.
6. BC Cancer Agency (bcca). About > Who We Are [Web page].
Vancouver, BC: bcca; n.d. [Available at: http://www.bccancer.
bc.ca/about/who-we-are; cited 10 July 2014]
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.
7. Greenberg A, Angus H, Sullivan T, Brown AD. Development
of a set of strategy-based system-level cancer care performance indicators in Ontario, Canada. Int J Qual Health C
2005;17:107–14.
8. Alberta Health Services (ahs). Provincial Advisory Council:
Cancer. Edmonton, AB: ahs; 2014. [Available online at: http://
www.albertahealthservices.ca/assets/about/ac/ahs-paccancer-brochure.pdf; cited 10 July 2014]
9. Saskatchewan Cancer Agency (sca). Community Oncology
Program of Saskatchewan (COPS) [Web page]. Regina, SK:
sca; 2014. [Available at: http://www.saskcancer.ca/Default.
aspx?DN=20625c14-d634-4748-9509-83d11437af7f; cited
10 July 2014]
10. Cancer Care Manitoba. About Us > Programs [Web page].
Winnipeg, MB: Action Cancer Manitoba; 2016. [Available at:
http://www.cancercare.mb.ca/home/about_us/programs/;
cited 10 July 2014]
11. Sullivan T, Huynh T, Aprikian A, et al. The Rossy Cancer
Network: a philanthropy-driven quality program. Oncology
Exchange 2013;12:25–8.
12. Canadian Cancer Society’s Steering Committee on Cancer
Statistics. Canadian Cancer Statistics 2012. Toronto, ON:
Canadian Cancer Society; 2012.
13. Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods:
characteristics and guidelines for use. Am J Public Health
1984;74:979–83.
14. Gear C, Koziol-McLain J, Wilson D, et al. Primary healthcare
response to family violence: a Delphi evaluation tool. Qual
in Primary Care 2012;20:15–30.
15. Wilson S, Hauck Y, Bremner A, Finn J. Quality nursing care
in Australian paediatric hospitals: a Delphi approach to
identifying indicators. J Clin Nurs 2011;21:1594–605.
16. Gagliardi AR, Simunovic M, Langer B, Stern H, Brown AD.
Development of quality indicators for colorectal cancer
surgery, using a 3-step modified Delphi approach. Can J Surg
2005;48:441–52.
17. Watkins K, Horvitz-Lennon M, Caldarone LB, et al. Developing
medical record-based performance indicators to measure the
quality of mental healthcare. J Healthc Qual 2011;33:49–66.
18. Wilson D, Koziol-McLain J, Garrett N, Sharma P. A hospita l-ba sed ch i ld protect ion prog ra m me eva luat ion
instrument: a modified Delphi study. Int J Qual Health C
2010;22:283–93.
19. Del Turco MR, Ponti A, Bick U, et al. Quality indicators in
breast cancer care. Eur J Cancer 2010;46:2344–56.
20. El Saghir NS, Keating NL, Carlson RW, Khoury KE, Fallowfield
L. Tumor boards: optimizing the structure and improving
efficiency of multidisciplinary management of patients with
cancer worldwide. Am Soc Clin Oncol Educ Book 2014:e461–6.
[Ava i lable on line at: http://meet ing libra r y.asco.org/​
content/11400461-144; cited 16 July 2015]
21. Prescribing and Primary Care Team. Quality and Outcomes
Framework Achievement, Prevalence and Exceptions Data,
2012/13. Leeds, UK: Health and Social Care Information
Centre; 2013.
22. Mant J. Process versus outcome indicators in the assessment of qualit y of health care. Int J Qual Health Care
2001;13:475–80.
23. Malin JL, Asch SM, Kerr EA, McGlynn EA. Evaluating the
quality of cancer care. Cancer 2000;88:701–7.
24. Piccirillo JF, Tierney RM, Costas I, Grove L, Spitznagel EL Jr.
Prognostic importance of comorbidity in a hospital-based
cancer registry. JAMA 2004;291:2441–7.
25. Maggard-Gibbons M. The use of report cards and outcome
measurements to improve the safety of surgical care: the
American College of Surgeons National Surgical Quality
Improvement Program. BMJ Qual Saf 2014;23:589–99.
89
PERFORMANCE INDICATORS ACROSS A NETWORK OF CANCER PROGRAMS, Khare et al.
26. Mountford J, Shojania KG. Refocusing quality measurement to best support quality improvement: local ownership of quality measurement by clinicians. BMJ Qual Saf
2012;21:519–23.
27. Desch CE, McNiff KK, Schneider EC, et al. American Society of
Clinical Oncology/National Comprehensive Cancer Network
quality measures. J Clin Oncol 2008;26:3631–7.
90
28. Stewart AK, Gay EG, Patel-Parekh L, Winchester DP, Edge
SB, Ko C Y. Prov ider feedback improves repor t ing on
quality measures: National profile reports for adjuvant
chemotherapy for stage iii colon cancer [abstract 6572]. J
Clin Oncol 2007;25:. [Available online at: http://meeting.
ascopubs.org/cgi/content/short/25/18_suppl/6572; cited
5 February 2016]
Current Oncology, Vol. 23, No. 2, April 2016 © 2016 Multimed Inc.