Download CLINICAL, SIGNIFICANCE AS IT RELATED TO EVIDENCE BASED

CLINICAL, SIGNIFICANCE AS IT RELATED TO EVIDENCE BASED
PRACTICE; A CONCEPT ANALYSIS
By
SUSAN KRISTINE SCHMIDT BRlTNER
A Non-Thesis Project submitted in partial fulfillment of
the requirements for the degree of
MASTER OF NURSING
WASHINGTON STATE UNIVERSITY
College of Nursing
DECEMBER 2010
To the Faculty of Washington State University:
The members of the Committee appointed to examine the dissertation/thesis of SUSAN
KRISTINE SCHMIDT BRUNER find it satisfactory and recommend that it be accepted.
Cindy Corbett, Ph.D., Chair
Brian Gates, Pharm.D.
~r~~~
Ryan P. Townsend, M.N.
For Submission to: Journal ofAdvanced Nursing
Abstract
Aim. Analyzes the concept of clinical significance in relation to evidence-based practice.
Background. There has been insufficient inquiry into the evidence base of the definition and use
of the term clinical significance in nursing. Delineating a common conceptual meaning and
definition for clinical significance will enhance its usefulness in research and practice.
Methods
Rodgers' evolutionary approach to concept analysis provides the framework for interpreting
clinical significance in the context of its uses, measurements, and defInitions in the nursing
literature. After applying inclusion and exclusion criteria and a sampling method, 53 articles
were included in the analysis of clinical significance. Using Rodgers' rigorous, evolutionary
method of concept analysis, the concept of clinical significance was examined for its
significance, use and application as it has unfolded between 1999 and 2010.
Results
Findings indicate that the nursing literature contains various, disparate definitions and
applications of clinical significance.
Conclusion
The disparate uses of clinical significance negatively impact standardizing and quantifying
research outcomes to discern evidence-based practices. The authors propose a definition of
clinical significance inclusive of the multifarious uses that were revealed in the literature and
conclude tllat there is a need for professional nursing consensus to define the term. A standard
operational definition of clinical significance would enable consistency as clinicians interpret
research findings and facilitate translating research to practice.
Introduction
In 2002, the Mayo Foundation for Medical Education, in collaboration with over 30
collective authors as members of the Clinical Significance Consensus Meeting, published a
series of reports describing several problems with the use of the term clinical significance (CS).
These problems included inconsistent measurements, changing definitions over time, varying
opinions among stakeholders, theoretical inconsistencies, and unclear guidelines for determining
CS in group versus individual differences or single item versus multi-item instrument scores. The
consensus panel asserted that these problems interfere with the ability of clinicians to clearly
interpret research data and detract from the credibility ofCS as a concept (Guyatt et. aI., 2002;
Sloan et. al., 2002; Sloan et. al., 2003).
The importance of CS as concept was illustrated in its comparison to statistical
significance: while the P values of a statistically significant fmding indicate the likelihood of the
change being caused by chance, CS goes beyond statistical significance to identify whether the
statistically significant change or difference was large enough to have implications for patient
care (Sloan et. aI., 2002; Pittman & Rawls, 2008). The potential utility of a concept that
consistently indicated significant patient change was perceived to have positive implications for
stakeholders including researchers, clinicians, society, and patients. Thus further research by
physician scientists ensued to resolve identified disparities and determine valid and reliable
measurements of CS. Substantive progress has been made to develop and define methods of
quantifying CS. The psychology/psychiatry (Jacobso~ Follette & Revenstorf, 1984; Jacobson &
Truax, 1991; Jacobson, Roberts, Berns & McGlinchey, 1999) and pain management (Mantha et.
al., 1993) specialties have operationalized CS and formal measurement methods. Oncology
(Frost et. aI., 2002; Guyatt et. al., 2002; Sloan et. aI., 2002; Sprangers et. aI., 2002 & Symonds,
Berzon, Marquis & Rummans, 2002), pulmonary medicine (Lacass et. al., 2002; Puhan et. al.,
2004), rehabilitation (Haley & Fragala-Pinkham, 2006), surgery (Copay et. al., 2008; Carreon,
Glassman, Campbell & Anderson, 2010), and other medical disciplines have made great strides
in this important undertaking. These combined medical efforts lend credence to the importance
of undertaking efforts to determine valid and reliable measures for CS.
As an indicator of patient change, CS has evolved in healthcare to provide measures
observable to the clinician/researcher and patient. The patient's self assessment of change has
evolved as the central measure of CS. One current interpretation of CS as a research and clinical
term is as follows: a minimal clinically significant/important difference represents the smallest
difference in score in the outcome of interest that informed patients or proxies perceive as
important, either beneficial or harmful, and which would lead the patient or clinician to consider
a change in patient management (Schunemann et. aI., 2005). Inclusion of the patient into the
calculation of CS has important implications for evidence-based practice (EBP) because patient
values are critical to the achievement of best practices (Jones, 2010; Melnyk & FineoutOverholt, 2011).
While medicine has made substantial progress in defining CS, nursing has not made
similar attempts to define the term and substantiate it as a measure of patient response to
treatment that fulfills evidence-based standards. Indeed, there has been no formal, integrated
theoretical research undertaken by the nursing discipline to define an operational use of the term
CS (Jeans, 1992; Burns & Grove, 2009; Dysvik et aI, 2009) and there is a general lack of clarity
about CS in the nursing literature. It has been variously defined as anything of relevance to
nursing (Jeans, 1992), anything that is worth changing practice for (Fawcett, 2009), and- a value
judgment (LeFort, 1993). In the contemporary nursing literature, CS has been variously defmed
as: a measure of patient response to treatment based on pre-determined thresholds (Pittman &
Rawls 2008); a change that is at least twice the standard deviation of difference in scores due to
measurement errors (Dysvik, Kvaloy, Stokkeland, Natvig, 2009), and meaningful individual
change exceeding the Reliable Change Index value that results in the final (post-test) score
falling within the 'normal range' (Baird, Worral, Haslam & Haslam, 2010). Clearly, within the
nursing discipline a standard definition of CS has not materialized.
Standardizing defInitions, measures, and methodology is essential to disseminating best
practices. The lack of clarity when defIning CS contributes to uncertainty about the quality of
evidence represented by a CS change. According to Rodgers, a concept analysis is necessary
when there are inconsistencies in terminology, utilization, or lack of theoretical coherence
(Rodgers, 2000). The purpose of this paper is to use Rodgers' evolutionary concept analysis to
discern a "state of the art" consensus for CS. A "state of the art" consensus is intended to
advance the development of the term to promote consistency in interpreting research fmdings
and facilitate translating research into evidence.
Methods
An extensive review of the literature was conducted. Literature searches were performed
to identify articles pertinent to clinical significance. The electronic databases CINAHL, lSI Web
of Knowledge, PubMed, Cochrane Database of Systematic Reviews, and Google Scholar were
searched using the following keywords: clinical significance, clinical significance measures,
clinically significant, clinically meaningful, clinically important, and evidence based practice.
The search included papers published between the January 1999 and December 2010. Additional
searches were completed on several websites, such as those of the Agency for Healthcare
Research and Quality, The Joint Commission, and the Center for Medicare and Medicaid
Services. In addition to computer searches, reference lists of articles on clinical significance
provided additional publications that were reviewed.
A search for previous clinical significance concept analyses provided no results.
Inclusion criteria for articles used for this concept analysis were that they were printed in
English, published in peer-reviewed journals, had a nurse as the first author, and were relevant to
the main topic of clinical significance. Approximately 83 articles were located and abstracts
read. A total of 56 articles were selected for detailed review and analyzed to obtain
understanding of the many facets of clinical significance. Of the 56 articles used for analysis,
three did not use clinical significance as a research term. The final number of articles evaluated
in this concept analysis was 53.
Clinical significance has been studied and discussed by a wide range of international
professionals. Nursing articles were co-authored by physicians, pharmacists, social workers, and
mathematician researchers living and working in the United States, Canada, England, Italy,
Germany, Denmark, Sweden, Finland, China, and Singapore. International interest is indicative
of the relevance of the concept ofCS.
Proposed Definition Based on Findings
Rodgers asserted that the identification of surrogate terms, (e.g. terms that are frequently
/1
interchanged with the concept of interest), promotes elucidating the unique and defining aspects
of the concept of interest. In the nursing literature, surrogate terms used most often as
synonymous with CS were the terms clinical importance, clinical relevance, and clinically
meaningful. While these terms were interchanged with CS, it is notable that CS was used more
exclusively when quantification, numbers or measures were applied to explain the importance of
the fmding; this observation is further described in subsequent sections. The common themes, as
they occurred in the nursing literature, are presented as the core attributes of the concept.
Synthesizing literature themes to identify core attributes of a concept provides the basis of the
"consensus of use" definition. Rodgers asserted that the "consensus of use" definition is
equivalent to a "state of the science" overview of a concept, thus it provides a baseline upon
which to guide theory development.
In accordance with the Rodgers' methodology, the first finding to be discussed is a
"consensus of use" definition of the tenn as derived from a thematic analysis of the literature.
The definition proposed is broadly inclusive of the various uses of the tenn: CS is a patient
related difference that the clinician/researcher has judgedfor nursing knowledge to guide
nursing interventions. This definition is reflective of the literature where in most cases CS was
described as resulting from changes in the status of patients which was judged (either
subjectively or quantitatively) as relevant to nursing knowledge and interventions. It is important
to note that this definition is proposed as an a posteriori observation of the existing literature;
further efforts are required to define CS more distinctly and specifically for the nursing
discipline.
Attributes of Clinical Significance
Patient related
CS is associated with three patient related themes: research directly involving patients;
patient-related policies and protocols, and medical equipment and procedures. The majority of
studies explicitly involved patients in either an inpatient or outpatient setting. Outpatient settings
were represented by individuals in rehabilitation (Riegel et. al., 2006; Baird et. al., 2010), or
nursing homes (Dowling, Graf, Hubbard & Luxenberg, 2007; Fjelltun et. aI., 2008), and those
receiving post-surgical care (Wilson & Hagadottir, 2006) or psychotherapeutic treatment
(England, Tripp-Reimer & Rubenstein, 2005) in their homes or in outpatient clinics. Inpatient
settings were represented by individuals receiving treatment for cancer (Cella et. aI., 2003; Bauer
& Romvari, 2009; Swanson & Koch, 2010) and psychiatric conditions (Bowers et. al., 2006), as
well as obstetrical care (Simpson, Parsons, Greenwood, & Wade, 2001; Callahan & Hynan,
2002; Waltman, Brewer, Rogers & May, 2004), pre and post-surgical care (Kurlowicz, 2001;
Gallagher & McKinley, 2009; ) and intensive care (Happ, Tuite, Dobbin, DiVirgilio-Thomas, &
Kitutu, 2004).
A smaller portion of the studies included indirect patient care issues - specifically
reviews of healthcare literature (Schaffer, Yoon, & Zadezensky, 2009) and evaluation of patient
care-related policies (Doughty et. aI., 2006) and protocols (Harrington, 2005). The smallest
category of studies assessed the accuracy ofmedical equipment and procedures (Gilbert, Barton
& Counsell, 2002; Farnell, Maxwell, Tan, Rhones & Philips, 2005; Zengin & Enc, 2008).
Difference: Change
CS was often asserted as having occurred when a difference was detected. The most
common differences were those designated as change. The most common changes attributed to
CS were patient responses to treatments. The usual indication of this was the improvement or
deterioration of a medical or psychiatric condition. Psychiatric treatment responses included
reduction of depression, anxiety or distress (Kurlowicz, 2001; Simpson, Parsons, Greenwood &
Wade, 2001; Callahan, & Borja, 2008; Gallagher & McKinley, 2009) and inpatient violence
(Bowers, Brennan, Flood, Lipang & Oladapo, 2006; Dowling, Graf, Hubbard, & Luxenberg,
2007; Selbaek, Kirkevold, & Engedal, 2008). Responses to treatment often involved pain
reduction and changed biopsychosocial functioning (Cella et aI, 2002; Frost et aI, 2002; Barrett,
Brown, & Mundt, 2007; Baird, Worral, Haslam, & Haslam, 2008; Dysvik, Kvaloy, Stokkeland
& Natvig, 2009).
Difference: Comparative
The second most common differences that were designated as CS in the literature were
comparative differences. Comparative differences occurred when a different condition was
detected in an individual, group of individuals, or issues related to patient care. In most studies, a
CS difference equated with detection of an abnormal psychiatric (Gallagher & McKinley, 2009),
medical (Barrett, Brown, & Mundt, 2007), functional (Dysvik et. aI., 2010) or pain condition
(Kokki et. al., 2003). In other studies the difference was related to an outdated policy (Doughty
et. aI., 2006), protocol (Harrington, 2005; Ryan-Wenger, Neal, Jones, & Lowe, 2010) or
equipment function (Gilbert, Barton, & Counsell, 2002). Adverse outcomes were also cited as
CS (Jenkins & Lindsey, 2010).
Clinician/researcher
Common themes ofjudging CS were researcher opinion (Wendler, 2003; Ward-Smith,
Korphage & Hutto, 2008; Hogan, 2009; Ryu et. al., 2010), and researcher opinion accompanied
by parameters established by expert opinion that involved: a) consensus panels (Farnell et. aI.,
2005; Doughty et al., 2006): b) government-sponsored guidelines (Skelly, Carlson, Leeman,
Soward, & Bums, 2009): c) institution-based guidelines (Gilbert, Barton, & Counsell, 2002;
Zengin & Enc, 2008): or d) recommended thresholds or cut-off scores suggested within
instruments (Dowling, Graf, Hubbard, & Luxenberg, 2007; Selbaek, Kirkevold & Engedal,
2008). A less common theme involved integrating patient judgments with researchers' judgments
to ascertain CS (Cella et. aI., 2003; Baird, Worral, Haslam, & Haslam, 2010; Dysvik et aI, 2010).
Judging CS was based upon a diversity of criteria and rationale. Often judgment was
based on a measure; the most common measure was mathematical. Mathematical measures
included applying instrument-specific numeric cut-off scores (Callahan & Borja, 2003; Dowling,
Graf, Hubbard, & Luxenberg, 2007; Selbaek, Kirkevold & Engedal, 2008; Swanson & Koch,
2010) using pre-determined, a-priori judgment by the researchers (Farnell, Maxwell, Tan,
Rhodes, & Philips, 2005), using a-posteriori judgments for moderate to large differences
(Kurlowicz, 2001; Happ, Tuite, Dobbin, DiVirgilio-Thomas, &
Kitu~
2004; Bowers, Brennan,
Flood, Lipang, & Oladapo, 2006; Schaffer, Yoon & Zadezensky, 2009; Skelly, Carlson, Leeman,
Soward, & Burns 2009), and using a-posteriori judgment of small to moderate changes or
differences (Simpson, Parsons, Greenwood, & Wade, 2001; Wendler, 2003). The second theme
involved measuring patient feedback accompanied by clinician-rating functioning (Cella et aI,
2002; Frost et aI, 2002; Barrett, Brown, & Mundt, 2007; Baird, Worral, Haslam, & Haslam,
2008; Dysvik, Kvaloy, Stokkeland & Natvig, 2009) CS judgments were not based on measures.
Instead, methods of determining CS were attributed to important occurrences including
identifying important critical occmrences (Jacobs, Apatov, & Glei 2005; Jenkins & Lindsey,
2010), discovering a deficient protocol (Ruchala, Metheny, Essenpreis, & Borcherding, 2002;
Schaffer, Yoon & Zadezensky, 2009) or detecting an important patient condition (Harrington,
2005; Hogan, 2009).
Nursing Knowledge to Guide Interventions
Generally, findings were accompanied by recommendations for application to clinical
practice. The recommendations varied in their level of specificity from general admonishments
for nurses to become more involved in policy making to prescriptive advice with specific
recommendations for practical application. Examples of recommendations to guide nursing
interventions included recommendations for medication prescribing (Schaffer, Yoon, &
Zadezensky, 2009) treating post-surgical pain (Kokki, Kankkunen- Pietila, & VehvilainenJulkunen, 2003; Wilson & Helgadottir, 2006), managing medication responses (Bauer, &
Romvari, 2009; Marrs & Zubal, 2009), and interpreting or implementing protocols, policies or
procedures (Kurlowicz, 2001; Gilbe~ Barton, & Counsell, 2002; Ruchala, Metheny, Essenpreis,
& Borcherding, 2002; Farnell, Maxwell, Tan, Rhones, & Philips, 2005; Harrington, 2005;
Doughty et al, 2006; Zengin & Enc, 2008; Hogan, 2009; Swanson & Koch, 2010).
Antecedents
The main antecedent for CS was that the research involved patient-related variables. The
broad objective of the studies was to provide relief of patient distress or deficits. Patient distress
or deficits included pain (Kokki, Kankkunen, Pietila, & Vehvilainen-Julkunen,2003; Wilson &
Helgadottir, 2006), cognitive emotional distress (Kurlowicz, 2001; Simpson, Parsons,
Greenwood, & Wade, 2001; Callahan & Borja, 2003; Wendler, 2003; Dowling, Graf, Hubbard,
& Luxenberg, 2007; Gallagher & McKinley, 2009), medical conditions (Skelly, Carlson,
Leeman, Soward, & Burns 2009; Ryu et. al., 2010), and quality of life (Frost et al, 2002;
Barrett, Brown, & Mundt, 2007; Baird, Worral, Haslam & Haslam, 2008). Topics that indirectly
affected patient distress or deficits involved assessing policies (Hogan, 2009; Schaffer, Yoon, &
Zadezensky, 2009), protocols (Harrington, 2005) or equipment (Zengin & Enc, 2008).
Consequences
The consequences of CS, based on the manner in which CS was applied in the literature,
include uncertainty about implementation of CS research fmdings. Because CS was often based
on non-standardized measures and definitions, the strength of the evidence is unclear. Strength of
evidence is defmed as the likelihood that a research finding will produce a measurable patient
outcome (Substance Abuse and Mental Health Services Association, 2006). As CS is currently
used in the research literature, it is not always associated with a measurable patient outcome.
Many of the practice recommendations were based on subjective researcher opinion as the basis
for determination of CS. Guidelines established for EBP place a low value on effecting nursing
practice changes that are based on opinion. (Melnyk & Fineout-Overholt, 2011). Additionally,
research that does not directly include patient feedback has come under scrutiny (Frost et. aI.,
2002; Lacass et. al., 2002; Puhan et. al., 2004; Haley & Frugala-Pinkham, 2006; Copay et. al.,
2008 & Carreon, Glassman, Campbell, & Anderson, 2010); placing further concerns about the
application of a CS finding to best practices. In recent years, lawsuits against nurse practitioners
have increased and one key recommendation to decrease risk included following evidence-based
practice guidelines (Edmunds & Scudder, 2009). In a worst-case-scenario, a practitioner who
does not understand or misinterprets CS in the literature could adjust nursing practice by
accepting as a priori the importance of a finding reported to be CS.
Related concepts
The term CS was often used interchangeably with related concepts to convey clinical
importance. These terms include "clinically relevant," "clinically important," and "clinically
meaningful." While CS was used interchangeably with these related concepts, it was
overwhelmingly used when quantification, numbers or measures were applied to substantiate a
research finding.
Discussion
The purpose of this concept analysis was to examine the predominant conceptual
meaning of CS using Rodgers' methodology. In an effort to bring a coherent understanding to
the concept, a definition of CS was developed based on common themes derived from its use
over the past ten years in the nursing literature. The end result was a proposed "state of the art"
understanding that is a synthesis of its myriad uses.
There are numerous problems associated with the use of CS in the nursing literature.
First, information from the analysis indicated a lack of consensus about the use of the term.
Thus, there is a multiplicity of opinions, definition and uses of CS. The absence of a consistent
definition creates problems for clinicians trying to interpret validity, reliability, relevance, and
the overall safety of findings that are labeled CS. Second, because CS was commonly based on
researcher judgment, the term is often used subjectively and findings are prone to bias in favor of
positive results. Third, the majority of the studies did not incorporate the perspective of the
patient. These problems strongly suggested that a finding labeled as CS should not necessarily
infonn decisions about nursing practice. In addition, the fmdings from this concept analysis
inferred a need to establish a common definition of CS and general guidelines for its
measurement.
In attempting to delineate benchmarks for CS in nursing, it is important to consider some
of the problems encountered by researcher/clinicians in non-nursing disciplines. Some authors
asserted that while an optimal and highly desirable solution would be to develop a single rule,
policy, or guideline for assessing CS, such a goal was unrealistic (Ogles, Lunnen, & Bonesteel,
2001; Cella et aI., 2002; Guyatt et aI. 2002; Sloan et al., 2002). Other authors assert that a single
quantitative method for determining CS was overly simplistic (Jacobson, Roberts, Berns, &
McGlinchey, 1999) or suggested that as long as a common definition of CS was agreed upon, a
variety of different measures will produce results that are reasonably equivalent (Atkins, Bedics,
McGlinchey, & Beauchaine, 2005) Another challenge was that in order to make measures ofCS
meaningful, the perspectives patients, caregivers, clinician/researchers, and society or health
policy makers needs to be identified because judgment of CS differed among stakeholders (Frost
eta aI., 2002; Hajiro, & Nishimura, 2002; Greenstein, 2003; Pittman, & Rawl, 2008). Finally,
regardless of efforts to objectively quantify CS, subjective value judgments, while (ideally)
expertly informed, pervaded clinical management and research decisions (Guyatt et aI., 2002;
Hajiro, & Nishimura, 2002; Greenstein, 2003; van Roon et al., 2009).
To resolve some of these problems, it is becoming clear that the patient perspective
should be the foundation of CS (Kazdin, 1999; Frost et al., 2002; Guyatt et al., 2002; Guyatt et
aI., 2007; Metz et aI., 2007; Copay et aI., 2009). While actively soliciting patient perspectives has
not traditionally been important for [empiric] research tradition, there has been an increased
emphasis on incorporating patient feedback about treatment into research decisions whenever
possible (Dysvik et al., 2009; Molnar et al., 2009). Indeed there are current efforts to increase
awareness of patient perspectives in clinical and drug trials (Molnar et al., 2009). An emphasis
on the patient as the foundation of data about CS is not only appropriate to help inform
clinician/researcher decisions, but data derived from patient input is considered appropriate for
other stakeholders including society and policy makers. Finally, the inclusion of the patient
perspective into the judgment of CS is congruent with the holistic foundation of nursing practice
and with EBP standards. Indeed, patient input is receiving increased emphasis within the EBP
paradigm (Fawcett, 2010; Melnyk & Fineout-Overholt, 2011). Efforts to quantify patient
response and ascribe meaning to treatment have resulted in measures of minimal and maximal
change, as well as measures of patient value judgments. These measures will be described next.
In delineating measures of patient response to treatment, there has been a need to identify
the minimal response to treatment. While there are a variety of methods to measure minimum
difference including minimum clinically important difference (MelD) and minimum clinically
important improvement (Mell), there are indications that the minimally clinically
significant/important difference (MID) will become the preferred standard to measure minimal
change (Schunemann et al., 2005; Barrett, Brown & Mundt, 2008). MID represents the smallest
difference in score in the domain of interest that patients perceive as important, either beneficial
or harmful, and which would lead the clinician to consider a change in the patient's management
(Guyatt et aI., 2007, p. 377). Variations of MID calculations may be modified to capture
intrapatient, interpatient or aggregate group differences (Cella et al., 2002). Additionally, select
statistical applications of MID, in conjunction with anchors derived from global and functional
scales or distributional methods can provide averaged MID values that are generalizable to other
studies (Copay et aI., 2008; Molnar et aI., 2009).
There are several ways to measure moderate to large or maximal change. Individual
differences are commonly measured with the Reliable Change Index, which indicates the
likelihood that the large magnitude of individual patient change is real and has not occurred by
chance (Jacobson, Follette & Revensdorf, 1984; Jacobson & Truax, 1991; Pittman & Rawl,
2008; Dysvik, Kvaloy, Stokkeland & Natvig, 2010). Measures of moderate to large group
treatment response are traditionally measured with effect size, a measure of the difference in
treatment effect between the treatment group and the comparator. However, this method has
come under scrutiny because it ignores the patient attribution of meaning (Cella et al., 2002;
Guyatt et al., 2002). Additionally, results expressed as effect size are not always generalizable to
other studies (Molnar et al., 2009). A number of measures are available, however, for
understanding the magnitude of effects that include patient perspective. Investigators may relate
changes in global score (e.g., quality of life score) to changes in scores in functional assessment,
clinical diagnoses, the patients' global ratings of the magnitude of change they have experienced,
or to the extent that patients rate themselves as feeling better or worse than other patients (Guyatt
et aI., 2007). Measures of maximum treatment response, while inclusive of pharmaceutical
treatment responses, often include responses to non-pharmaceutical treatments including
physical, psychological, rehabilitation, surgical, and multidisciplinary therapies (Cella et aI.,
2003; Dysvik et al., 2009; Metz et al., 2010).
There are other evolving aspects of CS that may be used to help the patient and clinician
make informed decisions about treatment plans that reflect patient values. Information about
likelihood of benefit to harm, lowest risk reduction, and highest benefit potential may be
measured in several ways; the most often cited are minimally clinically important difference
(MCID) and number needed to treat (NNT). MCID has been determined in various ways
including via anchor-based approaches, "between patients" score change, comparative
distributions, and social comparisons (Copay, Subach, Glassman, Polly, & Schuler, 2007). NNT
involves calculating the inverse of a percentage to determine the need to treat disorders caused or
exacerbated by a treatment effect. This is a percentage calculation that determines how many
patients need to receive a treatment in order to obtain one resulting outcome. If one of 100
individuals benefited from a treatment, then 100 individuals would have to be treated to obtain a
positive outcome, 200 for two, etc. (Melnyk & Fineout-Overholt, p. 91). These methods indicate
the plethora of evolving methods to quantify, predict, and assist patients to make informed
decisions about the CS of healthcare treatment options (Guyatt et al., 2007).
Implications for Further Research
While nursing has been relatively silent in regards to assigning a specific defmition and
measurement to CS, the medical community has been actively researching CS. In 2002, the CS
Consensus Group met to address problems and solutions related to CS. In 2007, they reconvened
and made suggestions for CS based on the ongoing medical trends and related research (Guyatt
et aI., 2007). The CS Consensus Group suggested that the definition of CS emphasize patient
response to a research or clinical treatment, and the importance and meaning the patient ascribes
to treatment effects (Guyatt et aI., 2007). Additionally, CS includes the risks and benefits of
alternative treatments and the likelihood of benefit or harm from those treatments (Guyatt et aI.,
2007). The CS Consensus Group suggested that measures to quantify CS are inclusive of the
methods described in the above section, and include the various measures of minimum and
maximum treatment effects, measures to indicate treatment outcome probabilities, and measures
of patient meaning ascribed to treatment (Guyatt et al., 2007).
In addition to emphasizing patient perspective, the CS Consensus Group suggested that
CS is most accurately appreciated by measuring multiple domains of patient experiences,
including cognitive, emotional, physical, functional, and social spheres of influence. This
conceptual model of CS assumes that patient response to treatment is experienced in multiples
aspects of their lives; thus, information of biological and physiological variables, symptoms
status, functional status; general health perceptions and overall quality of life provide the
evidentiary basis upon which informed determinations of CS are made. It is expected that
outcome measures specific to relevant medical or nursing research and/or clinical specialty are
included in the calculation of CS. Instruments to quantify the holistic measurement of CS from a
patient perspective continue to be developed; two examples of these instruments that can be
incorporated into the measurement ofCS include the Short Form-36 Health-Related Quality of
Life Questionnaire and the EuroQual-14. These measures are used widely in the US, Canada,
Australia, Japan and eight European countries (Hyak & Engle, 2010). While the patient
perspectives can make CS more holistic and meaningful, patient perspectives must also be
considered in relation to the research design. Ideally, patient perspectives to determine CS in
research will be conducted using a randomized clinical trial design to reduce the likelihood of the
Hawthorne effect.
There are compelling reasons for nursing to adapt the definition and suggested measures
recommended by the CS Consensus Group. For one, the measures are based on instruments that
have demonstrated validity and reliability. Second, the measures are quantifiable and replicable,
and are often the product of random controlled trials; thus the evidence produced is strong and
appropriate for application to best practice (Jones, 2010; Melnyk & Fineout-Overholt, 2011).
Third, the emphasis on patient perspective exemplifies the philosophical and theoretical
foundation upon which nursing is based. Finally, the patient perspective as measured in the
biopsychosocial domains reflects the core values of the holistic individual that are espoused by
nursing theorists.
The most immediate and tangible advantages to a common conceptual definition and
meaning of CS by nursing, regardless of which definition is selecte<L is less confusion and more
clarity. A consensus of meaning, a solid defmition and associated valid and reliable measures of
both quantitative and qualitative patient information may provide more fruitful and focused
research. From a clinician perspective, a solid defmition and explicit statistical nomenclature of
CS to address clinical applicability may result in improved interpretability of research findings.
User-friendly research findings may make research less burdensome for clinicians to read and
translate into best practices. Ultimately, the most enduring benefit of a common conceptual
definition and measurement for CS is the bridge it provides between research and practice, and
the facility with which it promotes the integration of research into evidence based practice.
Conclusion
Given the centrality of CS to interpreting research findings and applying them to practice,
there is a need to solidify the definition of and measurements for CS in nursing. National nursing
agencies including The National Institute for Nursing Research and Sigma Theta Tau
International should make standardizing CS a high priority for targeted funding. One method of
doing so would be to support a consensus convergence to review and select the optimal measures
of CS for nursing research. Research to increase knowledge about what constitutes measurement
and change or CS from the patient perspective is needed. Editors and peer reviewers should
encourage authors to include a discussion of CS. Discussions of CS should receive greater
emphasis in research journals. It is hoped that the preliminary fmdings from this concept
analysis presented in this article will facilitate the work of such a consensus forum.
References
Atkins, D. C., Bedics, J.D., McGlinchey, J. B. & Beauchaine, T. P. (2005). Assessing clinical
significance: Does it matter which method we use? Journal ofConsulting and Clinical
Psychology,73(5),982-989.
Baird, A., Worral, L., Haslam, C. & Haslam, R. (2008). Evaluation of a multi-disciplinary back
pain rehabilitation programme- individual and group perspectives. Quality ofLife
Research, 23(17), 357-366.
Barrett, B., Brown, R. & Mundt, M. (2008). Comparison of anchor-based and distributional
approaches in estimating important differences in common cold. Quality ofLife Research
20(17), 75-85.
Bauer, S. & Romvari, E. (2009). Treatment of chronic myeloid leukemia following Imatinib
resistance: a nursing guide to second-line treatment options. Clinical Journal ofOncology
Nursing, 1(5), 523-534.
Beaton, D. E., Boers, M. & Wells, G. A. (2002). Many faces of the minimal clinically important
Difference (MCID): a literature review and directions for future research. Current
Opinion in Rheumatology, 14, 109-114.
Bowers, L., Brennan, G., Flood, C., Lipang, M. & Oladapo, P. (2006). Preliminary outcomes ofa
trial to reduce conflict and containment on acute psychiatric wards: City Nurses. Journal
ofPsychiatric & Mental Health Nursing, 13(2), 165-172.
Brown, I. & Psaro~ A. (2007). Literature review of nursing practice in managing obesity in
primary care: developments in the UK. Journal ofClinical Nursing 17, 17-28.
Burns, N. & Grove, S. K. (2007). Understanding nursing research: Building an evidence-based
practice (4th Ed.). St. Louis: Saunders Elsevier.
Callahan, J. L. & Hynan, M. T. (2002). Identifying mothers at risk for postnatal emotional
distress: further evidence for the validity of the perinatal posttraumatic stress disorder
questionnaire. Journal ofPerinatology, 22(6), 448-454.
Callahan, J. L. & Borja, S. E. (2008). Psychological outcomes and measurement of matemal
posttraumatic stress disorder during the perinatal period. Journal ofPerinatal and
Neonatal Nursing, 22 (1), 49-59.
Carreon, L.Y., Glassman, S.D., Campbell, M. J. & Anderson, P.A. (2010). Neck disability index,
short form-36 physical component summary, and pain scales for neck and arm pain: the
minimum clinically important difference and substantial clinical benefit after cervical
spine fusion. Spine Journal, 1 0(6), 469-74.
Cella, D. F. & Fridley B. (2003) Practical guidelines for assessing the clinical significance of
health-related quality of life changes within clinical trials. Drug Information Journal
37:23-31.
Cella, D.F., Zagari, M. J., Vandoros, C., Gagnon D. D., Huntz, H. J., Nortier J.W.R. (2003).
Epoetin alfa treatment results in clinically significant improvements in quality of life
(QOL) in anemic cancer patients when referenced to the general population Journal of
Clinical Oncology 21(2), 366-373.
Center for Substance Abuse Treatment (2006). Understanding evidence-basedpractices for co-
occurring disorders. (COCE Overview Paper 6). Rockville, MD: Author. Retrieved
October 11, 2010 from
http://coce.samhsa.gov/cod_resources/PDF/Evidence-BasedPractices(OP6). pdf
England, M., Tripp-Reimer, T. & Rubenstein, L. (2005). Exploration of the psychometric
properties of an inventory of voice experiences. Archives ofPsychiatric Nursing 19, 202217.
Farnell, S., Maxwell, L., Tan, S., Rhodes, A. & Philips, B. (2005). Temperature measurement:
comparison of non-invasive methods used in adult critical care. Journal ofClinical
Nursing, 14, 632-639.
Fawcett, J., & Garity, J. (2009). Evaluating research/or evidence-based nursing practice.
Philadelphia: EA Davis Company.
Fjelltun, A.-M., Henriksen, N., Norberg, A., Gilje, F. & Normann, H.K. (2009). Functional
levels and nurse workload of elderly awaiting nursing home placement and nursing home
residents: a comparative study. Scandinavian Journal ofCaring Sciences, 23, 736-747.
Frost, M. H., Bonomi, A. E., Estwing - Ferrans, C., Wong, G.Y., Hays, R.D. & The Clinical
Significance Consensus Meeting Group (2002). Patient, clinician and population
perspectives on determining the clinical significance of quality-of-life scores. Mayo
Clinic Proceedings, 77, 488-4~94.
Gallagher, R. & McKinley, S. (2009). Anxiety, depression and perceived control in patients
having coronary artery bypass grafts. Journal ofAdvanced Nursing, 65(11), 2386-2396.
Gilbert, M., Barton, A. J. & Counsell, C. M. (2002). Comparison of oral and tympanic
temperatures in adult surgical patients (2002). Applied Nursing Research, 15(1),42-47.
Greenstein, G. (2003). Clinical versus statistical significance as they relate to the efficacy of
periodontal therapy. Journal ofthe American Dental Association, 134, 583-591.
Guyatt, G. H., Osoba, D., Wu, A. W., Wyrwich, K. W., Norman, G. R. & The Clinical
Significance Consensus Meeting (2002). Methods to explain the clinical significance of
health status measures. Mayo Clinic Proceedings, 77(4), 371-83.
Guyatt, G.H., Ferrans, C.E., Halyard, M.Y., Revicki, D.A., Symonds, T.L., Varricchio, C.G.,
Kotzeva, A., Valderas, J.M., Alonso, J.L., & The Clinical Significance Consensus
Meeting Group (2007). Exploration of the value of health-related quality-of-life
information from clinical research and into clinical practice. Mayo Clinic Proceedings,
82(10), 1229-1239.
Hajiro, T. & Nishimura, K. (2002). Minimally clinically significant difference in health status:
the thorny path of health status measures? European Respiratory Journal, 19, 390-391.
Haley, S. M. & Fragala-Pinkham, M. A. (2006). Interpreting change scores of tests and measures
used in physical therapy. Physical Therapy, 86(5), 735-43.
Happ, M. B., Tuite, P., Dobbin, K., DiVirgilio-Thomas, D. & Kitutu, J. (2004). Communication
ability, method, and content among nonspeaking nonsurviving patients treated with
mechanical ventilation in the intensive care unit. American Journal ofCritical Care, 13,
(3), 210-218.
Harrington, L. (2005). Potassium protocols: in search of evidence. Clinical Nurse Specialist 19
(3), 137-141.
Hayat, M. J. (2010). Understanding Statistical Significance. Nursing Research, 59 (3), 219-223.
Hays, R. D., Woolley, J. M. (2000). The concept of clinically meaningful difference in healthrelated quality-of-life research. How meaningful is it? Pharmacoeconomics, 419-423.
Hogan, R. (2009). Implementation of an oral care protocol and its effectiveness on oral
mucositis. Journal ofPediatric Oncology Nursing, 26, 125-134.
Jacobson, N. S., Follette, W.C., and Revenstorf, D. (1984). Psychotherapy outcome research:
Methods for reporting variability and evaluating clinical significance. Behavior Therapy,
15(4), 336-352.
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining
meaningful change in psychotherapy research. Journal ofConsulting and Clinical
Psychology, 59 (1), 12-19.
Jacobson, N.S., Roberts, L.J., Berns, S.B. & McGlinchey, J.B. (1999). Methods for defIning and
determining the clinical significance of treatment effects: description, application and
alternatives. Journal ofConsulting and Clinical Psychology, 67(3), 300-307
Jacobs, J. L., Apatov, N., Glei, M. (2007). Increasing vigilance on the medical/surgical floor to
improve patient safety. Journal ofA"tivanced Nursing 57(5), 472-481,
Jeans, M. (1992). Clinical significance of research: a growing concern. Canadian Journal of
Nursing Research, 24(1), 1-2.
Jenkins, S. D., Lindsey, P. L. (2010). Clinical nurse specialists as leaders in rapid response.
Clinical Nurse Specialist 24(1), 24-30.
Jones, K. R. (2010). Rating the level, quality, and strength of the research evidence. Journal of
Nursing Care Quality, 25, 304-312.
Kazdin, A. E. (1999). The meanings and measurement of clinical significance. Journal of
Consulting and Clinical Psychology, 67(3), 332-339.
Kvien, T. K., Heidberg, T. & Hagen, K. (2007). Minimally clinically important
improvement/difference (MCII/MCID) and patient acceptable symptom state (PASS):
What do these concepts mean? Annals ofthe Rheumatic Diseases, 66, 40-41.
Kokki, A., Kankkunen, P., Pietila A. M., & Vehvilainen-Julkunen, K. (2003). Validation of the
parents' postoperative pain measure in Finnish children aged 1-6 years. Scandinavian
Journal ofCaring Sciences, 17, 12-18.
Kurlowicz, L. H. (2001). Benefits of psychiatric consultation-liaison nurse interventions for
older hospitalized patients and their nurses. Archives ofPsychiatric Nursing 15 (2), 5361.
Lacasse, Y., Brosseau, L., Milne, S., Martin, S., Wong, E., Guyatt, G. H. & Goldstein, R. S.
(2002). Pulmonary rehabilitation for chronic obstructive pulmonary disease. Cochrane
Database Systematic Review, 3: CD003793.
Lee, S. Y. & Kimble, L. P. (2009). Impaired sleep and well-being in mothers with low-birthweight infants. Journal ofObstetric and Gynecological Nursing, 38, 676-685.
LeFort, S. M. (1993). The statistical versus clinical significance debate. Image - The Journal of
Nursing Scholarship, 25(1), 57-62.
Mantha, S., Thisted, R., Foss, J., Ellis, J. E. & Roizen, M. F. (1993). A proposal to use
confidence intervals for visual analog scale data for pain measurement to determine
clinical significance. Anesthesia Analg., 77(5): 1041-7.
Marrs, J., Zubal, B. A. (2009). Oncolocy nursing in a new era: optimizing treatment with
bevacizumab. Clinical Journal ofOncology Nursing, 13(5),564-572.
Melnyk, B. M. & Fineout-Overholt, E. (2005) Evidence-based practice in nursing & healthcare:
A guide to best practice. Philadelphia: Williams & Wilkins.
Metz, S.M., Wyrwich, K.W., Babu, A.N., Kroenke, K., Tierney, W.M. & Wolinsky, F.D. (2006).
Validity of patient-reported health-related quality of life global ratings of change using
structural equation modeling. Quality ofLife Research, 16: 1193-1202.
Molnar, F. J., Man-Son-Hing, M. & Fergusson, D. (2009). Systematic review of measures of
clinical significance employed in randomized controlled trials of drugs for dementia.
Journal ofAmerican Geriatric Society, (57), 3: 536-46.
Nisbet, A. T. & Mooney-Cotter, F. (2009). Comparison of selected sedation scales for reporting
opioid-induced sedation assessment. Pain Management Nursing, 10 (3): 154-164.
Ogles, B. M., Lunnen, K. M., and Bonesteel, K. (2001). Clinical significance: history,
application, and current practice. Clinical Psychology Review, 21(3): 421-446.
Pittman, J. & Rawl, S. M. (2008). Statistical and clinical significance are they one and the same?
Journal of Wound Ostomy Continence Nursing, 35(4): 374-376.
Puhan, M. A., Behnke, M., Devereaux, P. J., Montori, V. M., Braendi, 0., Frey, M. &
Schunemann, H. M. (2004). Measurement of agreement on health-related quality of life
changes in response to respiratory rehabilitation by patients and physicians - a
prospective study. Respiratory Medicine, 98(12): 1195-202.
Rodgers, B. L. (1989). Concepts, analysis and the development of nursing knowledge: the
evolutionary cycle. Journal ofAdvanced Nursing, 14 (4) 330-336.
Rodgers, B. L. & Cowles, K. V. (1997). A conceptual foundation for human suffering in nursing
care and research. Journal ofAdvanced Nursing, 25, 1048-1053.
Rodgers, B. L. (2000). Concept Analysis: An Evolutionary View. In B. L. Rodgers & K. A.
Knafl (Eds.). Concept Development in Nursing (2nd Ed.). Philadelphia: Saunders.
Ruchala, P. L., Metheny, N., Essenpreis, H. & Borcherding, K. (2002). Current practice in
oxytocin dilution and fluid administration for induction of labor. Journal ofGYN
Nursing, 31, 545-550.
Ryu, E., Kim, K., Cho, M.S., Kwon, I.G., Kim, H.S. & Fu, M. (2010). Symptom clusters and
quality oflife in Korean patients with hepatocelluar carcinoma. Cancer Nursing 33(1): 310.
Schaffer, S.D., Yoon, S. & Zadezensky, I. (2009). A review of smoking cessation: potentially
risky effects on prescribed medications. Journal ofClinical Nursing, 18, 1533-1540.
Schunemann, H. M., Puhan, R., Goldstein, R., Jaeschke, R. J. & Guyatt, G. H. (2005).
Measurement properties and interpretability of the chronic respiratory disease
questionnaire (CRQ). Journal ofCOPD, 2 (1),81-89.
Selbaek, G., Kirkevold,
o. & Engedal, K. (2008). Psychiatric and behavioural symptoms and the
use of psychotropic medication in special care units and regular units in Norwegian
nursing homes. Scandinavian Journal ofCaring Science, 22,568-573.
Shih-Yu, L. & Kimble, L. P. (2009). Impaired sleep and well-being in mothers with low-birthweight infants. Journal ofObstetric, Gynecologic & Neonatal Nursing 38(6),676-685.
Simpson, M., Parsons, M., Greenwood, J. & Wade, K. (2001). Raspberry leaf in pregnancy: its
Copay, A. G., Glassman, S, D., Polly, D.W. & Schuler, T. C. (2007). Understanding the
minimum clinically important difference: a review of concepts and methods.
Spine Journal, 7, 541-546.
Copay, A. G., Glassman, S. D., Subach, B. R., Berven, S., Schuler, T. C. & Carreon, L. Y.
(2008). Minimum clinically important difference in lumbar spine surgery patients: a
choice of methods using the Oswestry Disability Index, Medical Outcomes Study
Questionnaire Short Form 36, and pain scales. Spine Journal, 8(6),968-74.
DiCenso, A. (2001). Clinically useful measures of the effects of treatment. Evidence-Based
Nursing, 4, 36-39.
Doughty, D., Ramundo, J., Bonham, P., Beitz, J., Erwin-Toth, P., Anderson, R. & Rolstad B.S.
Sloan, J. A., Cella, D., Frost, M. H., Guyatt, G. H., Sprangers, M. A. G., Symonds, T. & The
Clinical Significance Consensus Meeting Group (2002). Assessing the clinical
significance in measuring oncology patient quality of life: introduction to the symposium,
content overview, and definition of terms. Mayo Clinic Proceedings, 77: 367-370.
Sloan, J. A., Vargas-Chanes, D., Kamath, C. C., Sargent, D. I., Novotny, P. .I., Atherton, P.,
Allmer, C., Fridley, B. L., Frost M. H. & Loprinzi, C.L. (2003) Detecting worms, ducks
and elephants: A simple approach for defining clinically relevant effects in quality-of-life
measures. Journal ofCancer Integrative Medicine 1(1):41-47.
Sprangers, M. A., Moinpour, C. M., Moynihan, T. I., Patrick, D. L., Revicki, D. A. & The
Clinical Significance Consensus Meeting Group (2002). Assessing meaningful change in
quality of life over time: a users' guide for clinicians. Mayo Clinic Proceedings, 77
(6):561-71.
Substance Abuse and Mental Health Services Administration, and Center for Mental Health
Services (2006). Understanding evidence-based practices for co-occurring disorders.
(COCE Overview Paper 6). Rockville, MD: Author. Retrieved September 29,2010 from
http://coce.samhsa.gov/cod_resources/PDF/Evidence-BasedPractices (OP6).pdf
Symonds, T., Berzon, R., Marquis, P., Rummans, T. A. & The Clinical Significance Consensus
Meeting Group (2002). The clinical significance of quality-of-life results: practical
considerations for specific audiences. Mayo Clinic Proceedings, 77(6): 572-580.
Swanson, J. & Koch, L. (2010). The role of the oncology nurse navigator in distress management
of adult inpatients with cancer: a retrospective study. Oncology Nursing Forum, 3 7(1):
69-76.
Tubach, F., Ravaud, P., Beaton, D., Boers, M., BOIrlbardier, C., Felson, D. T., Van der Heijde,
D., Wells, G. & Dougados, M. (2007). Minimal clinically important ilnprovement and
patient acceptable symptom state for subjective outcome measures in rheumatic
disorders. Journal ofRheumatology, 34, (5), 1188-1193
Van Roon, E. N., van den Bernt, P. M., Jansen, T. L., Houtman, N. M., van de Laar, M. A. &
Brouwers, J. R. (2009). An evidence-based assessment of the clinical significance of
drug-drug interactions between disease-modifying antirheumatic drugs and nonantirheumatic drugs according to rheunatologists and pharmacists. Clinical Therapetuics,
31(8): 1734-1746.
Waltman, P. A., Brewer, J. M., Rogers, B. P. & May, W. L. (2004). Building evidence for
practice: a pilot study of newborn bulb suctioning at birth. Journal ofMidwifery &
Women's Health 49(1): 32-38.
Ward-Smith, P., Korphage, R. M. & Hutto, C. J. (2008). Cost comparison between two pediatric
palliative care programs. Nursing Economics, 26(3), 2008.
Wendler, M.C. (2003). Effects of Tellington touch in healthy adults awaiting venipuncture.
Research in Nursing & Health, 26, 40-52.
Wilson, M. E. & Helgadottir, H. L. (2006). Patterns of pain and analgesic use in 3-to-7-year-old
children after tonsillectomy. Pain Management Nursing, 7(4): 159-166.
Zengin, N. & Enc, N. (2007). Comparison of two blood sampling methods in anticoagulation
therapy: venipuncture and peripheral venous catheter. Journal ofClinical Nursing, 17(3):
386-393.