Download Treating depression in nursing homes

Survey
yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project

Document related concepts

Critical Psychiatry Network wikipedia , lookup

Psychiatric and mental health nursing wikipedia , lookup

History of psychiatric institutions wikipedia , lookup

Mental health professional wikipedia , lookup

Moral treatment wikipedia , lookup

Emergency psychiatry wikipedia , lookup

Postpartum depression wikipedia , lookup

Major depressive disorder wikipedia , lookup

Controversy surrounding psychiatry wikipedia , lookup

Biology of depression wikipedia , lookup

Behavioral theories of depression wikipedia , lookup

Evolutionary approaches to depression wikipedia , lookup

Depression in childhood and adolescence wikipedia , lookup

Transcript
CLINICAL PRACTICE
Treating Depression in Nursing Homes: Practice Guidelines in the Real World
Deborah Wagenaar, DO; Christopher C. Colenda, MD, MPH; Michelle Kreft, BS; Julie Sawade, BS;
Joseph Gardiner, PhD; Elena Poverejan, PhD
Depression in nursing home residents is a common phenomenon, though there is a wide range in the severity of
this disorder as experienced by elderly adults in the United
States. Treating older patients for depression is costly in
both human and financial terms. The authors review guidelines and recommendations for the diagnosis of depression and medical treatment of depressed elderly adults
from the 1993 Clinical Practice Guidelines for Depression
in Primary Care by the US Agency for Health Care Policy
and Research (now the Agency for Healthcare Research
and Quality), making note also of similar guidelines provided by the American Medical Directors Association in
1996 and those provided by the American Psychiatric Association in 2000.
In November 1999, the authors used a Delphi survey
to gather data from prescreened panelists (N = 10) to have
the panel review and clarify the importance and feasibility
of each “A” rated item in the US Agency for Health Care
Policy and Research guidelines. This research allows health
care providers to evaluate the importance and feasibility of
items related to the delivery of mental health services for
the elderly in nursing homes.
D
epression is currently found in one third of nursing
home residents in the United States.1,2 The severity of
depressive disorders ranges from minor to major and can
occur at any time during a resident’s placement in a nursing
home. Evidence suggests that newly admitted nursing home
residents have greater functional impairment, use 7% additional staff time, and are 1.5 times more likely to die within 12
months of admission.3,4 Older adults with depression also
incur more outpatient visits, medication costs, and laboratory charges.4
Depression is underrecognized and undertreated. In a
large nursing home study, Rovner et al2 found that only 14%
Dr Wagenaar is an assistant professor in the Department of Psychiatry at
Michigan State University (MSU) in East Lansing. Dr Colenda is dean of the College of Medicine at Texas A&M University in College Station. Also at MSU, Ms
Kreft and Ms Sawade are students in the College of Osteopathic Medicine.
Dr Gardiner is a professor at MSU’s Department of Epidemiology, where Dr
Poverejan was a graduate student at the time of this study.
Address correspondence to Deborah Wagenaar, DO, Assistant Professor,
Department of Psychiatry, Michigan State University, A231 E Fee Hall, East
Lansing, MI 48824-1316.
E-mail: [email protected]
of patients identified by psychiatric evaluation as having clinical depression were documented as having this disorder in
their medical records. Heston et al5 suggest that only 20% of
patients who have depression receive antidepressant treatment.
Practice guidelines have been proposed as a solution to
improve the medical management of depression in this population. A number of public and private organizations have
generated their own sets of guidelines and recommendations.
In 1993, the US Agency for Health Care Policy and Research
(AHCPR), now the Agency for Healthcare Research and
Quality, developed diagnostic and treatment guidelines for
depression that rate the quality of scientific evidence and provide diagnostic and treatment algorithms.6,7 In 1996, the American Medical Directors Association (AMDA)8 updated and
adapted the AHCPR guidelines for use in the nursing home
setting. In addition, a workgroup for the American Psychiatric
Association (APA)9 created practice guidelines in 2000 for
treating patients with major depressive disorder, offering
treatment-specific recommendations for use by general psychiatrists.
As most clinicians know, translating practice guidelines into the clinical practice setting successfully can be a
challenge. Psychiatrists face financial, attitudinal, ethical,
and medical barriers posed by the practical realities of clinical practice in the long-term care setting. Despite these barriers, practice guideline adherence literature suggests that
patients have better clinical outcomes with greater adherence
to such guidelines. Nursing homes are one place in which
the implementation of clinical pathways may guide depression care.
Are depression practice guidelines really useful in a
nursing home practice? To assess their feasibility and importance, we abstracted the “A” rated elements of the AHCPR
Clinical Practice Guidelines for Depression in Primary Care
into a set of declarative statements. The 1993 AHCPR guidelines were chosen because they were developed by national
expert consensus and include categorized recommendations
based on the strength of scientific evidence. Although many
other practice guidelines have been developed since, such as
those by the AMDA or the APA, we chose to review the
AHCPR guidelines because of their extensive dissemination
and strong support in the literature, and because of their comprehensive nature.
Wagenaar et al • Clinical Practice
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/931998/ on 08/02/2017
JAOA • Vol 103 • No 10 • October 2003 • 465
CLINICAL PRACTICE
Each AHCPR guideline declarative statement used in
our Delphi survey represented one guideline recommendation. We used only guideline statements that were rated “A,”
those given the highest degree of clinical and scientific evidence. For this project, we asked national and local opinion
leaders in the areas of geriatric psychiatry, geriatric psychology, and primary care medicine to evaluate the importance
and the feasibility of each declarative statement in the treatment
and care of older adults with depression.
We selected the review panel randomly from a list of
members of the American Association of Geriatric Psychiatry and the AMDA. Panelists were contacted and the purpose
of the panel was explained. As primary care physicians often
treat depression in both nursing home and ambulatory settings, it was important to include these specialists as well.
All of the panelists had clinical geriatric experience and an
interest in mental health issues.
Methods
Delphi Process
The Delphi technique is a nominal group research methodology designed to achieve group consensus using anonymity,
iteration, controlled feedback, and statistical group response.10
We chose to use the Delphi method because it allows experts
to compare and contrast their responses with those of their
colleagues. The Delphi method seeks expert agreement by
asking panelists to rate each issue using a 7-point Likert scale
and statistically comparing individual scores with a group
mean score for each declarative statement. Because group consensus was a key element in looking at the feasibility of each
statement, we found this method to be the most appropriate for
conducting our research.
Experts completed the first survey ratings, and we calculated means for each individual item. In a second identical
survey, individuals were asked to reassess the scores they originally provided for any items where their individual scores
varied by less than 1 numerical point above or below the group
mean. In the second survey, panelists were thus allowed to
change their individual scores after considering the group
mean if they desired. Importance and feasibility scores from the
second survey process were then used as consensus items.
Overall trends to move toward the mean were noted with
individual responses.11-13
Subjects
The Michigan State University Institutional Review Board
approved the project. Each subject signed a consent form before
participating in the project. The Michigan State University
Human Subjects Committee approved the wording used on
these consent forms.
Eleven subjects were recruited for the Delphi panel. Each
subject was a nationally or regionally recognized expert in the
field of geriatric psychiatry, geriatric psychology, or primary care
medicine. Subjects were selected for their expertise in late life
depression, as determined by academic productivity, including
grants, publications, and national reputation. All volunteer
subjects had clinical and research experience in long-term care
settings, as evidenced by their publication and/or academic
track records. Three panelists were osteopathic physicians,
nine were allopathic physicians, and two panelists held PhD
degrees. One or more panelists represented each region of the
United States.
Survey Process
The authors contacted each prospective panelist individually to
explain the panel and the purpose of the study. Each of the 11
panelists volunteered to participate in the Delphi survey questionnaire process. The survey was anonymous and results were
confidentially secured. Those subjects who agreed to participate but did not return surveys within 1 month were contacted
again and encouraged to return the survey materials.
Results were analyzed and a second Delphi survey was
returned to each participant. One panelist elected to drop out of
the second Delphi round.
Instrument
This study was performed using a survey questionnaire
format and consisted of 101 key items abstracted as important
depression diagnostic and treatment elements from the 1993
AHCPR Clinical Practice Guidelines for Depression in Primary
Care. The AHCPR guidelines were chosen because they too
were developed using group consensus methodology and
rated recommendations based on the strength of scientific
evidence.6,7
Additional input into the creation of this survey instrument was obtained by informal discussions among investigators
and additional faculty members who participated in the survey
pilot at Michigan State University in East Lansing. The survey
was initially piloted by five allopathic physicians. We made
further revisions to the design of this instrument based on the
feedback received.
The survey initially asked panelists to report the following
demographic data: specialty, age, years in practice, status as medical director in a nursing home, degree(s), and percentage of practice in a nursing home. Four sections followed, focused on the diagnosis and treatment of major depression and minor depression.
Survey instructions for each section asked panelists to rate on a
7-point Likert scale (1, least; 7, most) each item for importance and
feasibility in the assessment of patients for depression in a nursing
home setting. Diagnostic items included clinical assessment
parameters, symptoms, screening tool use, and routine laboratory
testing. For example, panelists were asked to use the Likert scales
provided to rate the statement “If major depression is suspected
in a nursing home patient, the clinical assessment should include
and document suicidal ideation/attempts” twice, one time for the
importance of this evaluation when making a diagnosis of depression and a second time for the feasibility of this evaluation in the
nursing home setting.
466 • JAOA • Vol 103 • No 10 • October 2003
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/931998/ on 08/02/2017
Wagenaar et al • Clinical Practice
CLINICAL PRACTICE
Therefore, the importance and feasibility of several assessment measures were evaluated by each of the participating
panelists. In terms of assessment instruments, the Minimum
Data Set (MDS) was chosen because it is ubiquitous in nursing
home practice. The MDS is the current mechanism used to
identify depression in nursing home residents. The Geriatric
Depression Scale (GDS), the Center for Epidemiologic Studies
Depression Scale (CES-D), and the Cornell Scale for Depression
in Dementia (CSDD) are all standardized screening instruments in the medical literature relating to depression. We chose
to use these instruments because each represents a valid and reliable assessment measure of depression in patients.
Treatment parameters included prescribed antidepressant therapy, psychotherapy, electroconvulsive therapy (ECT),
and criteria for mental health referral. Each panelist was further
asked to rate separately the importance and feasibility of cognitive-behavioral and interpersonal psychotherapy in the
nursing home setting. An importance and feasibility score was
determined for each of these treatment modalities. Finally,
panelists were asked to rank their preferred choices when prescribing antidepressants in the medical treatment of depression
in nursing home patients.
Data Analysis
Data were analyzed using Statistical Analysis System software
(SAS Institute Inc, Cary, NC). This study assessed a number of continuous variables for feasibility and importance using a 7-point
Likert scale (1, least; 7, most). Summary statistics were reported for
the first and second surveys using means for continuous measures. In the second survey, respondents received the survey with
their previous scores marked, as well as the group mean marked
for each item. By and large, panelists chose to move their initial
response toward the group mean for most items. Data are presented as frequency measures in Tables 1 and 2.
Results
Demographics
Ten participants completed the entire Delphi survey process.
Six were psychiatrists, two were internists, one was a psychologist, and one was a primary care physician. Panelists’
ages ranged from 41 to 64 years, with 60% aged 51 years or
younger. Half of the survey participants were in practice 15
years or fewer. Only two panelists were nursing home medical
directors with 10% of their total practice time spent in nursing
homes. All had added qualifications in geriatrics for their
respective fields.
Importance of AHCPR Guidelines
Each item was rated for clinical importance and is reported in
Table 1. For both major and minor depression, over 70% of the
group endorsed suicidal ideation and/or attempts, anxiety
symptoms, dementia symptoms, psychotic symptoms, and
medical comorbidity as important information to obtain during
an evaluation. Seventy percent of the group believed depressed
mood, an increase or decrease in psychomotor energy, lack of
interest or pleasure, lack of reactivity to pleasant stimuli, early
morning awakening, weight change, and anhedonia were
important symptoms to assess. Only 20% of the group believed
that diurnal mood variation (worsening depression in the
Table 1
Survey Panel: Items as Rated for Clinical Importance
During Evaluation
Depression, %
Major
Minor
Item
Suicidal ideation and/or attempts
Psychotic symptoms
Medical comorbidity
Depressed mood
Anxiety symptoms
Lack of interest or pleasure
Anhedonia
Lack of reactivity to pleasant stimuli
Increase or decrease in
psychomotor energy
Early awakening (morning)
Weight changes
Worsening depression (morning)
100
100
100
100
90
90
90
80
100
100
100
90
100
100
90
60
70
70
70
20
100
40
80
30
Minimum Data Set*
Geriatric Depression Scale†
Center for Epidemiologic Studies
Depression Scale‡
Cornell Scale for Depression in Dementia§
10
90
10
30
10
20
10
10
Electrolytes and blood chemistry
Thyroid-stimulating hormone
B12
Complete blood count
Folate
Serology
T3
90
90
70
70
40
0
0
100
90
70
90
20
0
0
*Source: US Department of Health and Human Services, Centers for
Medicare & Medicaid Services. Minimum Data Set (MDS) 3.0. Available at:
http://www.cms.gov/quality/mds30/. Accessed October 2, 2003.
†Source: Yesavage JA. Use of self-rating depression scale in the elderly.
In: Poon LW, ed. Handbook for Clinical Memory Assessment of Older
Adults. Washington, DC: American Psychological Association; 1986:213217. Available at: http://www.stanford.edu/~yesavage/GDS.html. Accessed
September 17, 2003.
‡Source: Radloff LS. Mood Disorders Measures: Center for Epidemiologic
Studies Depression Scale [CES-D]. 1977. In: Task Force for the Handbook of
Psychiatric Measures. Handbook of Psychiatric Measures. Washington, DC:
American Psychiatric Association; 2000:523-526. Available at:
http://www.sph.unc.edu/iprc/longscan/Measures%20Manual/
Ages5to11/m7.pdf. Accessed September 17, 2003.
§Source: Alexopoulos GS, Abrams RC, Young RC, Shamoian CA. Cornell
Scale for Depression in Dementia. Biol Psychiatry. 1988;23:271-284.
Wagenaar et al • Clinical Practice
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/931998/ on 08/02/2017
JAOA • Vol 103 • No 10 • October 2003 • 467
CLINICAL PRACTICE
morning hours) was important when assessing patients for
depression. Ratings were similar for both major and minor
depression.
The GDS was the only assessment tool the panel overwhelmingly endorsed (90%) as being important to the diagnosis
of depression. The other instruments, such as the MDS, CES-D,
and the CSDD, were not rated as being highly important for
diagnosis purposes. Laboratory assessment tools varied in importance with the complete blood count, electrolytes, B12 level, and
thyroid-stimulating hormone being endorsed by over 70% of
the panel as being important to obtain before giving a diagnosis
of depression.
All subjects believed that the use of prescribed antidepressants in combination with psychotherapy was important when
treating elderly patients for depression in a nursing home. The
panelists believed that combined treatment with psychotherapy
and antidepressant medications was important for the nursing
home resident. Antidepressant treatments of 4 to 6 weeks’ duration, with reevaluation at 6 weeks, and ongoing treatment for
4 to 9 months for positive responders was endorsed as being
important by at least 90% of the panel. Psychotherapies were rated
poorly as only 20% of the group endorsed cognitive-behavioral
or interpersonal modes of therapy as important. In contrast, 90%
of the group believed that ECT was important for use in treating
nursing home patients with depression.
Panelists ranked their choices for antidepressant class in
this population. In order of those most likely to be used, the
ranking was as follows: selective serotonin reuptake inhibitors,
atypical antidepressants, mixed antidepressants, secondary
amine antidepressants, tertiary amine antidepressants, psychostimulants, monoamine oxidase inhibitors, and herbal preparations.
Feasibility of AHCPR Guidelines
Feasibility was assessed for each item on the survey and is
reported in Table 2. Similar feasibility scores were seen for both
major and minor depression items. The entire group believed that
the assessment of suicidal ideation and/or attempts, anxiety
symptoms, comorbid medical illnesses, psychotic symptoms,
and dementia symptoms were feasible when assessing patents
for depression in a nursing home. Similarly, over 80% of the
group endorsed the symptoms of depressed mood, psychomotor
energy changes, lack of interest and reactivity, diurnal mood
variation, and weight change as feasible to obtain. Anhedonia,
weight change, and diurnal mood variation were endorsed as feasible by only 30% of the panelists.
Although rated as not important, 90% of the group believed
the MDS was feasible to obtain. Only 10% rated the GDS, CESD, or CSDD as feasible in the nursing home setting. The panel
believed that all laboratory tests were feasible as well.
The contrast between importance and feasibility is highlighted with treatment. Although antidepressant and psychotherapy combination therapy was rated as important, none
of the group believed this combination (or psychotherapy alone)
was feasible. Only 10% of the group endorsed psychotherapy
(including cognitive-behavioral and interpersonal) as feasible.
Antidepressant treatment without psychotherapy was endorsed
as feasible by 80% of panel participants.
With respect to antidepressant treatment, feasibility ratings parallel those for importance. Antidepressant treatment
for at least 4 to 6 weeks, reassessment of partial response at
Table 2
Survey Panel: Items as Rated for Feasibility of Detection
During Evaluation
Depression, %
Item
Major Minor
Suicidal ideation and/or attempts
Psychotic symptoms
Medical comorbidity
Depressed mood
Anxiety symptoms
Lack of interest or pleasure
Increase or decrease in psychomotor energy
Worsening depression (morning)
Weight changes
Early awakening (morning)
Anhedonia
Minimum Data Set*
Geriatric Depression Scale†
Cornell Scale for Depression in Dementia‡
Center for Epidemiologic Studies
Depression Scale§
Electrolytes and blood chemistry
Thyroid-stimulating hormone
B12
Complete blood count
Folate
Serology
T3
100
100
100
100
90
90
80
80
60
60
30
100
100
100
90
100
100
90
100
90
90
90
90
60
20
90
20
40
10
40
100
100
100
100
100
100
100
100
100
100
100
100
100
100
*Source: US Department of Health and Human Services, Centers for
Medicare & Medicaid Services. Minimum Data Set (MDS) 3.0. Available at:
http://www.cms.gov/quality/mds30/. Accessed October 2, 2003.
†Source: Yesavage JA. Use of self-rating depression scale in the elderly.
In: Poon LW, ed. Handbook for Clinical Memory Assessment of Older
Adults. Washington, DC: American Psychological Association; 1986:213217. Available at: http://www.stanford.edu/~yesavage/GDS.html. Accessed
September 17, 2003.
‡Source: Alexopoulos GS, Abrams RC, Young RC, Shamoian CA. Cornell
Scale for Depression in Dementia. Biol Psychiatry. 1988;23:271-284.
§Source: Radloff LS. Mood Disorders Measures: Center for Epidemiologic
Studies Depression Scale [CES-D]. 1977. In: Task Force for the Handbook of
Psychiatric Measures. Handbook of Psychiatric Measures. Washington, DC:
American Psychiatric Association; 2000:523-526. Available at:
http://www.sph.unc.edu/iprc/longscan/Measures%20Manual/
Ages5to11/m7.pdf. Accessed September 17, 2003.
468 • JAOA • Vol 103 • No 10 • October 2003
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/931998/ on 08/02/2017
Wagenaar et al • Clinical Practice
CLINICAL PRACTICE
6 weeks, and ongoing treatment for 4 to 9 months were viewed
as being feasible by at least 90% of panel participants. Only
weekly visits to assess medication response met with poor feasibility ratings.
Although ECT had high importance ratings, the feasibility
of this treatment was rated as poor. Eighty percent of the panel
believed that ECT was not feasible to arrange within a nursing
home setting.
Comment
This study demonstrates the fact that although multiple types
of clinical practice guidelines exist for the medical treatment
and management of depression, consensus is still possible on
key importance and feasibility items. In this study, such items
included the expression and clinical evaluation of depressive
symptoms and diagnostic testing and screening measures. These
items were clearly endorsed as both important and feasible for
good quality depression care in the end stages of life.
However, this study also disclosed a number of discrepancies between importance and feasibility in terms of combined
modes of therapy, ECT, and psychotherapy. Experts endorsed
the importance of these treatment modalities but did not see
them as being feasible within a nursing home setting. The reasons for these discrepancies are unknown and suggest possible
topics for future research. While combination modes of therapy
are endorsed as important, clinical access and financial, attitudinal, and patient-selection criteria barriers may impair feasibility.
In this study, we were unable to address the heuristics behind
our findings but are convinced further research should explore
why important mental health services are not feasible in a
nursing home setting.
In contrast, some assessment parameters were considered
unimportant but feasible. The MDS is one such example.
Although it is a mandatory assessment tool for all nursing home
residents, the expert panel did not find it valuable. The MDS
mood subscale poorly correlates with other well-proven assessment measures.14 In contrast, the GDS was viewed as important
but not feasible. It is not a mandatory assessment measure in
nursing home practice. However, incorporating the GDS into the
MDS process would be a move toward using a valid and reliable screening measure for the diagnosis of depression.
Interestingly, ECT is endorsed as an important treatment
modality by 90% of panelists treating major depression, but the
same percentage rated it as not being feasible in this clinical setting. Electroconvulsive therapy is an effective and safe treatment
for older adults being treated for major depression and should
be accessible to all patients. The reasons for this lack of feasibility
may include transportation, safety concerns, ageism, or lack of
knowledge about the efficacy of the treatment. Further research
exploring these barriers and the usefulness of ECT in longterm care settings should be explored.
Questions remain, however. How do these results benchmark to actual implementation of practice guidelines in longterm care settings? For other conditions, practice guidelines
have been developed, but others have found that moving their
application into a real-world setting can be challenging when
the clinician is faced with financial, attitudinal, medical, and
complex psychosocial barriers. Our data, from a panel of
national experts, suggest that feasibility issues (ie, actually
finding ways to get the job done) may be a significant barrier
to providing treatment to nursing home residents who have
depression. As a result, the formal guidelines and recommendations issued by public and private organizations may not
fit real-world patients, leaving clinicians confused about how
to apply such guidelines effectively.
References
1. Parmelee PA, Katz IR, Lawton MP. Depression among institutionalized
aged: assessment and prevalence estimation. J Gerontol. 1989;44:M22-M29.
2. Rovner BW, German PS, Brant LJ, Clark R, Burton L, Folstein MF. Depression
and mortality in nursing homes [published correction appears in JAMA
1991;265:2672]. JAMA. 1991;265:993-996.
3. Fries BE, Mehr DR, Schneider D, Foley WJ, Burke R. Mental dysfunction
and resource use in nursing homes. Med Care. 1993;31:898-920.
4. Callahan CM, Hui SL, Nienaber NA, Musick BS, Tierney WM. Longitudinal
study of depression and health services use among elderly primary care
patients. J Am Geriatr Soc. 1994;42:833-838.
5. Heston LL, Garrard J, Makris L, Kane RL, Cooper S, Dunham T, Zelterman
D. Inadequate treatment of depressed nursing home elderly. J Am Geriatr Soc.
1992;40:1117-1122.
6. Depression Guideline Panel. Depression in Primary Care: Volume 1. Detection and Diagnosis. Clinical Practice Guideline, No 5. Rockville, Md: US Department of Health and Human Services, Public Health Service, Agency for Health
Care Policy and Research; 1993. AHCPR publication 93-0550. Available at:
http://www.mentalhealth.com/bookah/p44-d1.html. Accessed September 17,
2003.
7. Depression Guideline Panel. Depression in Primary Care: Volume 2. Treatment of Major Depression. Clinical Practice Guideline, No 5. Rockville, Md: US
Department of Health and Human Services, Public Health Service, Agency for
Health Care Policy and Research; 1993. AHCPR publication 93-0551. Available
at: http://www.mentalhealth.com/bookah/p44-d2.html. Accessed September
17, 2003.
8. American Medical Directors Association. Clinical Practice Guideline: Depression. Columbia, Md: American Medical Directors Association, 1996.
9. Karasu TB, Gelenberg A, Gelenberg A, Wang P. Work Group on Major
Depressive Disorder. Practice Guideline for the Treatment of Patients With Major
Depression. Washington DC: American Psychiatric Association, 2000. Available at: http://www.psych.org/clin_res/Depression2e.book.cfm. Accessed
September 17, 2003.
10. Jones J, Hunter D. Consensus methods for medical and health services
research [review]. BMJ. 1995;311:376-380. Available at: http://bmj.bmjjournals.com/cgi/content/full/311/7001/376. Accessed September 17, 2003.
11. Ashton CM, Kuykendall DH, Johnson ML, Wun CC, Wray NP, Carr MJ, et
al. A method of developing and weighting explicit process of care criteria
for quality assessment. Med Care. 1994;32:755-770.
12. Keeney S, Hasson F, McKenna HP. A critical review of the Delphi technique
as a research methodology for nursing. Int J Nurs Stud. 2001;38:195-200.
13. Pearson SD, Margolis CZ, Davis S, Schreier LK, Sokol HN, Gottlieb LK. Is consensus reproducible? A study of an algorithmic guidelines development process. Med Care. 1995;33:643-660.
14. Frederiksen K, Tariot P, De Jonghe E. Minimum Data Set Plus (MDS+)
scores compared with scores from five rating scales. J Am Geriatr Soc.
1996;44:305-309.
Wagenaar et al • Clinical Practice
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/931998/ on 08/02/2017
JAOA • Vol 103 • No 10 • October 2003 • 469