Download Title of Presentation Date - American Society For Experimental

Including the Patient’s Voice
February 20, 2014
David Cella, PhD
Professor and Chair
Department of Medical Social Sciences
Northwestern University Feinberg School of Medicine
American Society for Experimental NeuroTherapeutics | 16th Annual Meeting
Disclosure
National Institutes of
Health
Grantee
American Society for Experimental NeuroTherapeutics | 16th Annual Meeting
Learning Objectives
1) Understand the options that ensure the content
validity and patient-centeredness of questions that
get included into patient-reported clinical trial data
2) Be familiar with the family of NIH-sponsored
person-centered outcome measures
a) PROMIS
b) Neuro-QoL
c) NIH Toolbox
American Society for Experimental NeuroTherapeutics | 16th Annual Meeting
Distinguishing PROs from Other Assessments
Patient-reported outcome (PRO)
ClinRO
Standardized assessment based on
patient self-report, completed alone
(Examples: SF-36; PROMIS; Neuro-QoL;
Toolbox Emotion; MSQLI; PDQ-39)
Assessment based on physician
interpretation of patient status
(Examples: Ham-D, Karnofsky; EDSS;
Rankin/mRS)
Performance-based assessment
Clinical assessment
Objective test measuring patient
performance
(Examples: reaction time, 6MWT, FEV1;
Toolbox Motor, Sensory and Cognition)
Based on clinical examination and
diagnostic evaluation
(Examples: disease stage; histology;
pathology; EEG; radiography)
5
5
“Performance Measure”
“Self report”
Both provide information, r = 0.40
Content Validity: Content Consensus through
Qualitative Research
Concept Elicitation
(Focus Groups & Interviews)
Generated
Words &
Phrases
Instrument Evaluation
(Cognitive Interviews)
Consensus
Wording
Items & Response
Options
Interpretation
& Meaning
Structure
Recall, Instructions
Format
Developer
Expertise
7
ISPOR Task Force on Content Validity of Existing Instruments. Value in Health. 2009. (Figure 2)
What is Saturation?

Defined as “Data adequacy”

Operationalized as “collecting data until
no new information is obtained”
Glaser and Strauss (1967)

“Boredom that occurred when
investigators had ‘heard it all’”
– Margaret Mead
Rule of Thumb for Number of Interviews
•
Bertaux (1981) - 15 is the smallest acceptable sample size in qualitative
research.
•
Kuzel (1992) - 6-8 interviews for a homogeneous sample and 12-20
data sources “when looking for disconfirming evidence or trying to
achieve maximum variation.”
•
Morse (1994) - at least 6 participants for phenomenological studies,
and 35 participants for ethnographies, grounded theory studies, and
ethno-science studies
•
Creswell (1998) - 5-25 interviews for a phenomenological
study and 20-30 for a grounded theory study.
•
Bernard (2000) - most ethnographic studies are based on 36
interviews
•
Guest et al (2006) – conducted 60 interviews and achieved saturation
by 12
• None of these sources except Guest provided
any evidence for their recommendations
Rule of Thumb for Number of Focus Groups
• Vaughan et al. (1996) – 3-4 focus
groups per any defined group
Construction of a PRO Instrument:
an Iterative Process*
Establish
- target population
- scope of assessment
- concepts to include
- available resources
Revise and finalize instrument
Pilot test candidate instrument
Develop items based on
- literature review
- focus groups
- in-depth interviews
Item reduction
Evaluate psychometric properties
Pretest in sample of target
population
*Note: The sequence of steps may vary
Evaluate cross-cultural equivalence
Planning for PRO Assessment in a Clinical Trial
 All endpoints require thoughtful development and proper
validation.
 Review medical and health services research literature
 Review various PRO bibliographies/websites/guides
 Select instruments that best match relevant domains
– Sensitivity of measurement
– Coverage of domains
 Align endpoints chosen with clinical trial population and
endpoint model
12
Bloating
bothers me!
I’m worried
and concerned
Heartburn
disturbs my sleep
I hate my life
I can’t bend
over or exercise
I can’t eat and
drink whatever
I like
PROMIS Cooperative Group
2004-2014 Highlights
50 protocols aligned with evolving PROMIS standards
50,000 people have contributed data
2,000 in qualitative research
45,000 in quantitative research
15,000 children
3,000 adult proxies for children
30,000 adults on their own behalf
…including more than 5,000 Spanish-speaking adults and children
14
PROMIS Domain Framework
Symptoms
Physical Health
Function
Affect
Self-Reported
Health
Mental Health
Behavior
Cognition
Social
Health
15
Relationships
Function
Cycle of Development and Validation
Qualitative
Research
and Item
Writing
16
Testing
Item
Bank
Analysis
Interpretation
Refining
General Population
Clinical Samples
The PROMIS Metric
T Score
Mean = 50
SD = 10
Referenced to the US General
Population
17
PROMIS Basic Tools
Derived from Item Banks
Computerized Adaptive Testing (CAT)
 Dynamic testing averaging 6 items per domain
Fixed Length Forms
 By individual domain (8-10 items)
 By health profile (-29, -43, -57)
Global Health Index
18
0
50
100
Physical Functioning Item Bank
Item
1
Item
2
Item
3
Item
4
Item
5
Item
6
Item
7
Item
8
Are you able to get in and out of bed?
Are you able to stand without losing your balance for 1 minute?
Are you able to walk from one room to another?
Are you able to walk a block on flat ground?
Are you able to run or jog for two miles?
Are you able to run five miles?
Item
9
Item
n
Neurological Outcomes Beyond Mobility
• It’s important to be able to walk to the store.
• It’s also important to remember why you did.
• Also important to manage finances, maintain a
household, plan social events, sexual function
QoL, etc.)
• Are the important things measured by the usual
outcome tools?
20
Neuro-QoL: PRO Measures for
Neurology Research and Practice
•
•
•
•
NINDS-funded initiative
Use in chronic neurologic conditions
Can be administered as CATs or short forms
Most scores can be linked to PROMIS
www.NeuroQol.org
Neuro-QoL Domain Framework
Physical
Health
Self-Reported
Health
Mental
Health
Social
Health
Symptoms
Function
Emotional Health
Cognitive Health
Delirium as a model for a factor affecting specific domains
Mobility hardly different, applied cognition very different.
AC - Executive AC - General
Function
Concerns
Fine
Motor
Physical
Function
Mobility
Satisfaction
with social
roles and
activity
Fatigue
55
45
50
50
55
45.5
44.2
42.5
40
41.7
39.7
60
39.3
37.8
35
65
35.1
33.6
20
67.0
31.3
30
25
25
46.5
45
70
75
25.5
Ever delirious
Never delirious
Delirium retrieved from EDW and charted bedside assessments
(~2400 assessments in ~100 pts)
80
T-Score (50=normal)
T-Score (50 ± 10 normal)
50.2
What it is:




26
Brief unified set of
measures
Use in large longitudinal,
epidemiological, clinical
trials
Measures the same
constructs over lifespan
Where possible, objective
measures over self-report
What it is not:


Not a diagnostic tool
Not conceptualized to
substitute for the indepth assessment of a
domain or sub-domain
Toolbox Domains
Cognition
Emotion
Motor
Sensory
How do I select an instrument?
• There is no simple formula or algorithm
– Research is needed
– Consultation can help
•
•
•
•
34
Identify participant age group (pediatric vs adult)
Select language(s)
Select relevant domains
Select mode of administration (paper, web,
offline computer, interview)
Instrument Selection
• What disease/condition?
– Expected range?
• Why are you capturing PROs?
– Need for label claim?
– Clinically meaningful change?
– Desire to cover specific content?
• Assess need for brevity versus precision
– Expected change?
• Assess available reliability and clinical
validation data
35
Impact on Clinical Care and Practice
• There are several patient-centered outcome tools
ready for clinical and research use.
• Many are freely available
• All were developed with patient-centric or person-centric
methods.
• Domain content is abundant
• Precise, valid measurement is possible without
burdening patients
• Further work can enhance clinical utility

Assessment Center video tutorials
Thank You
•
•
•
•
www.nihpromis.org
www.neuroqol.org
www.nihtoolbox.org
www.assessmentcenter.net