Download Development and cross-sectional validation of the Childhood

Andrew H. Liu, MD,a Robert Zeiger, MD,b Christine Sorkness, PharmD,c Todd Mahr,
MD,d Nancy Ostrom, MD,e Somali Burgess, PhD,f Jacqueline Carranza Rosenzweig,
PharmD, MS,g and Ranjani Manjunath, MSPHg Denver, Colo, San Diego, Calif,
Madison and LaCrosse, Wis, Boston, Mass, and Research Triangle Park, NC
Background: For children younger than 12 years old with
asthma, there are several quality-of-life instruments, clinical
diaries, and questionnaires assessing symptoms; however, a
validated tool for assessing asthma control is currently lacking.
Objective: To develop and validate the Childhood Asthma
Control Test (C-ACT), a self-administered tool for identifying
children aged 4-11 years whose asthma is inadequately
controlled.
Methods: A 21-item questionnaire was administered to 343
patients with asthma and their caregivers, randomly assigning
75% (n 5 257) for development and cross-sectional validation
of the tool and 25% (n 5 86) to a confirmatory sample.
Stepwise logistic regression was used to reduce the 21 items to
those best able to discriminate control as defined by the
specialist’s rating of asthma control.
From aNational Jewish Medical and Research Center and the University of
Colorado School of Medicine, Denver; bKaiser Permanente, San Diego;
c
the University of Wisconsin School of Pharmacy, Madison; dthe Gundersen Lutheran Clinic, LaCrosse; ethe Allergy and Asthma Medical
Group and Research Center, San Diego; fMapi Values, Boston; and
g
GlaxoSmithKline, Research Triangle Park.
Supported by GlaxoSmithKline.
Disclosure of potential conflict of interest: A. H. Liu has consulting arrangements with GlaxoSmithKline, AstraZeneca, and Schering-Plough; has
received grant support from GlaxoSmithKline, Microlife, Ross, and
Novartis; and is on the speakers’ bureau for GlaxoSmithKline, Merck,
Schering-Plough, and AstraZeneca. R. Zeiger has consulting arrangements
with Aerocrine, AstraZeneca, Dynavax, Genentech, GlaxoSmithKline,
Merck, and Novartis; has received grant support from AstraZeneca,
GlaxoSmithKline, Merck, Sanofi-Aventis, and Teva Pharmaceuticals; and
has lectured for AstraZeneca. C. Sorkness has consulting arrangements
with AstraZeneca and GlaxoSmithKline; has received grant support from
GlaxoSmithKline; and is on the speakers’ bureaus for GlaxoSmithKline.
T. Mahr has received grant support from GlaxoSmithKline, Alcon,
AstraZeneca, and Novartis and is on the speakers’ bureau for Alcon,
AstraZeneca, GlaxoSmithKline, Merck, Schering-Plough, Novartis,
Genentech, Verus, and Sanofi-Aventis. N. Ostrom has received grant support from Alcon, Allux, Altana, AstraZeneca, Clay-Park Labs, Critical
Therapeutics, Genentech, GlaxoSmithKline, Hoffman-LaRoche, Medicinova, MedPointe, Merck, Novartis, Pharmaxis, Rigel, Sanofi-Aventis,
Schering-Plough, and Wyeth; has consulting arrangements with Aperon Biosystems, AstraZeneca, Genentech, and Verus Pharmaceuticals; and is on the
speakers’ bureau for AstraZeneca, Genentech, Novartis, GlaxoSmithKline,
IVAX, KOS, Pfizer, Sanofi-Aventis, and Schering-Plough. S. Burgess has
received grant support from GlaxoSmithKline. J. C. Rosenzweig and
R. Manjunath are employed by GlaxoSmithKline.
Received for publication September 8, 2006; revised November 20, 2006;
accepted for publication December 13, 2006.
Available online March 15, 2007.
Reprint requests: Andrew H. Liu, MD, National Jewish Medical and Research
Center, 1400 Jackson Street (K1023), Denver, CO 80206. E-mail: liua@
njc.org.
0091-6749/$32.00
Ó 2007 American Academy of Allergy, Asthma & Immunology
doi:10.1016/j.jaci.2006.12.662
Results: Seven items were selected from regression analyses of
the development sample to comprise the C-ACT. The scores of
each item were summed for a total score (0-27), with lower
scores indicating poorer control. Summed scores discriminated
between groups of patients differing in the specialists’ rating of
asthma control (F 5 36.89; P < .0001), the need for change in
patients’ therapy (F 5 20.07; P < .0001), and % predicted FEV1
(F 5 2.66; P 5 .0494). A score of 19 indicated inadequately
controlled asthma (specificity 74%, sensitivity 68%). These
analyses were confirmed in the confirmatory sample.
Conclusion: The C-ACT is a validated tool to assess asthma
control and identify children with inadequately controlled asthma.
Clinical implications: The C-ACT can be valuable in clinical
practice and research based on its validation, ease of use, input
from the child and caregiver, and alignment with asthma
guidelines. (J Allergy Clin Immunol 2007;119:817-25.)
Key words: Asthma, pediatric, control, health outcomes, healthrelated quality of life, symptom assessment, questionnaire, tool
The National Asthma Education and Prevention
Program (NAEPP)1 and the the Global Initiative for Asthma
(GINA)2 define the goals of asthma control as minimal or no
symptoms during the day or night, full physical activity including exertion, prevention of exacerbations, maintenance
of (near) normal pulmonary function, decreased use of
rescue b2-agonist medication, and minimal or no adverse
effects from medications. Reliable assessment of asthma
control is essential to manage asthma effectively and to
initiate or change pharmacotherapy.3,4
There are fundamental challenges when assessing
asthma control in children, including selecting meaningful
measures to assess asthma symptoms and determining the
most reliable source of the information (parent/caregiver,
child, and/or health care provider). Developmental issues
are believed to influence the accuracy of a young child’s
symptom reporting.5 For example, some children may
have difficulty associating time with asthma events and
may be reluctant to acknowledge their asthma symptoms,
because they do not recognize abnormal symptoms or they
want to be perceived as ‘‘normal.’’6,7 As a result, caregivers are often asked to assess their child’s symptoms.
However, numerous studies have demonstrated poor
correlations between the symptom reports of children and
those of their parents.8-12 Indeed, some studies suggest
that child symptom reporting should not be discounted
or ignored. In a study by Lara et al10 assessing the validity
of exercise-related symptom reporting by children with
asthma compared with their parents, child-reported
817
Asthma diagnosis and
treatment
Development and cross-sectional validation
of the Childhood Asthma Control Test
818 Liu et al
Asthma diagnosis and
treatment
Abbreviations used
AUC: Area under the curve
C-ACT: Childhood Asthma Control Test
FVC: Forced vital capacity
GINA: Global Initiative for Asthma
NAEPP: National Asthma Education and Prevention
Program
NPV: Negative predictive value
PAQLQ: Pediatric Asthma Quality of Life Questionnaire
PACQLQ: Pediatric Asthma Caregiver’s Quality of Life
Questionnaire
PPV: Positive predictive value
ROC: Receiver operating characteristic
coughing and wheezing correlated with FEV1 and observed symptoms due to exercise; in contrast, parentreported symptoms did not correlate. Guyatt et al11
observed in younger children (age <11 years) that their
symptom reports correlated strongly with changes in quality-of-life measures, although parents’ ratings of asthma
symptoms showed moderate correlations with FEV1 and
asthma control but not quality-of-life measures. They concluded that, in younger children, clinicians are likely to
obtain important and complementary information from
children and their parents.
Owing to these challenges, there is a need for a simple
yet reliable measurement tool to assess asthma control in
children to help them achieve the goals of asthma care.4
Several patient-completed instruments have been developed and validated to assess asthma control13,14 in adults
with asthma. For younger (ie, age <12 years) children with
asthma, there are several health-related quality-of-life
instruments, clinical diaries, and questionnaires assessing symptoms directed towards children or their caregivers6,15-18; however, a standardized, self-administered,
and validated tool for assessing asthma control in younger
children is currently lacking.
The Childhood Asthma Control Test (C-ACT) was
developed to assess asthma control in children 4-11 years
of age with asthma for use in the clinic and at home. The
C-ACT was designed to be self-administered, to incorporate input from parent and child, to capture the multidimensional nature of asthma control, and to demonstrate
good predictive properties for assessing asthma control.
This article details the development and cross-sectional
validation of the C-ACT.
METHODS
Development of the working questionnaire
The initial conceptual framework of domains for ‘‘asthma
control’’ was developed from the national NAEPP1 guidelines, the
international GINA2 guidelines, and working group input (10 childhood asthma and allergy specialists). To support this framework
and determine further concepts, the literature was reviewed for existing generic and asthma-specific questionnaires as well as relevant
literature for developing age-appropriate questions for younger children with asthma.13,19-26
J ALLERGY CLIN IMMUNOL
APRIL 2007
The questions, including recall periods and response formats,
were created and refined based on information gathered from 4
rounds of interviews with 22 children with asthma and 14 caregivers
who were white (45%), African American (32%), or of other ethnicity
(23%). Patients interviewed were also of different levels of caregiverassessed asthma severity, with 36% considered to have ‘‘very mild or
mild’’ asthma, 32% considered to have ‘‘moderate’’ asthma, and 28%
considered to have ‘‘severe or very severe’’ asthma. Differing recall
periods and Likert scales were chosen for the caregiver and child
questions based on the literature on age-appropriate question formats
for younger children5 and feedback from child and caregiver interviews. The child questions ask about the present, without reference
to recall period, because interviewed children 4-6 years of age had
difficulty recalling beyond 1 day.
We also learned in the patient interviews that younger children
often require more guidance in responding to the child-reported
questions, perhaps to the extent of the parent/caregiver reading the
questions aloud to the child. Acknowledging that there are developmental differences in children’s abilities to understand and respond to
items rated on Likert scales, a scale using a child’s face was developed
to facilitate comprehension by younger children. Several types of
faces were tested, including a generic face, a boy’s face, and a girl’s
face; the young boy’s face was preferred by the majority of children.
It was also found that younger children tended to use more
extreme responses and had difficulties understanding a neutral state,
ie, ‘‘neither happy or unhappy.’’ Consequently, a 4-point Likert scale
was chosen (ie, ‘‘very bad,’’ ‘‘bad,’’ ‘‘good,’’ or ‘‘very good’’) to
address this issue. For the caregiver questions, a standard recall
period of 4 weeks was selected.
Based on these interviews, a working questionnaire was generated
containing a total of 21 questions (8 child-completed and 13
caregiver-completed questions) (see Fig E1 in this article’s Online
Repository at www.jacionline.org). Questions related to asthma
symptoms and impact on everyday functioning and quality of life
were directed to the children and caregivers; questions pertaining to
asthma exacerbations, health care resource utilization (including
medication), and missed school or daycare were directed only to
the caregivers.
Cross-sectional study design
A 3-month, cross-sectional, nonrandomized study of 343 children
and their parents/caregivers was conducted to identify the optimal
child and caregiver questions to comprise the final C-ACT questionnaire from the 21-item working questionnaire. Children 4-11 years of
age were identified and enrolled from 9 specialist asthma clinics at
their usual care visit, including those who may have experienced or
were experiencing an exacerbation. Patients were required to have
a diagnosis of asthma without other respiratory conditions, have
experienced symptomatic improvement to a short-acting b-agonist
bronchodilator, and/or had a history of documented reversible airway
disease. Institutional Review Board approval was granted by local
committees, and signed informed consent/assent was obtained from
children and parents/caregivers.
Study forms, the working questionnaire, and supplemental health
status questionnaires (Pediatric Asthma Quality of Life Questionnaire
[PAQLQ]22; Pediatric Asthma Caregiver’s Quality of Life
Questionnaire [PACQLQ]27) were completed by children and their
caregivers during a routine previously scheduled physician office
visit. Caregivers were instructed to help the child understand and to
read the question if needed, but to allow the child to select the
responses himself. Specialists were blinded to patient questionnaire
responses. Following completion of questionnaires by parents/caregivers and their children, pediatric asthma and allergy specialists assessed the patient’s asthma severity, control, and medication regimen
by history, physical exam, and spirometry (FEV1 and FEV1/FVC) in
a routine manner. Spirometry was performed pre- and postbronchodilator in study participants who were older and able to comply with
American Thoracic Society standards28 (n 5 270).
Data analyses
Data from the cross-sectional study were analyzed to develop and
validate the final C-ACT based on the 21-item working questionnaire.
The total sample (n 5 343) was randomly divided into a 75%
development sample (n 5 257) and a 25% confirmatory sample (n 5
86). The development sample was used for item selection, evaluation of
the questionnaire’s psychometric properties (ie, construct, concurrent,
clinical, and predictive validities), and determination of the scoring
algorithm. Analyses of the confirmatory sample were used to verify the
75% sample results. All analyses were performed using SAS Version
9.1.3 for Windows or Multitrait/Multi-item Analysis Program–
Revised for Windows, Version 1.0 (SAS Institute, Cary, NC).
Item selection. Stepwise logistic regression was performed to
reduce the 21-item working questionnaire to the set of items that were
most predictive in determining asthma control. A P value of .10 was
selected a priori as the criterion for item retention. The criterion
measure was the specialist’s rating of asthma control, dichotomized
as ‘‘controlled’’ (including ‘‘well controlled’’ and ‘‘completely controlled’’) versus ‘‘uncontrolled’’ (including ‘‘not controlled at all,’’
‘‘poorly controlled,’’ and ‘‘somewhat controlled’’). Formal conservative tests were then used to evaluate the goodness of fit of the final
model consisting of the most predictive child and parent/caregiver
items. The overall performance of the model was evaluated using
area under the curve (AUC) from the receiver operating characteristic
(ROC) curves. The Hosmer-Lemeshow goodness-of-fit test was used
to determine how well the data fit a logistic model.29
Validation. Demographic and clinical characteristics were examined using descriptive statistics. Internal consistency reliability was
also evaluated using Cronbach a. Concurrent validity of the instrument was calculated using the Pearson correlations between the score
on the final C-ACT instrument and the scores on the PAQLQ and
PACQLQ. The clinical validity of the C-ACT was evaluated (ie, the
extent to which the C-ACT can discriminate among clinically diverse
groups) by percentage predicted FEV1 and FEV1/FVC ratio (<60%,
60% to 79%, 80% to 99%, and 100%), specialist’s assessment of control and change in therapy (step up, no change, and step down).
Analysis of variance methods were used to evaluate the ability of
the C-ACT score to discriminate between these groups derived
from the clinical measures.
Known-group validity and ROC analyses were performed to
discriminate ‘‘uncontrolled’’ versus ‘‘controlled’’ patients based on
the C-ACT score cut points. The odds ratios, sensitivity, specificity,
positive predictive value (PPV), negative predictive value (NPV),
percentage correctly classified, and AUC (or c-statistic) were estimated for each scoring option drawn at different cut points. The 25%
confirmatory sample was used to verify the results. A P value of .05
was selected a priori and used for all the analyses of the reduced
instrument. All tests of significance were 2-sided.
RESULTS
The sample characteristics of the children (n 5 343) are
presented in Table I. The mean age was 8.1 years; 61%
were male; and 74.34% were in ‘‘good to very good’’
health based on parent/caregiver report. Approximately
63% of caregivers had a college or graduate degree and
about 63% were employed either part or full time.
Specialists characterized 62% of the participants with
mild, 36% with moderate, and 2% with severe asthma.
Liu et al 819
Specialists concluded that 70.5% of their patients had
asthma that was well to completely controlled (ie, ‘‘controlled’’). No change in therapy was necessary for 69%,
and a step up in therapy was recommended for 21% of
patients.
Item selection
Seven of 21 items (4 child-reported and 3 caregiverreported) were selected based on their ability to predict
asthma control (see Fig E2 in this article’s Online
Repository at www.jacionline.org). Maximum likelihood
estimates based on logistic regression are provided for
the 7 most predictive items (Table II). The 7-item model
showed good statistical fit, with an AUC of 0.798 and percentage concordant and percentage tied results of 79.5 and
0.5, respectively. The Hosmer-Lemeshow goodness-of-fit
test indicated sufficient fit (0.5322) to the 7-item logistic
model. The child-completed items use a 4-point response
scale and the parent/caregiver-completed items use a 6point response scale, with lower scores indicating poorer
control. The items were summed to obtain a total score
for the C-ACT, with a potential range in score from 0-27.
Construct and concurrent validity
In the development sample (n 5 257), the observed
C-ACT score ranged from 6 to 27. No floor or ceiling
effects were observed. Multitrait analysis showed that the
item convergent validity of the C-ACT score was good
(Pearson item-scale correlation corrected for overlap;
range, 0.41-0.68). The Cronbach coefficient alpha for
the C-ACT score was 0.79 and satisfied the minimum
recommended level of 0.70. The concurrent correlations
between the C-ACT score and the PAQLQ and PACQLQ
domains were moderate to strong (0.47 and 0.68, respectively). The results of the construct validity analyses were
similar in the subgroups of children 4 to 8 years of age and
in children 9 to 11 years of age (data not shown).
Clinical validity
In the development sample, mean C-ACT scores were
lowest in children with pre-bronchodilator FEV1 lower
than 60% of predicted (16.33) and FEV1/FVC ratios lower
than 60% (17.20) (Table III). Conversely, mean C-ACT
scores were highest in children with prebronchodilator
FEV1 higher than 80% of predicted (20.51) and FEV1/
FVC ratios higher than 80% (20.42), without differentiation seen between the 2 highest categories.
For specialist’s rating of asthma control, 29.5% were
considered inadequately controlled in the development
sample and 28% in the confirmatory sample. The mean
scores were significantly different between groups (P <
.0001), ordered in an expected manner with the highest
mean score (22.27) found among children who where
rated as ‘‘completely controlled’’ by their specialists and
the lowest mean score (13.62) found among children
whose asthma was rated as ‘‘poorly controlled’’ (Table
III). None of the children had their asthma rated as
‘‘not controlled at all’’ by their specialist, so 4 categories
were included for this analysis: ‘‘poorly controlled,’’
Asthma diagnosis and
treatment
J ALLERGY CLIN IMMUNOL
VOLUME 119, NUMBER 4
820 Liu et al
J ALLERGY CLIN IMMUNOL
APRIL 2007
TABLE I. Sample characteristics in the cross-sectional validation study (n 5 343)
Asthma diagnosis and
treatment
Demographic characteristics
Age (y), mean (SD)
Female, n (%)
Ethnicity, n (%)*
Afro-Caribbean/African American
Asian/Indian
Hispanic/Latino/Spanish American
North American/European/white
Native American
Other
Caregiver education level, n (%)
Some high school
High school diploma or GED
Some college
College degree (2-year or 4-year)
Graduate degree
Other
Caregiver work status, n (%)
Working full- or part-time (paid employment)
Full-time homemaker
Unemployed
Student
Retired
Other
Clinical characteristics
General health, n (%)
Excellent
Very good
Good
Fair
Poor
Specialist-assessed asthma severity, n (%)*
Mild
Moderate
Severe
% predicted FEV1, mean (SD)*
Albuterol use (no. of puffs), mean (SD)
Daytime*
Nighttime*
Asthma symptom score, n (%)*
0: No symptoms
1: Symptoms for 1 short period during the past 4 wk
2: Symptoms for 2 or more short periods during the past 4 wk
3: Symptoms for most of the time which did not affect normal activities
4: Symptoms for most of the time with did affect normal activities
5: Symptoms so severe that work or normal activities could not be performed
Specialist global assessment of control,* n (%)
Not controlled at all
Poorly controlled
Somewhat controlled
Well controlled
Completely controlled
Specialist-assessed change in therapy,* n (%)
Step down
No change
Step up
8.1 (2.39)
132 (38.60)
38
15
20
233
2
34
(11.11)
(4.39)
(5.85)
(68.13)
(0.58)
(9.94)
13
45
51
137
80
16
(3.80)
(13.16)
(14.91)
(40.06)
(23.39)
(4.67)
216
93
12
6
1
15
(62.97)
(27.11)
(3.50)
(1.75)
(0.29)
(4.37)
59
129
126
26
3
(17.20)
(37.61)
(36.73)
(7.58)
(0.87)
211
123
7
94.04
(61.88)
(36.07)
(2.05)
(19.81)
1.20 (2.27)
0.86 (3.25)
108
116
59
28
23
4
(31.49)
(33.82)
(17.20)
(8.16)
(6.70)
(1.17)
1
24
76
178
63
(0.29)
(7.02)
(22.22)
(52.05)
(18.42)
34 (10.06)
234 (69.23)
70 (20.71)
*Missing data: ethnicity (n 5 1), specialist-assessed asthma severity (n 5 2), % predicted FEV1 (n 5 69), daytime albuterol use (n 5 20), nighttime albuterol
use (n 5 18), asthma symptom score (n 5 5), specialist global assessment of control (n 5 1) and change in therapy (n 5 5).
Liu et al 821
J ALLERGY CLIN IMMUNOL
VOLUME 119, NUMBER 4
TABLE II. Item selection: analysis of maximum likelihood estimates
Estimate
Child-completed
How is your asthma today?
How much of a problem is your asthma when you run, exercise, or play sports?
Do you cough because of your asthma?
Do you wake up during the night because of your asthma?
Parent/caregiver-completed
During the last 4 weeks, how many days did your child have any daytime asthma symptoms?
During the last 4 weeks, how many days did your child wheeze during the day because of asthma?
During the last 4 weeks, how many days did your child wake up during the night because of asthma?
SE
0.43
0.19
0.24
0.28
0.24
0.21
0.26
0.24
20.39
20.33
20.13
0.18
0.19
0.17
TABLE III. Comparison of mean C-ACT scores across groups differing in asthma control (75% development
sample, n 5 257)
Clinical Parameter
% Predicted FEV1 (n 5 212)
FEV1/FVC (n 5 212)
Specialist-assessed change in
child’s therapy (n 5 250)
Specialist assessment of asthma
control (n 5 254)
Category
n
Mean score
SD
F*
<60%
60% to 79%
80% to 99%
100%
<60%
60% to 79%
80% to 99%
100%
Step up
No change
Step down
Poorly controlled
Somewhat controlled
Well controlled
Completely controlled
9
36
87
78
5
47
151
7
26
167
57
21
54
134
45
16.33
19.47
20.51
20.19
17.20
19.08
20.42
20.14
17.00
20.99
20.08
13.62
17.89
21.14
22.27
6.59
4.60
4.36
4.11
5.26
5.11
4.22
3.98
4.59
3.87
4.41
4.90
3.98
3.46
3.28
2.66
P value*
.0494
1.76
.1554
20.07
<.0001
36.89
<.0001
*Because the assumptions of normality were met, the P value has been calculated with parametric 1-way analysis of variance test for differences.
Prebronchodilator.
‘‘somewhat controlled,’’ ‘‘well controlled,’’ and ‘‘completely controlled.’’ For specialist-assessed change in therapy, differences in the C-ACT scores were significant (P <
.0001), with a lower mean score (17.00) found in the group
whose treatment was recommended for ‘‘step up’’ compared with those whose treatment was recommended
for ‘‘no change’’ (20.99) or ‘‘step down’’ (20.08). These
results were verified in the 25% confirmatory sample
(Table IV).
C-ACT score cut points
Cut-point scores of 18 to 22 demonstrated the highest
area under the ROC curve (range 0.672 to 0.713), with
sensitivity (ie, patients classified by C-ACT as uncontrolled for those who are uncontrolled by specialist
assessment) ranging from 63% to 92% and specificity
(ie, patients classified by C-ACT as controlled for those
who are controlled by specialist assessment) ranging from
42% to 80% (Fig 1 and Table V).
The selection of a cut-point score was based on the
following factors: (1) balance between sensitivity and
specificity; (2) high sensitivity for adequate accuracy in
identifying those patients who are uncontrolled; (3) high
AUC and percentage correctly classified; and (4) clinical
validity. A cut point of 19 demonstrates a sensitivity of
68% and a specificity of 74%. At scores of 19,
approximately 52% of those classified by the C-ACT
as uncontrolled were actually uncontrolled (PPV 52%),
and 84.6% of those classified by the C-ACT as controlled were actually controlled (NPV 85%)—such that
72% of the children were correctly classified. These
findings were strongly verified in the 25% confirmatory
sample with, at scores of 19, PPV 70%, NPV 88%, and
83% correctly classified (Table VI). The clinical validity
results further support a cut point of 19 or less in the
C-ACT as indicative of inadequate asthma control. The
C-ACT mean scores are less than 19 for patients characterized as somewhat (18) or poorly (14) controlled,
requiring step up in therapy (17), and with <60% FEV1
(16) (Fig 2).
DISCUSSION
The cross-sectional validation of the C-ACT demonstrates the reliability and predictive properties of the tool
Asthma diagnosis and
treatment
Items
822 Liu et al
J ALLERGY CLIN IMMUNOL
APRIL 2007
TABLE IV. Confirmation of results relevant to the comparison of mean C-ACT scores across groups differing
in asthma control (25% confirmatory sample, n 5 86)
Grouping variable
Asthma diagnosis and
treatment
% Predicted FEV1 (n 5 58)
FEV1/FVC (n 5 58)
Specialist-assessed change in
child’s therapy (n 5 83)
Specialist assessment of asthma
control (n 5 83)
Group
n
Mean score
SD
x 2*
<60%
60% to 79%
80% to 99%
<80%
80% to 99%
100%
Step up
No change
Step down
Poorly controlled
Somewhat controlled
Well controlled
Completely controlled
9
28
21
14
41
3
10
65
8
4
19
43
17
16.33
21.21
22.00
18.14
21.58
21.33
17.10
20.77
21.87
17.00
17.42
20.95
23.29
4.03
3.15
2.85
4.33
3.05
4.16
3.57
3.86
1.25
3.46
3.45
3.49
2.28
14.07
.0009
8.07
.0176
11.36
.0034
31.29
<.0001
P value*
*Because of violation of assumptions of normality, the P value has been calculated with nonparametric Kruskal-Wallis test for differences.
Prebronchodilator.
FIG 1. C-ACT receiver operating characteristic curves. Sensitivity versus 1 2 specificity plotted for the
development (75% of study cohort) and validation (25% of cohort) samples.
to assess asthma control in children 4-11 years of age. The
tool shows good ROC characteristics relative to the
specialists’ ratings of asthma control, as well as good
performance of C-ACT scores in their ability to discriminate based on various levels of clinical variables, including spirometry and specialist’s recommendation of a
change in therapy. In addition, correlation of the C-ACT
with PAQLQ22 and PACQLQ27 assessments indicates
that the C-ACT captures asthma-related quality-of-life
parameters. Findings were further verified in the 25%
confirmatory sample, indicating good reproducibility and
consistency of the results.
The specialist’s assessment of asthma control as the
criterion measure was considered to be the most complete
and clinically meaningful outcome for item selection in
the absence of a ‘‘gold standard’’ by which asthma control
Liu et al 823
J ALLERGY CLIN IMMUNOL
VOLUME 119, NUMBER 4
Cut-point score
18
19
20
21
22
Sensitivity
Specificity
PPV (%)
NPV (%)
Correctly classified (%)
AUC
0.63
0.68
0.73
0.83
0.92
0.80
0.74
0.63
0.59
0.42
56.6
52.0
45.5
45.6
40.1
83.6
84.6
85.0
89.0
92.7
74.8
72.0
66.1
65.7
57.1
0.713
0.709
0.682
0.707
0.672
TABLE VI. Summary of the performance of the C-ACT score for each scoring option at different cut points in
predicting specialist’s assessment of uncontrolled asthma (25% confirmatory sample, n 5 86)
Cut-point score
18
19
20
21
22
Sensitivity
Specificity
PPV (%)
NPV (%)
Correctly classified (%)
AUC
0.56
0.70
0.91
0.96
0.96
0.90
0.88
0.72
0.50
0.42
68.4
69.6
55.3
42.3
38.6
84.4
88.3
95.6
96.8
96.2
80.7
83.1
77.1
62.7
56.6
0.733
0.789
0.815
0.728
0.687
can be assessed. This criterion measure has been used to
develop other well accepted questionnaire tools that are
often used to assess asthma control in adults and older
children.13,14 To reduce the subjective nature of the criterion measure, specialists used clinical evaluation (spirometry, physical exam) with patient history (eg, symptoms,
exacerbations, quality of life, and activity limitations) to
assess a patient’s asthma control and were blinded to
patient responses on the questionnaires.
Lung function measures were not incorporated as an
item for potential inclusion in the C-ACT, because this
would limit its use to clinical settings and to older children
capable of performing spirometry. For a validated asthma
control assessment instrument in adults, the omission of
FEV1 made minimal difference in the validity, reliability,
and responsiveness of the instrument.30 Instead, in the
present study, spirometry was used as a component in
the specialist’s assessment of asthma control.
Remarkably, the 4 child-reported questions regarding
daytime and exercise-induced asthma symptoms, nighttime awakenings due to asthma, and self-report of asthma
(ie, ‘‘how is your asthma today?’’), as well as the caregiver
questions on daytime and nighttime symptoms, are consistent with NAEPP1 and GINA2 assessment guidelines.
The statistical predictive properties found during validation yielded both the child and caregiver reports of nighttime awakenings. Because of concern that these items may
be redundant, statistical correlations were conducted and
shown to be imperfect (r 5 2.50). This weak correlation
is consistent with other studies revealing divergence between child-reported and parent-reported daytime, nighttime, and exercise-induced symptoms,10,12,31 thereby
highlighting the importance of incorporating both child
and caregiver perspectives when assessing asthma control.
This study captured a relatively mild and controlled
sample of children with asthma, which may limit the
generalizability of these findings to other cohorts. We did
not aim to recruit children who were ethnically diverse or
from inner-city settings that may have a high burden of
moderate to severe disease and higher resource utilization
(eg, emergency department [ED] visits and hospitalizations). However, we conducted clinical validity analyses
among all groups of children in our sample (poorly
controlled to completely controlled). The clinical validity
findings demonstrate that the tool is able to discriminate
among various levels of control, even in those children
characterized by clinicians as being poorly controlled or
somewhat controlled. Future studies can refine our understanding of the optimal application of this tool in specific
populations of interest (eg, inner-city children with
asthma).
In addition, the C-ACT does not include a direct
measure of exacerbations. Although the initial pool of
items in the working questionnaire (see Fig E1 in this article’s Online Repository at www.jacionline.org) included
multiple caregiver questions assessing asthma exacerbations (ie, asthma attack in past 4 weeks, ED visit in past
year, unscheduled visit in past year, and hospitalization
in past year), these items were not selected, based on logistic regression. However, caregivers reported relatively
high frequencies of attacks (37%), ED visits (29%), unscheduled visits (58%), and hospitalizations (5%). An explanation for this finding may be that exacerbations were
captured or embedded within other symptom-related questions underlying the specialist’s determination of asthma
control. Because the inherent variability of asthma in children is such that some may appear to have good control in
the clinic, and subsequently develop a severe exacerbation, it is prudent to recognize that this cross-sectional
study is unable to determine if the C-ACT can predict
exacerbation risk. Ongoing longitudinal studies using the
C-ACT may determine if this tool is helpful in this regard.
Asthma diagnosis and
treatment
TABLE V. Summary of the performance of the C-ACT score for each scoring option at different cut points in
predicting specialist’s assessment of uncontrolled asthma (75% development sample, n 5 257)
824 Liu et al
J ALLERGY CLIN IMMUNOL
APRIL 2007
Asthma diagnosis and
treatment
FIG 2. C-ACT cut-point score of 19 compared with clinical measures.
The utility of a patient-based assessment tool in clinical
practice is suggested to be dependent on the easy interpretability of the score and the minimization of false
negatives.30 In selecting a score cut point, there is always a
tradeoff of sensitivity vs. specificity: moving to a higher
cutoff value improves sensitivity (the percentage of truly
uncontrolled patients who are correctly classified as such),
but this also results in a larger number of false positives
which reduces specificity. For the C-ACT, selection of a
cut point was determined based on the idea of using the
tool as an indicator for uncontrolled asthma to encourage
communication between parents/caregivers, children, and
clinicians while taking into account the tradeoffs associated with false-positive misclassifications versus falsenegative misclassifications.
Overall, to minimize the chance of false negatives, a
cut point of 19 was selected on a continuous scale
to indicate uncontrolled asthma. Two other studies have
suggested that an intermediate area of control may
exist.32,33 For the Asthma Control Test, scores >19 were
also associated with ‘‘well controlled’’ or ‘‘totally controlled’’ asthma, scores of <16 were considered ‘‘poorly
controlled’’ or ‘‘not controlled at all,’’ and scores of
16-19 corresponded to ‘‘somewhat controlled’’ asthma.33
Similarly, using the Asthma Control Questionnaire in
adults, Juniper et al32 found that to be confident that a patient has well controlled asthma, the optimal cut point is
0.75, but to be confident that the patient has inadequately
controlled asthma the optimal cut point is 1.50, with the
meaning of values 0.75-1.50 being less certain.
The development of any predictive tool and its cut
points is an ongoing process that involves further validation studies of the C-ACT in varying samples (eg,
demographic and clinical differences). As a result, we are
conducting a longitudinal validation study of the C-ACT
to determine its test-retest reliability and responsiveness.
Within that study, we will assess the tool’s ability to
measure change in asthma control over time and evaluate a
spectrum of cut points that may aid clinicians in being able
to further distinguish a child’s asthma control (eg, somewhat controlled vs. poorly controlled asthma).
In conclusion, the C-ACT is an accurate and reliable
way to assess asthma control in children 4-11 years old.
As a simple self-administered questionnaire, it is suitable
for clinical practice and research settings. It is intended
to complement pulmonary function testing and other
assessments by health care providers and is consistent
with NAEPP1 and GINA2 guidelines. It is designed to
enhance communication of asthma control between children and their parents/caregivers to quickly provide health
care providers with a complete picture of the child’s level
of asthma control.
We thank the members of the Childhood Asthma Control Test
Working Group, including Jonathan Bernstein, MD, Michael Blaiss,
MD, Bradley Chipps, MD, Theresa Guilbert, MD, Priti Jhingran,
PhD, Craig LaForce MD, David Stempel, MD, and Michael Welch,
MD, for their contributions; Bruce Crawford, MA, MPH, Elizabeth
Piault, MA, and Kathleen Rosa, PhD, for their analytical assistance;
and Michael Schatz, MD, for his input during manuscript development. This work is dedicated to the children with asthma and their
parents/caregivers who participated in this study.
REFERENCES
1. The NAEPP Expert Panel Report: guidelines for the diagnosis and
management of asthma—update on selected topics 2002 (EPR—Update
2002). NIH Publication No. 02-5075. Besthesda (MD): National Institutes of Health; 2003.
2. Global Strategy for Asthma Management and Prevention NIH Publication No 02-3659. Besthesda (MD): National Institutes of Health; 2005.
Update available at: http://www.ginasthma.org.
3. Cockcroft DW, Swystun VA. Asthma control versus asthma severity.
J Allergy Clin Immunol 1996;98:1016-8.
4. Yawn BP, Brenneman SK, Allen-Ramey FC, Cabana MD, Markson LE.
Assessment of asthma severity and asthma control in children. Pediatrics
2006;118:322-9.
5. Fritz GK, Yeung A, Wamboldt MZ, Spirito A, McQuaid EL, Klein R,
Seifer R. Conceptual and methodologic issues in quantifying perceptual
accuracy in childhood asthma. J Pediatr Psychol 1996;21:153-73.
6. Santanello NC. Pediatric Asthma assessment: validation of 2 symptom
diaries. J Allergy Clin Immunol 2001;107:S465-72.
7. Eiser C, Morse R. Quality-of-life measures in chronic diseases of childhood. Health Technol Assess 2001;5:1-157.
8. Fritz GK, Overholser JC. Patterns of response to childhood asthma.
Psychosom Med 1989;51:347-55.
9. Townsend M, Feeny DH, Guyatt GH, Furlong WI, Seip AE, Dolovich J.
Evaluation of the burden of illness for pediatric asthmatic patients and
their parents. Ann Allergy 1991;67:403-8.
10. Lara M, Duan N, Sherbourne C, Lewis M, Landon C, Halfon N, Brook
RH. Differences between child and parent reports of symptoms among
latino children with asthma. Pediatrics 1998;102:68.
11. Guyatt GH, Juniper EF, Griffith LE, Feeny DH, Ferrie PJ. Children and
adult perceptions of childhood asthma. Pediatrics 1997;99:165-8.
12. Children and asthma in America. Available at: http://www.asthmainamerica.
com. Accessed: November 30, 2005.
13. Nathan RA, Sorkness CA, Kosinski M, Schatz M, Li JT, Marcus P, Murray JJ, Pendergraft TB. Development of the asthma control test: a survey
for assessing asthma control. J Allergy Clin Immunol 2004;113:59-65.
14. Juniper EF, O’Byrne PM, Guyatt GH, Ferrie PJ, King DR. Development
and validation of a questionnaire to measure asthma control. Eur Respir J
1999;14:902-7.
15. Skinner EA, Diette GB, Algatt-Bergstrom PJ, Nguyen TH, Clark RD,
Markson LE, Wu AW. The Asthma Therapy Assessment Questionnaire
(ATAQ) for children and adolescents. Dis Manage 2004;7:305-13.
Liu et al 825
16. Santanello NC, Barber BL, Reiss TF, Friedman BS, Juniper EF, Zhang J,
et al. Measurement characteristics of two asthma symptom diary scales
for use in clinical trials. Eur Respir J 1997;10:646-51.
17. Sly PD. Relationship between change in PEF and symptoms: questions to
ask in paediatric clinics. Eur Respir J Suppl 1997;24:80S-3S.
18. Lara M, Sherbourne C, Duan N, Morales L, Gergen P, Brook RH. An English
and Spanish pediatric asthma symptom scale. Med Care 2000;38:342-50.
19. Mishoe SC, Baker RR, Poole S, Harrell LM, Arant CB, Rupp NT.
Development of an instrument to assess stress levels and quality of life
in children with asthma. J Asthma 1998;35:553-63.
20. Christie MJ, French D, West A. 1991 guidelines for the diagnosis and
management of asthma. NIH publication no. 97. Bethesda (MD): National Institutes of Heart/National Heart, Lung and Blood Institute; 1997.
21. Le Coq EM, Colland VT, Boeke AJP, Bezemer DP, Van Eijk JTM.
Reproducibility, construct validity, and responsiveness of the ‘‘How
Are You?’’ (HAY), a self-report quality of life questionnaire for children
with asthma. J Asthma 2000;37:43-58.
22. Juniper EF, Guyatt GH, Feeny DH, Ferrie PJ, Griffith LE, Townsend M. Measuring quality of life in children with asthma. Qual Life Res 1996;5:35-46.
23. Santanello NC, Davies G, Galant SP, Pedinoff A, Sveum R, Seltzer J,
et al. Validation of an asthma symptom diary for interventional studies.
Arch Dis Child 1999;80:414-20.
24. Asmussen L, Olson LM, Grant EN, Fagan J, Weiss KB. Reliability and validity of the children’s health survey for asthma. Pediatrics 1999;104:e71.
25. Asthma assessment questionnaire for your child. American Academy
of Allergy, Asthma & Immunology. Available at: http://www.aaaai.org/
members/resources/initiatives/pediatricasthmaguidelines/handouts/sample.
pdf. Accessed May 2006.
26. Millard M. Rules of two. Available at: http://www.baylorhealth.com/
MedicalSpecialties/Asthma/asthmaprograms.htm#Rof2. Accessed May
2006.
27. Juniper EF, Guyatt GH, Feeny DH, Ferrie PJ, Griffith LE, Townsend M.
Measuring quality of life in the parents of children with asthma. Qual
Life Res 1996;5:27-34.
28. American Thoracic Society. Standardization of spirometry—1994
update. Am J Respir Crit Care Med 1995;152:1107-36.
29. Hosmer D, Lemeshow S. Applied logistic regression. New York (NY):
Wiley & Sons; 1989.
30. Juniper EF, O’Byrne PM, Roberts JN. Measuring asthma control in
group studies: do we need airway calibre and rescue b2-agonist use?
Respir Med 2001;95:319-23.
31. Panditi S, Silverman M. Perception of exercise induced asthma by children and their parents. Arch Dis Child 2003;88:807-11.
32. Juniper EF, Bousquet J, Abetz L, Bateman ED. GOAL Committee.
Identifying ‘‘well-controlled’’ and ‘‘not well-controlled’’ asthma using
the Asthma Control Questionnaire. Respir Med 2006;100:616-21.
33. Schatz M, Sorkness CA, Li JT, Marcus P, Murray JJ, Nathan RA, et al.
Asthma Control Test: reliability, validity, and responsiveness in patients
not previously followed by asthma specialists. J Allergy Clin Immunol
2006;17:549-56.
Asthma diagnosis and
treatment
J ALLERGY CLIN IMMUNOL
VOLUME 119, NUMBER 4