Download view web only data 151 KB

Norms and Standardization of Child and Parent Report
Questionnaires and Researcher’s Report Form
Child Report Questionnaire
Distribution of scores
Analyses were carried out to norm and standardize 311 questionnaires completed by
children and adolescents who stutter (CWS). As described in the article, the
questionnaires were based on 1 which was modified for assessment of CWS (a copy of
the Child Report Questionnaire is included as supplementary material). Reponses
were obtained from 252 boys and 59 girls. The range of ages of the CWS was from 8
years 0 months to 19 years 7 months (mean was 13 years 1 month and sd was 2 years
6 months).
The data were approximately normally distributed, as seen in Figure 1 which shows
raw scores and a normal distribution fitted to those scores. The distribution of the raw
scores was tested for skewness and kurtosis. The skewness statistic provides
information about the symmetry of the distribution and the kurtosis statistic provides
information about the peakiness of the distribution. If the distribution was perfectly
normal a skewness and kurtosis value of 0 would be obtained. The skewness and
kurtosis values of the raw scores on the Child Report Questionnaire were converted to
z scores and compared against the values that would be expected by chance alone.
Values above 1.96 (p < .05) were considered to be significantly different from chance.
Neither skewness (z = 1.239) nor kurtosis (z = -1.201) was significant. Statistics about
the distribution of the scores are given in Table 1. The statistics show the distribution
of the data showed was fitted reasonably well by a normal distribution.
Figure 1. Histogram showing distribution of scores on the Child Report Questionnaire
with normal distribution curve fitted.
70
60
50
40
30
Frequency
20
10
0
7.5
10.0
12.5 15.0
17.5 20.0
22.5 25.0
Total score on child questionnaire
27.5 30.0
32.5
35.0
Table 1. Descriptive statistics for total scores on the Child Report Questionnaire.
Mean
Median
Std. deviation
Skewness
Std. error of skewness
Kurtosis
Std. error of kurtosis
19.17
19.00
5.306
0.171
0.138
-0.398
0.276
Construct reliability (internal consistency of the questionnaire)
With regard to questionnaires in general, construct reliability refers to the extent to
which the instrument is an effective measure, whilst validity is the extent to which the
questionnaire is measuring what it purports to measure. If a questionnaire is unreliable
it is impossible for it to be valid; on the other hand it is possible for a questionnaire to
be reliable but invalid. Therefore, we report on reliability followed by validity.
Cronbach's alpha statistic was used as an indication of how well the set of items (or
questions) from the questionnaire measured a single unidimensional latent construct. 2
When the data do not measure a single dimension (they are multidimensional),
Cronbach's alpha is low. An alpha value of 0.8 or higher was used here as a measure
to see if the psychometric instrument developed was useful. 3 The Cronbach alpha
coefficient for the Child Report Questionnaire was 0.81. This indicates high internal
consistency of the items, so the Questionnaire is reliable.
Test-retest reliability
Next reliability across test occasions (test-retest reliability) was assessed. Fifty-one of
the participants were tested twice (the median test-retest interval was 13 days). The
correlation coefficient of the two sets of scores was .74 which indicates test-retest
reliability is acceptable. 4 Minimum score, maximum score, mean and standard
deviation are given separately for the two test occasions in Table 2.
Table 2. Descriptive statistics for total scores on two test occasions for the Child
Report Questionnaire.
Time 1
Time 2
N
Minimum
Maximum
Mean
51
51
10
9
32
28
18.98
18.76
Std.
deviation
4.860
4.348
Internal (construct) validity
Internal (construct) validity was assessed by examining the structure of the test. This
was done by looking at the relationship between responses to different test items. An
unrotated principal axis factor was used to do this and the analysis supported the
interpretation that the test measured two factors. The principal factor accounted for
44.9% of variance and all items loaded at .45 or above on this factor. The second
factor accounted for a further 13.6% of the variance.
The identity of the factors was established by rotating the factors. In this analysis, it
was apparent that items 1, 2, 7 and 8 loaded heavily on the first factor (.6 and above)
and items 3, 4, 5, and 6 loaded heavily on the second factor (again .6 and above).
Examination of the Child Report Questionnaires showed that the items loading on the
first factor were all related to how the child currently felt (for example, “How would
you currently rate your speech”). The items that loaded on the second factor were all
related to comparative assessments (for example, “How much are you
stuttering/stammering now, compared to before you first saw your speech therapist”).
The interpretation of both factors is in terms of judgments either about current
stuttering alone or how this related to their stuttering in the past. The questionnaire
that measures both factors provides a valid measure of stuttering over time.
External (criterion) validity
Criterion validity is a measure of how well set of variables predict an outcome based
on information from other related variables. To assess criterion validity, the raw total
score from the Child Report Questionnaire and stuttering severity (SSI-3) scores 5 for
the corresponding child for 183 of the children were available. The Child Report
Questionnaire showed a high correlation with the SSI-3 scores (r = .66). A linear
regression model showed that the raw score from the Child Report Questionnaire was
a significant predictor of the child’s SSI-3 score (the regression model accounted for
43.1% of the variance in SSI-3 scores). ANOVA indicated that the amount of
variance accounted for was significantly above zero (the null expectation) F(1,180) =
138.11, p <.001).
Standardization
Total raw scores from the 311 questionnaires used for the criterion validity analysis
were standardized to t scores with a mean of 50 and an sd of 10. Scores below 21 on
the Child Report Questionnaire were considered fluent. This was based on analyses of
groups of 35 fluent speakers aged 8-10 and 19 aged 12 plus who had SSI-3 scores up
to, but not above 20.6 An SSI-3 score of 21 is at the 41st percentile of SSI-3, and SSI-3
scores of 21 and below represent mild or very mild stuttering. 5 When the 183
questionnaires from respondents where we also had SSI-3 scores were examined, 71
out of 87 (81.6%) low scores on the Child Report Questionnaire related to children
whose SSI-3 scores were lower than or equal to 20 on the SSI-3 (at the 40th percentile
and below on the SSI-3, which is referred to as mild or very mild stuttering). Eightysix out of 96 (89.6%) high scores on the Child Report Questionnaire related to
children whose SSI-3 scores were greater than or equal to 21. Percentiles for the Child
Report Questionnaire raw scores are given in Table 3.
Table 3. Raw Child Report Questionnaire scores and corresponding percentiles based
on 311 questionnaires. SSI-3 taxonomic severity ratings and actual SSI-3 values are
given for the same percentiles.
Total Raw Score
on Child Report
Questionnaire
9 – 10
11 – 12
13 – 15
16 – 17
18 – 21
22 – 23
24 – 25
26 – 28
Percentile
1-4
5 - 11
12 - 23
24 - 40
41 - 60
61 – 77
78 – 88
89 – 95
SSI-3 severity
rating for same
percentiles
Very mild
Very mild
Mild
Mild
Moderate
Moderate
Severe
Severe
Raw SSI-3 scores
6-8
9-10
11-15
16-20
21-23
24-27
28-31
32-35
96 – 99
29 and up
Very Severe
36 and up
Parent Report Questionnaire
Distribution of scores
Analyses were carried out to norm and standardize the 302 questionnaires completed
by parents of CWS. The questionnaires were again based on 1 but this time the
questions asked about the parent’s view about the CWS (so the questions were altered
to refer to the child in the third person). There were 246 boys and 56 girls about
whom the parents reported. The ages of the CWS ranged from 8 years 0 months to 19
years 7 months (mean was 12 years 11 months and sd was 2 years 6 months). The
parental data were normally distributed, as shown in Figure 2. The skewness and
kurtosis values of the raw scores on the Parent Report Questionnaire were converted
to z scores and compared against values that would be expected based on chance
alone and found not to be significant (Skewness z = 0.857; Kurtosis z = -1.485).
Statistics about the distribution are given in Table 4 which, as was found with the
child version, showed there was no discrepancy from normality.
Figure 2. Histogram showing distribution of scores on the Parent Report
Questionnaire with normal distribution curve fitted.
60
50
40
30
Frequency
20
10
0
7.5
12.5
10.0
17.5
15.0
22.5
20.0
27.5
25.0
32.5
30.0
37.5
35.0
40.0
Total score on Parent Questionnaire
Table 4. Descriptive statistics for the total scores on the Parent Report Questionnaire.
Mean
Median
Std. deviation
Skewness
Std. error of Skewness
Kurtosis
Std. error of kurtosis
21.64
21.00
6.294
0.120
0.140
-0.617
0.280
Construct reliability (internal consistency of the questionnaire)
Cronbach's alpha was again used as an indication of how well the set of items from
the questionnaire measured a single unidimensional latent construct. The Cronbach
alpha coefficient for the Parent Report Questionnaire was 0.87. This indicates high
internal consistency so the questionnaire is reliable. 3
Test-retest reliability
Test-retest reliability was assessed on 51 participants who were tested twice (the
median test-retest interval was 13 days). The correlation coefficient for the two sets of
scores was .85, which indicates good reliability. 4 Minimum score, maximum score,
mean and standard deviation are given separately for the two test occasions in Table
5.
Table 5. Descriptive statistics on two test occasions for the Parent Report
Questionnaire.
Time 1
Time 2
N
Minimum
Maximum
Mean
51
51
8.00
10.00
33.00
39.00
21.1373
21.6078
Std.
deviation
6.22903
6.40649
Internal (construct) validity
An unrotated, principal axis factor analysis showed all of the questions in the Parent
Report Questionnaire loaded at .6 or above on a single factor. This factor accounted
for 54.3% of the variance in the questionnaire. All the questions in the Parent Report
Questionnaire refer to stuttering (current and past) by the child. As such, the factor
provides an integrated impression of stuttering over time. Recall that current and past
stuttering weighted on different factors for the children’s responses. These differences
in the way parents and children evaluate stuttering is interesting, but cannot be
discussed here. The statistical analysis suggests that the parent questionnaire provides
a valid measure of stuttering across time for their child.
External (criterion) validity
Criterion validity was assessed using the raw total scores from the Parent Report
Questionnaire and the corresponding SSI-3 scores for the child. These pairs of scores
were available for 182 parent-child pairs. The Parent Report scores correlated highly
with the SSI-3 score (r = .65). A linear regression analysis showed that the raw score
from the Parent Report Questionnaire was a significant predictor of SSI-3 score (the
regression model accounted for 44.9% of the variance in SSI-3 scores). ANOVA
indicated that the amount of variance accounted for was significantly different above
zero F(1,180) = 148.30, p <.001.
Standardization
Total raw scores from the 302 Parent Report Questionnaires used in the criterion
validity analysis were standardized to t scores with a mean of 50 and an sd of 10. The
same cutoff of 21 as was used with the child form. Eighty-five out of 93 (91.4%) high
scores on the Parent Report Questionnaire related to children whose SSI-3 scores
were greater than or equal to 21. Seventy-two out of 89 (80.9%) low scores on the
Parent Report Questionnaire related to children whose SSI-3 scores were less than or
equal to 20. Percentiles for the Parent Report Questionnaire raw scores are given in
Table 6. The percentiles are given so the SSI-3 column corresponds to that reported
by Riley. 5
Table 6. Raw Parent Report Questionnaire scores and corresponding percentiles based
on 302 questionnaires. SSI-3 taxonomic severity ratings and actual SSI-3 values are
given for the same percentiles.
Total Raw Score
on Parent Report
Questionnaire
9 – 10
11 – 14
14 -1 6
17 – 19
20 – 23
24 – 27
27 – 29
30 – 32
33 and up
Percentile
1–4
5 – 11
12 – 23
24 – 40
41 - 60
61 – 77
78 – 88
89 – 95
96 – 99
SSI-3 severity
rating for same
percentiles
Very mild
Very mild
Mild
Mild
Moderate
Moderate
Severe
Severe
Very Severe
Raw SSI-3 scores
6-8
9-10
11-15
16-20
21-23
24-27
28-31
32-35
36 and up
Researcher’s Report Form
Distribution of scores
Analyses were carried out to norm and standardize the 328 Researcher’s Report
Forms about the CWS. The reports were intended for assessment of CWS and to
assist designation as persistent or recovered. There were 267 boys and 61 girls and the
sample ranged in age from 8 years 1 month to 19 years 11 months (with a mean of 13
years 0 months and an sd of 2 years 7 months).
As seen in Figure 3, the data were normally distributed (Skewness z = 0.770; Kurtosis
z = -1.789). Statistics about the distribution are given in Table 7. These show that the
distribution of raw scores from the Researcher’s Report Form was not significantly
different from normal.
Figure 3. Histogram showing distribution of scores on the Researcher’s Report Form
with normal distribution curve fitted.
80
60
40
Frequency
20
0
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
Total score on researcher report
Table 7. Descriptive statistics of total scores on the Researcher’s Report Form.
Mean
Median
Std. deviation
Skewness
Std. error of skewness
Kurtosis
Std. error of kurtosis
4.00
4.00
1.909
-0.104
0.135
-0.858
0.268
The Researcher’s Report Form was designed as a scale to rate observations about the
children and, as such, was not a psychometric measure. Consequently, construct
reliability, test-retest reliability and internal validity were not relevant and were not
computed.
External (criterion) validity
There were SSI-3 scores for 153 of the children who were assessed by a Researcher’s
Report Form. The raw total scores from the Researcher’s Report Form correlated
highly with the SSI-3 score (r=.77). A linear regression analysis showed that the raw
score from the Researcher’s Report Form was a significant predictor of SSI-3 score,
and this accounted for 58.4% of the variance in SSI-3 scores. ANOVA indicated that
the amount of variance accounted for was significantly above zero F(1,151) = 214.82,
p<.001.
Standardization
The total raw scores from 328 Researcher’s Report Forms were standardized to t
scores with a mean of 50 and an sd of 10. A cutoff value of 5 was used to distinguish
low dysfluency scores (lower values) from high dysfluency scores. Seventy-four out
of 89 (83.2%) high scores on the Researcher’s Report Form related to SSI-3 scores
greater than or equal to 21 (at the 41st percentile and above of SSI-3) which
corresponds to stuttering rated as moderate, severe or very severe). Fifty-five out of
64 (85.9%) low scores on the Researcher’s Report Form related to SSI-3 scores less
than or equal to 20 (at the 40th percentile and below on SSI-3, where stuttering was
classed as mild or very mild). Percentiles for the Researcher’s Report Form raw
scores are given in Table 8 below.
Table 8. Raw Researcher’s Report Form scores and corresponding percentiles based
on 328 questionnaires. SSI-3 taxonomic severity ratings and actual SSI-3 values are
given for the same percentiles.
Total Raw
Score on Parent
Report
Questionnaire
0–1
1
2
2–3
4–5
5
6
6–7
8 and up
Percentile
1-4
5 - 11
12 - 23
24 - 40
41 - 60
61 – 77
78 – 88
89 – 95
96 – 99
SSI-3 severity
rating for
same
percentiles
Very mild
Very mild
Mild
Mild
Moderate
Moderate
Severe
Severe
Very Severe
Raw SSI3 scores
6-8
9-10
11-15
16-20
21-23
24-27
28-31
32-35
36 and
up
References
1. Boberg E, Kully D. Long-term results of stuttering treatment program. J Speech
Hear Res 1994;37:1050-1059.
2. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika
1951;16:297-333.
3. Bryman A. Cramer D. Quantitative Data Analysis. London: Routledge, 1999.
4. Murphy KR, Davidshofer CO. Psychological testing: Principles and applications
(4th edition). New Jersey: Prentice Hall International, Inc., 1996
5. Riley GD. Stuttering Severity Instrument for Children and Adults, Third Edition.
Austin, TX: Pro-Ed, 1994.
6, Howell P, Davis S, Williams R. Late childhood stuttering. J Speech Lang Hear Res
in press.