Download The California School Psychologist

The California School Psychologist
2007, Volume 12
CONTENTS
Editorial
Shane R. Jimerson
The California School Psychologist Provides Valuable
Information to Promote School Success Among Students
with Emotional or Behavioral Disorders
5
Articles
Alberto F. Restori
Functional Assessment-Based Interventions for Children
Frank M. Gresham
At-Risk for Emotional and Behavioral Disorders
Tae Chang
Howard B. Lee
Wilda Laija-Rodriquez
9
Paul Muyskens
Doug Marston
Amy L. Reschly
The Use of Response to Intervention Practices for
Behavior: An Examination of the Validity of a Screening
Instrument
31
Emily S. Fisher
Katie E. Doyon
Enrique Saldaña
Megan Redding Allen
Comprehensive Assessment of Emotional Disturbance:
A Cross-Validation Approach
47
Katina M. Lambros
Shirley K. Culver
Aidee Angulo
Pamela Hosmer
Mental Health Intervention Teams: A Collaborative Model
to Promote Positive Behavioral Support for Youth with
Emotional or Behavioral Disorders
59
Stephen E. Brock
Amanda Clinton
Diagnosis of Attention-Deficit/Hyperactivity Disorder
(AD/HD) in Childhood: A Review of the Literature
73
Gail B. Adams
Thomas J. Smith
Sara E. Bolt
Patrick Nolten
Current Educational Practices in Classifying and Serving
Students with Obsessive-Compulsive Disorder
93
Renée M. Tobin
Frank J. Sansosti
Laura Lee McIntyre
Developing Emotional Competence in Preschoolers:
A Review of Regulation Research and Recommendations
for Practice
107
Erin Dowdy
Randy W. Kamphaus
A Comparison of Classification Methods for Use in
Predicting School-Based Outcomes
121
The California School Psychologist, Vol. 12, pp. 5 – 7, 2007
Copyright 2007 California Association of School Psychologists
The California School Psychologist Provides Valuable
Information to Promote School Success Among Students
with Emotional or Behavioral Disorders
Shane R. Jimerson
University of California, Santa Barbara
This volume of The California School Psychologist provides valuable information to promote the
success of students with emotional or behavioral disorders, as well as other informative articles. These
articles provide valuable information for school psychologists and other professionals working in the
schools, and also contribute to the literature and scholarship that aim to promote the educational success
of all students. Previous articles published in The California School Psychologist, including the recent
volumes addressing a) school engagement, b) strength-based assessment, c) response-to-intervention
(RtI), and d) autism are available on-line at www.education.ucsb.edu/school-psychology.
The first article (Restori, Gresham, Chang, Lee, & Laija-Rodriquez, 2007) reports the results of a study
using functional assessment-based interventions to support fifth-grade students at-risk for emotional and
behavioral disorders. Students whose behavior was found to be functionally related to either task-avoidance or attention-seeking were randomly assigned to a treatment strategy that was primarily either antecedent or consequent-based. This study reports the results of an ABAB single-case design to examine the
effects of treatment strategies. This study also conducted a comparison of treatment strategies that were
primarily antecedent or consequent-based. The authors highlight that antecedent-based treatment strategies (i.e., self-monitoring and task-modification) were more effective than consequent-based treatment
strategies (i.e., differential reinforcement) for increasing academic engagement and reducing disruptive
behavior. They also discuss the implications for using functional assessment strategies.
The second article (Muyskens, Marston, & Reschly, 2007) shares the results of a study that examines
the validity of a screening instrument to identify students at risk for behavior problems. Within a Responseto-Intervention framework, the authors discuss using a screening measure (a 12-item behavior screener
completed by the teacher for each student in the Fall) designed to identify students at risk for behavior
difficulties and targeting these students for early intervention. This study included 22,056 kindergarten
through 8th grade students from the Minneapolis Public Schools. Results revealed that teacher-completed
Fall scores on this measure were significantly correlated with suspensions, achievement scores, and
attendance data over the course of the school year. The authors emphasize that providing teachers with
a tool which in less than a minute can help them identify students likely to present significant behavioral
or academic concerns over the course of the school year warrants further consideration.
The third article (Fisher, Doyon, Saldaña, & Redding-Allen, 2007) discusses a comprehensive assessment of emotional disturbance, advocating cross validation procedures, and using the RIOT approach
(Review, Interview, Observe, Test). The authors advocate that using all four RIOT processes together
allows for comprehensive assessment for emotional disturbance. The authors also encourage school
psychologists to consider a student’s strengths, cultural factors, and the interaction between the student
and the environment when interpreting the results of assessments. Using the multi-faceted information
yielded through a comprehensive assessment, school psychologists may provide recommendations to
help remediate areas of weakness and promote social and cognitive competence.
Address correspondence and reprint requests to Shane R. Jimerson; University of California, Santa Barbara; Department of Counseling, Clinical, and School Psychology; Center for School-Based Youth Development; Phelps Hall; Santa
Barbara, CA 93106-9490 or e-mail [email protected]
The California School Psychologist, 2007, Vol. 12
The fourth article (Lambros, Culver, Angulo, & Hosmer, 2007) describes a collaborative intervention model for promoting mental health and positive social adjustment for youth with emotional or
behavioral disorders (EBD). The mental health intervention team (MHIT) includes a collaborative partnership between the Mental Health Resource Center (MHRC) and the Emotional Disturbance Program
(ED) and also includes research and evaluation consultation from the Child and Adolescent Services
Research Center (CASRC). The MHIT is a collaborative service delivery model using school-based
mental health teams to implement evidence-based interventions to promote positive social adjustment
for youth with EBD and their families as well as support classroom teachers. The authors advocate
that through integrating program components and combining fiscal resources to develop the MHIT, a
more efficient use of resources may be possible, resulting in increased services to students and teachers
beyond what is offered when mental health and educational programs work in isolation.
The fifth article (Brock & Clinton, 2007) reviews recent literature related to the diagnosis of Attention-Deficit/Hyperactivity Disorder (AD/HD) in childhood. This article discusses diagnostic criteria
presented in the Diagnostic and Statistical Manual of Mental Disorders and explores the diagnostic
procedures for AD/HD recommended in current publications. The authors highlight that there is no
single diagnostic procedure, or set of procedures, has been identified that will diagnose AD/HD with
perfect reliability.
Rather, this literature review reveals that rating scales, interviews, laboratory/ psychological testing,
and observations are the most frequently recommended AD/HD diagnostic techniques. This information
can be valuable for school psychologists and other education professionals and mental health professionals who are responsible for assessing and identifying students with AD/HD.
The sixth article (Adams, Smith, Bolt, & Holten, 2007) describes a study of contemporary educational practices in classifying and serving students with obsessive-compulsive disorder (OCD). This
study used a survey to examine current practices for classifying and serving students with a primary
diagnosis of OCD. The results indicated that fewer than one percent of the students served by school
psychologists had a primary diagnosis of OCD, and that the majority of these students were served under
IDEA. Of the students receiving services under IDEA, half were classified under E/BD and one-third
under OHI. Approximately two-thirds of the students with OCD were educated in less restrictive settings
(e.g., regular classroom with or without resource/part-time special class). The authors conclude that the
results suggest a pattern of ambiguity and uncertainty surrounding the appropriateness of IDEA categories for OCD, concerns regarding the stigma of labeling, and problems related to providing appropriate
services to these students.
The seventh article (Tobin, Sansoti, & McIntyre, 2007) focuses on the development of emotional
competence in preschoolers. This article provides a review of emotional regulation research from the
past two decades. Specifically, the developmental literature for both regulation and dysregulation of
emotion is reviewed and the implications for both assessment and intervention with preschool children
are discussed. Regulation has been implicated in the development of emotional and behavioral disorders
in childhood. The literature reveals that emotion dysregulation is one of the most common reasons families seek psychological services and behavioral supports. The authors emphasize that interventions to
support children with regulatory difficulties will be enhanced when they are informed by basic psychological research on the topic. The authors also highlight the role that school psychologists and schoolbased interventions may play in supporting appropriate regulatory strategies for young children.
The eighth article (Dowdy & Kamphaus, 2007) reports the results of a study comparing three classification methods (categorical, dimensional, person-oriented) in predicting school-based outcomes.
The California School Psychologist
Elementary school-age children (N=558) were administered the Behavior Assessment System for Children – Teacher Rating Scale and educational outcome variables were collected seven months later.
Results revealed that all three methods for predicting educational outcomes were modest and were best
able to predict later grade point averages. The authors emphasize the relative superiority of personoriented and dimensional methods of classification and also call for further investigations.
This collection of articles provides valuable information that may be used by educational professionals working with children, families, and colleagues to enhance the academic success and promote
positive developmental trajectories of students. The authors of the manuscripts in this volume provide
valuable information and insights that advance our understanding of numerous important topics. The
California School Psychologist contributes important information regarding promoting the social and
cognitive competence of all students.
References
Adams, G. B., Smith, T. J., Bolt, S. E., & Holten, P. (2007). Current educational practices in classifying and serving
students with Obsessive-Compulsive Disorder. The California School Psychologist, 12, 93 – 105.
Brock, S. E. & Clinton, A. (2007). Diagnosis of Attention-Deficit/Hyperactivity Disorder (AD/HD) in childhood: A
review of the literature. The California School Psychologist, 12, 73 – 91.
Dowdy E. & Kamphaus, R. W. (2007). A comparison of classification methods for use in predicting school-based
outcomes. The California School Psychologist, 12, 121 – 132.
Fisher, E. S., Doyon, K. E., Saldaña, E., & Redding-Allen, M. (2007). Comprehensive assessment of emotional
disturbance: A cross-validation approach. The California School Psychologist, 12, 47 – 58.
Lambros, K. M., Culver, S. K., Angulo, A. & Hosmer, P. (2007). Mental health intervention teams: A collaborative
model to promote positive behavioral support for youth with emotional or behavioral disorders. The California
School Psychologist, 12, 59 – 71.
Muyskens, P., Marston, D., & Reschly, A. L. (2007). The use of response-to-intervention practices for behavior: An
examination of the validity of a screening instrument. The California School Psychologist, 12, 31 – 45.
Restori, A. F., Gresham, F. M., Chang, T., Lee, H. B., & Laija-Rodriquez, W. (2007). Functional assessment-based
interventions for children at-risk for emotional and behavioral disorders. The California School Psychologist,
12, 9 – 30.
Tobin, R. M., Sansoti, F. J., & McIntyre, L. L. (2007). Developing emotional competence in preschoolers: A review
of regulation research and recommendations for practice. The California School Psychologist, 12, 107 – 120.
The California School Psychologist, Vol. 12, pp. 9 – 30, 2007
Copyright 2007 California Association of School Psychologists
Functional Assessment-Based Interventions for Children
At-Risk for Emotional and Behavioral Disorders
Alberto F. Restori
California State University, Northridge
Frank M. Gresham
Louisiana State University
Tae Chang,
Howard B. Lee &
Wilda Laija-Rodriquez
California State University, Northridge
Functional assessments were conducted to identify the variables maintaining disruptive behavior in
eight, typically developing fifth-grade students enrolled in general education classrooms. Participants
whose behavior was found to be functionally related to either task-avoidance or attention-seeking
were randomly assigned to a treatment strategy that was primarily either antecedent- or consequentbased. An ABAB single-case design was employed to analyze the effects of treatment strategies. The
current study also conducted a comparison of treatment strategies that were primarily antecedent- or
consequent-based. Results showed that antecedent-based treatment strategies (i.e., self-monitoring and
task-modification) were more effective than consequent-based treatment strategies (i.e., differential
reinforcement) for increasing academic engagement and reducing disruptive behavior. Implications
regarding the use of functional assessment with typically developing students at-risk for emotional
and behavioral problems enrolled in general education classrooms and the effects of antecedent- and
consequent-based treatment strategies as a function of behavior are discussed.
KEYWORDS: Emotional and Behavioral Disorders, Functional Assessment, Behavioral Assessment,
Behavioral Interventions, Treatment Strategies.
Students with emotional and behavioral disorders (EBD) are characterized by a number of behavioral, social, and academic characteristics that pose challenges to teachers and administrators. When
more global intervention efforts such as primary and secondary prevention programs prove insufficient
for shaping behaviors, more ideographic efforts, such as functional assessment-based interventions, are
invoked (Horner & Sugai, 2000; Lane, Robertson, & Graham-Bailey, 2006).
Functional assessment involves the full range of procedures (e.g., interviews, direct observations,
and rating scales) used to identify the antecedent conditions that set the stage for undesirable (target)
behaviors to occur and the maintaining consequences (Gresham, Watson, & Skinner, 2001; Horner,
1994). These data are used to develop a hypothesis statement that can then be tested via experimental
manipulation of environmental events. Subsequently, an intervention is designed based on the function
of the target behavior.
While the original research on function-based interventions originated in analogue conditions
(Iwata, Dorsey, Slifer, Bauman, & Richmond, 1982), this ideographic approach to intervention has also
proved successful in self-contained (Dunlap et al., 1993), inclusive (Kamps, Wendland, & Culpepper,
Please send correspondence to Alberto Restori, PhD, Assistant Professor, Department of Educational Psychology
and Counseling, California State University, Northridge, 18111 Nordhoff Street, Northridge, California 91330.
Email: [email protected]
10
The California School Psychologist, 2007, Vol. 12
2006; Lane, Weisenbach, Little, Phillips, & Wehby, 2006; Lewis & Sugai, 1996; Umbreit & Blair, 1997;
Umbreit, Lane, & Dejud, 2004), and preschool (Umbreit, 1996) settings with a wide range of students,
including students with and at risk for EBD (Kern, Delaney, Clarke, Dunlap, & Childs, 2001; Kern,
Hilt, & Gresham, 2004; Lane, Umbreit, & Beebe-Frankenberger, 1999; Sasso, Conroy, Stichter, & Fox,
2001).
Despite these successful demonstrations of functional assessment-based interventions in applied
settings, some argue that the literature base is limited by the absence of functional analyses; the lack
of a systematic approach to the process; and questionable reliability and validity of some of the tools
employed (Sasso et al., 2001). In addition, other concerns focus on the ability to achieve the appropriate
balance between scientific rigor and feasibility when conducting research in applied settings (Scott et
al., 2004) and the goal of focusing more primarily on antecedent-based, rather than consequent-based
interventions (Restori et al., in review).
Historically, many teachers and researchers have relied heavily on consequent-based interventions
in which the target behavior must occur and subsequently be shaped by the consequences that follow
(Lewis & Sugai, 1996; Martens, Peterson, Witt, & Cirone, 1986). That is, consequences, often in the
form of punitive responses, are applied after the occurrence of an academic or behavioral problem.
Research by Newcomer and Lewis (2004) indicates that such a consequent-based approach is not likely
to offer the best approach for remediation of academic and behavioral challenges.
One review of the research on behavior disorders indicated that only 11.1% of the individuals treated
for maladaptive behavior received treatments that were based primarily upon the manipulation of antecedent variables (Lennox, Miltenberger, Spengler, & Erfanian 1988). More recently, increased attention
has been placed on the value of antecedent-based interventions in which environmental and curricular
modifications are made to prevent the problem behavior from occurring (Clarke et al., 1995; Dunlap,
White, Vera, Wilson, & Panacek, 1996; Kern et al., 2001; Kern & Clemens, 2007). While most function-based interventions contain both components (antecedent adjustments and modification of the reinforcement schedules), one could argue that an intervention could be either primarily antecedent-based or
primarily consequent-based. Given the increased emphasis on prevention, one question arises as to the
extent to which interventions that are primarily antecedent- or consequent-based are equally effective in
producing meaningful, lasting change.
To this end, Restori and colleagues (in review) conducted as series of function-based interventions, some primarily antecedent-based and others primarily consequent-based, with eight second grade
students with disruptive behavior patterns. Functional assessments were conducted to identify the maintaining variables, which yielded those maintained by task-avoidance or attention-seeking. Results of a
series of ABAB withdrawal designs suggested that antecedent-based treatment strategies (i.e., self-monitoring and task-modification) were as effective and efficient as consequent-based treatment strategies
(i.e., differential reinforcement) for increasing academic engagement and reducing disruptive behavior
for these general education students. However, questions arise as to the consistency of these findings
with older students, who may be less amenable to intervention efforts (Walker, Ramsey, & Gresham,
2004). That is, previous research has demonstrated that disruptive patterns of behavior become more
stable as children grow older and are likely to be more resistant to change (e.g., Olweus, 1979; Walker
et al., 2004). Therefore, the authors of the current study investigated whether antecedent-based and
consequent-based interventions were equally effective for upper elementary students with disruptive
behaviors.
Functional Assessment-based Interventions
11
Purpose
This study seeks to extend the Restori and colleagues (in review) investigation by examining the
efficacy of function-based interventions that were either primarily antecedent-based or primarily consequent-based with fifth-grade students whose patterns of disruptive behavior were similar to that of their
second grade sample. We hypothesize that results will be comparable to those in the study of second
grade students, but that the magnitude of improvement may be less given that behavior patterns become
more resistant to intervention efforts over time (Walker et al., 2004).
Method
Participants and Setting
Eight fifth-grade, general education students from two elementary schools in southern California
were selected to participate in the study. Written consent to conduct the study was granted by the Director
of Special Education of the school district, school principals, and the fifth grade teachers of the two
schools included in the study. Parents of the participants gave written permission for their child to be
included in the study and had the option to withdraw their child from the study at any time. All of the
participants gave their verbal consent to be included in the study and were given the option to withdraw
from the study with their parent’s permission. The district serves an urban, ethnically diverse population with socioeconomic status ranging from lower to upper-middle class. Four of the participants were
African American, two Latino, and two Anglo. All eight participants were male and five of the eight
participants were enrolled in general education classes for their entire school day. Three of the participants received special education support (i.e., resource specialist program [RSP]) for an hour per day
(i.e., homework club). None of the participants had a neurological, psychiatric, or physical disability
that could prevent him from behaving appropriately (e.g., autism) or that would interfere with academic
performance (e.g., mental retardation). None of the participants in the study was classified as Emotionally Disturbed (ED), nor enrolled in a Special Day Class (SDC). Information regarding special education
support, disabilities, and/or placement was obtained from parent and teacher interviews, school records,
and student observations.
Procedures
Fifth grade teachers from the two elementary schools were asked to nominate his or her three to five
most disruptive students. Students nominated for the study had varying degrees of suspensions, referrals
to the school principal or counselor, and out-of-class suspensions (OCS), however, school personnel
agreed that all of the students nominated for inclusion to the study demonstrated significant behavioral
problems and were considered at-risk for emotional and behavioral disorders. A functional assessment
consisting of direct student observations employing an A-B-C approach, (i.e., descriptive assessment),
interview with the classroom teacher, and completion of the Social Skills Rating System-Teacher form
(SSRS-T; Gresham & Elliott, 1990) was conducted with each participant to determine the degree of
disruptive, off-task, and on-task behaviors. Preliminary classroom observations were 15 minutes in duration and used for gathering baseline data of the referred students as well as to identify other potential
participants for the study. Only students whose disruptive behavior was clearly identified as having a
functional relationship to either task-avoidance or attention-seeking were included in the study. Students
whose behavior served a dual function (i.e., task-avoidance and attention-seeking), was undifferentiated (i.e., not clearly task-avoidance or attention-seeking), or whose behavior was functionally related
12
The California School Psychologist, 2007, Vol. 12
to sensory reinforcement were excluded from the study. Although some students may have multiple or
undifferentiated functions of behavior, previously cited research has demonstrated that a comprehensive
functional assessment is likely to result in an accurate hypothesis statement regarding a child’s primary
function of behavior within a given setting. After completion of the identification process, the eight
participants were randomly assigned to receive a treatment strategy that was either primarily antecedentor consequent-based and matched to their individual function of behavior.
Students selected to participate in the study met the following criteria. First, participants must
exhibit disruptive behavior for a minimum of 25% of the intervals observed during four or more of
the baseline observations. Second, students must be academically engaged during less than 25% of the
intervals observed during four or more of the baseline observations. Classroom observations and teacher
reports indicate that the majority of students engaged in disruptive behavior less than 10% of the time
and were academically engaged or engaged in task-related activities at least 90% of the time. Third, the
overall Social Skills and Academic Competence scores of the SSRS-T was below the 25th percentile and
the overall Problem Behaviors score of the SSRS-T was above the 75th percentile. Finally, as previously
stated, the function of behavior (i.e., task-avoidance or attention-seeking) must be clearly identified.
Participants’ data were coded and reported by letter and number to maintain their anonymity. The
letter referred to the treatment strategy the participant received. Participants receiving an antecedentbased treatment strategy were coded with the letter A and participants receiving a consequent-based
treatment strategy were coded with the letter C. Participants 1 and 2 for both the A and C groups designate the participants whose function of behavior was attention-seeking. Likewise, participants 3 and 4
for both the A and C groups designate the participants whose function of behavior was task-avoidance.
For example, participant A1 was a participant assigned to an antecedent-based treatment strategy whose
disruptive behavior was functionally related to attention-seeking.
Measures
Disruptive behavior. Disruptive behavior was defined employing a modified version of the seven
general categories of behavior incompatible with learning described by Becker, Madsen, Arnold, and
Thomas (1967). The following four general categories were used to define disruptive behavior for the
current study: (a) unauthorized out-of-seat behaviors, (b) disruptive noise, (c) disturbing others, and (d)
talking without teacher permission. Out-of-seat behaviors include any unauthorized or non-task related
movement within the classroom. Disruptive noise included excessive flipping of pages, pencil tapping,
or any non-task related noise (e.g., humming). Disturbing others included physical contact with another
student, their desk, any objects on another student’s desk; and aggressive behavior (e.g., hitting another
student). Talking without teacher permission included responding to a teacher’s question without being
called upon to reply, talking to another student while the teacher is giving a lesson, or talking during a
written assignment.
Academic engagement. The current study adopted the definition of Academic Engaged Time (AET)
from Walker and Severson’s (1991) Systematic Screening for Behavior Disorders (SSBD). Academic
engagement was defined as a student properly working on assigned academic material. An academically
engaged student is (a) attending to the material and task, (b) making appropriate motor and/or verbal
responses (e.g., writing, computing, answering questions), and (c) asking for assistance (when appropriate) in an acceptable manner. For example, a student listening to a teacher’s lesson or computing
assigned math problems is considered to be academically engaged. A student not actively working on a
class assignment, not attending to the teacher’s lesson, and/or breaking classroom rules are examples of
Functional Assessment-based Interventions
13
not exhibiting academically engaged behavior.
Technical Instrumentation
Social Skills Rating System – Teacher (SSRS-T; Gresham & Elliott, 1990). The SSRS-T provides
a broad assessment of a student’s social behaviors that can affect teacher-student relations, peer acceptance, and academic performance. The SSRS-T was standardized on a representative sample of students
3-18 years. The SSRS-T documents the perceived frequency and importance of behaviors influencing
students’ development of social competence and adaptive functioning at school. The teacher version
contains ratings of three social skill domains (Cooperation, Assertion, and Self-Control) and three
problem behavior domains (Internalizing Problems, Externalizing Problems, and Hyperactivity Problems). The SSRS-T also contains a teacher rating of academic competence. Extensive evidence of reliability, as well as content, social and criterion-related validity is provided in the SSRS manual.
Observation and Recording Procedures
Students selected as participants for the study exhibited high rates of disruptive behavior and low
rates of academic engagement as a function of either task-avoidance or attention-seeking. Observations
for all participants in all phases of the study were conducted during reading or reading related lessons
based on information gathered from teacher interviews. Although teachers reported that the participants
demonstrated disruptive behavior throughout the day, they reported that such behaviors appeared to be
more frequent during language arts instruction. Participants were observed in 10-second intervals for
15 minutes. A partial-interval, time-sampling procedure was used for recording off-task and disruptive
behavior and a whole-interval, time-sampling procedure for recording academic engagement. At the end
of each 10-second interval, the participant’s behavior was recorded indicating academic engagement
(AE), off-task (OFF), or disruptive behavior (DIS). When a participant exhibited any off-task or disruptive behavior during a given interval, DIS or OFF was recorded on the observation form. If a participant
displayed both disruptive and off-task behaviors within a given interval, DIS was recorded on the observation form. Participants had to remain academically engaged throughout the entire 10-second interval
to be considered academically engaged (AE). The percentages of disruptive behavior, off-task behavior,
and academic engagement were calculated for each student after each 15-minute observation. Although
there was some risk of over-estimating disruptive behavior and underestimating academic engagement
using this observation system, this approach would likely produce more meaningful information when
comparing treatment outcomes to baseline levels of each behavior. To calculate the percentage of intervals in which disruptive behavior, off-task behavior, or academic engagement occurred, the number of
intervals in which each occurred was divided by the total number of intervals and multiplied by 100.
Each 15-minute observation session was considered one data point and a minimum of four data points
was required to establish a stable rate of responding in both baseline and treatment phases.
Inter-observer agreement of direct observations was conducted by the first author and a research
assistant (RA) by observing the target student at the same time from different places in the classroom
employing the previously described observation and recording procedures. A minimum of 20% of all
observation sessions included inter-rater reliability probes to ensure adequate levels of reliability. Interrater reliability probes were conducted across all phases of the study. The percentage of inter-observer
agreement for all reliability probes was 80% or above.
14
The California School Psychologist, 2007, Vol. 12
TABLE 1. Description of Antecedent-Based Treatment Strategies for Fifth Grade Participants
Student
Assignment
Hypothesis
Modifications
A1
Students were
expected to read
silently. Questions
pertaining to reading
assignment were
asked after specified
period of time.
Classroom points
were contingent on
all students’
participation.
AttentionSeeking
Self-Monitoring
1.Explain self-monitoring
procedure to A1.
2.Provide self-monitoring form.
3.Provide 5 verbal/physical
prompts.
4.Student and teacher monitor
on-task behavior.
5.Compare self-monitoring forms
6.Provide verbal praise for
accurate self-monitoring.
7.Provide preferred activity for
accurate self-monitoring and ontask-behavior.
A2
Students were
expected to read
silently or complete
unfinished work
during this time.
Students were
expected to read or
work silently without
disturbing other
students.
AttentionSeeking
Self-Monitoring
1.Explain self-monitoring
procedure toA2.
2.Provide self-monitoring form.
3.Provide 5 verbal/physical
prompts.
4.Student and teacher monitor
on-task behavior.
5.Compare self-monitoring forms.
6.Provide verbal praise for
accurate self-monitoring.
7.Provide preferred activity for
accurate self-monitoring and ontask-behavior.
Functional Assessment-based Interventions
A3
A4
Students were
expected to read
silently. Questions
pertaining to reading
assignment were
asked after
specified period of
time. Classroom
points were
contingent on all
students
participation.
Task
Avoidance
Students were
expected to read
silently from
pre-assigned
material. Students
were expected to
complete
associated
worksheets and not
disturb other students.
Task
Avoidance
Task-Modification
1.Tell A3 to read grade level book
for 10 minutes before he can read
preferred book.
2.Allow A3 to select a book/story
he likes.
3.Pair A3 with a classmate to read
with.
4.After 10 minutes, ask A3
questions about story.
5.Provide verbal praise for
answering questions and provide
time to read preferred story.
Task-Modification
1.Tell A4 to read grade level book
for 10 minutes before he can read
preferred book.
2.Allow A4 to select a book/story
he likes.
3.Pair A4 with a classmate to read
with.
4.After 10 minutes, ask A4
questions about story.
5.Provide verbal praise for
answering questions and provide
time to read preferred story
15
16
The California School Psychologist, 2007, Vol. 12
TABLE 2. Description of Consequent-Based Treatment Strategies for Fifth Grade Participants.
Student
Assignment
Hypothesis
Modifications
C1
Students were expected
to read silently from
pre-assigned material.
Students were expected
to complete associated
worksheets and not disturb
other students.
AttentionSeeking
DRO with preferred activity
1.Assign C5 a book to read.
2.Provide immediate verbal praise
for academic engagement in the
absence of disruptive behavior 5
times within 15 minute
observation period.
3.Provide free-time for to draw for
appropriate behavior.
C2
C3
C4
Students were expected to
read silently or complete
unfinished work during
this time. Students were
expected to read or work
silently without disturbing
other students.
AttentionSeeking
Students were expected
to complete math worksheets, which included
word problems. Students
were expected to work
silently without disturbing
other students.
Task
Avoidance
Students were expected
to read silently from
pre-assigned material.
Students were expected
to complete associated
worksheets and not disturb
other students. Two tables
were selected each day to
read in small groups for
the teacher.
Task
Avoidance
DRO
1.Provide reading assignment.
2.Provide immediate verbal praise
for academic engagement in the
absence of disruptive behavior 5
times within 15 minute
observation period.
DRO
1.Provide math worksheet.
2.Provide immediate verbal praise
for academic engagement in the
absence of disruptive behavior 5
times within 15 minute
observation period.
DRO
1.Assign reading or spelling lesson.
2.Provide immediate verbal praise
for academic engagement in the
absence of disruptive behavior 5
times within 15 minute
observation period.
Functional Assessment-based Interventions
17
Interventions
Participants for the current study were assigned to a treatment strategy that was either primarily
antecedent- or consequent-based and matched to their function of behavior. Antecedent-based treatment
strategies consisted of self-monitoring for the two participants whose disruptive behavior was functionally related to attention-seeking and task-modification for the two participants whose disruptive behavior
was functionally related to task-avoidance. Consequent-based treatment strategies consisted of differential reinforcement of other behaviors (DRO) for three of the participants and DRO with preferred activity
for one participant (see Tables 1 and 2 for descriptions of each participant’s treatment plans). Although
each participant received his own individual treatment plan, the following is a general overview of the
interventions used, by whom and how they were implemented, and other pertinent information related
to the intervention strategies used.
Self-monitoring. The current study modified the self-monitoring procedures described by Shapiro
and Cole (1992) to fit the general education classroom and curriculum. In the current study, the teacher
was trained by the primary investigator to implement the self-monitoring procedure. The classroom
teacher was required to tap on the student’s shoulder when it was time to self-monitor their behavior. Selfmonitoring was done on a variable-interval 3-minute schedule (VI-3m). At that time, both the teacher
and student monitored the student’s behavior. During natural breaks in the school day (e.g., recess and
lunch-time), the teacher and student reviewed the student’s self-monitoring forms. Reinforcement in the
form of verbal praise and access to preferred activities were dispensed for accurate self-monitoring and
on-task behavior.
Task-modification. The current study modified the task-modification procedure described by Dunlap
and Kerns (1996) to fit the general education classroom and curriculum. In the current study, the teacher
was trained by the primary investigator to implement the task-modification procedure. Participants
receiving a task-modification intervention strategy were to select two books to read. One book would be
a preferred book, the other, a grade-level book. The participant was to read the grade-level book with a
preferred, competent reading peer for 10 minutes. Once the passage was read, the teacher asked questions pertaining to the reading to assess comprehension. After reading the grade-level book, the participant was permitted to read his preferred book and provided with verbal praise.
Differential reinforcement. Differential reinforcement typically involves withholding reinforcement
of an undesirable behavior (extinction) and delivering reinforcement contingent on other appropriate
behavior (Marcus & Vollmer, 1996). Teachers were trained by the primary investigator to deliver a modified differential reinforcement of other behaviors (DRO). DRO is the delivery of reinforcement immediately following the performance of a desired behavior in the absence of the target behavior (Cooper,
Herron, & Heward, 2007). Participants were provided with verbal praise for academic engagement in
the absence of disruptive behavior.
Experimental Design and Data Analysis
Intervention outcomes were analyzed using a within-series, ABAB reversal design. In this design,
each participant serves as his own control, thereby, minimizing the idiosyncratic interactions that have
the potential for confounding the interpretations and results when comparing one subject with another.
In the current study, baseline and reversal to baseline phase data were collected until a stable trend, with
a minimum of four data points per phase was established. In both of these phases, the classroom teacher
delivered instruction using his or her own behavior management system without any additional interventions for the participants of the current study. Similarly, during both treatment phases, data were collected
18
The California School Psychologist, 2007, Vol. 12
for a minimum of four data points for each treatment phase to determine the effects of the intervention
strategies. Two follow-up phases at 15 and 30 days were also conducted to evaluate the maintenance of
increased academic engagement and reduced disruptive behavior as a function of treatment. During the
follow-up phases, teachers were not required to implement the student’s previously developed individual
treatment plans. Graphed data were analyzed using traditional visual inspection (Johnston & Pennypacker, 1993) and mean score comparison techniques to determine the effects of treatment strategies.
Treatment Integrity
Treatment integrity is defined as the degree to which a treatment plan is implemented as intended
(Gresham, 1989). Unless an intervention is carried out exactly as planned, results regarding change in the
target behavior cannot be attributed to the intervention. Fuchs and Fuchs (1989) recommend the use of
a component analysis checklist, wherein, a list of the components of an intervention strategy is provided
to the teacher. In the current study, the teacher and researcher checked off each component of the intervention strategy as they were implemented during each observation. The component analysis checklist
was reviewed by the primary investigator to determine if a particular component of the intervention
strategy was consistently ignored. Although the percentage of treatment integrity for a given treatment
strategy was not calculated, a visual inspection of the component analysis checklists completed by both
the teacher and researcher indicate that the components of each intervention strategy were implemented
with high integrity.
Results
Figures 1 and 2 display the percentage of intervals of academic engagement and disruptive behavior
for each participant across baseline, treatment, and follow-up phases. Overall, both antecedent- and
consequent-based treatment strategies were effective, regardless of function of behavior. Table 3 provides
the means and standard deviations for both academic engagement and disruptive behavior during each
phase of the study. Results of the SSRS-T pre and post measures indicate relatively small and inconsistent changes on its various scales regardless of treatment strategy or function of behavior (see Table 4).
Although results of the SSRS-T pre and post measures were relatively small and inconsistent, teachers
reported they observed positive changes in the social skills domains assessed by the SSRS-T in all of
their students included in the study. Furthermore, the data from both Figures and Table 3 provide clear
evidence of meaningful and substantial change in behavior resulting from both antecedent- and consequent-based treatment strategies.
Academic engagement
Most participants displayed stable patterns of low academic engagement during the initial baseline
sessions. Although all participants met the pre-established criteria of academic engagement for inclusion
into the study, when compared to each other they exhibited considerable variability in academic engagement during the initial baseline phase of the study. Overall, participants were academically engaged an
average of 22.4% (SD=16.9) of the intervals during the initial baseline phase. A low pattern of academic
engagement was not as distinct during the reversal to baseline phase as it was during the initial baseline
phase. During the reversal to baseline phase, overall average academic engagement was observed during
37.4% (SD=22.9) of the intervals.
All participants showed a significant increase in academic engagement during both treatment phases.
Functional Assessment-based Interventions
19
Figure 1. Percentage of intervals for academic engagement and disruptive behavior during baseline,
treatment, and follow-up phases.
20
The California School Psychologist, 2007, Vol. 12
Functional Assessment-based Interventions
21
Figure 2. Percentage of intervals for academic engagement and disruptive behavior during baseline,
treatment, and follow-up phases..
22
The California School Psychologist, 2007, Vol. 12
Functional Assessment-based Interventions
23
TABLE 3. Means and Standard Deviations for Academic Engagement and Disruptive Behavior of
Eight 5th-Grade Students
Academic Engagement
Overall
Baseline
Treatment 1
Baseline
Treatment 2
Follow Up 1
Follow Up 2
Attn. Seek / Antecedent
Baseline
Treatment 1
Baseline
Treatment 2
Follow Up 1
Follow Up 2
Attn. Seek / Consequent
Baseline
Treatment 1
Baseline
Treatment 2
Follow Up 1
Follow Up 2
Task Avoid / Antecedent
Baseline
Treatment 1
Baseline
Treatment 2
Follow Up 1
Follow Up 2
Task Avoid / Consequent
Baseline
Treatment 1
Baseline
Treatment 2
Follow Up 1
Follow Up 2
Disruptive Behavior
M
SD
M
SD
22.35
86.90
37.41
83.08
75.29
80.00
16.89
12.26
22.86
18.57
25.04
20.99
59.53
6.77
38.31
6.49
15.62
10.81
20.44
7.63
23.90
8.04
22.99
15.98
M
SD
M
SD
21.53
83.67
50.11
76.58
74.78
81.00
14.45
9.27
28.11
25.18
32.46
13.11
68.40
9.33
35.78
10.42
22.78
8.75
19.18
6.28
28.52
12.62
32.47
7.27
M
SD
M
SD
26.93
89.13
33.50
88.79
63.75
82.00
23.45
10.13
26.31
11.16
24.46
18.57
53.13
5.13
40.40
6.21
15.25
12.75
27.07
4.67
31.79
6.49
12.42
13.65
M
SD
M
SD
14.92
96.89
34.50
96.00
85.00
85.75
14.95
4.26
14.55
3.46
7.35
6.18
60.17
0.89
37.70
0.85
5.00
4.25
17.59
2.32
11.40
1.21
6.22
4.79
M
SD
M
SD
24.85
75.88
32.80
68.92
78.25
71.25
10.25
13.32
19.70
16.42
20.07
38.67
56.08
13.13
39.10
9.00
10.50
17.50
12.27
9.70
23.16
4.63
13.40
29.77
24
The California School Psychologist, 2007, Vol. 12
TABLE 4. SSRS pre-test and post-test results. Standard scores and percentiles for Social Skills,
Problem Behaviors, and Academic Competence.
Student
Social Skills
Pre-test
Post-test
Problem Behaviors
Pre-test
Post-test
Academic Competence
Pre-test
Post-test
A1
85 (16%)
91 (27%)
118 (88%) 123 (94%)
82 (12%)
91 (27%)
A2
73 (4%)
78 (7%)
127 (96%) 125 (95%)
78 (7%)
83 (13%)
A3
70 (2%)
95 (37%)
127 (96%)
95 (37%)
85 (16%)
83 (13%)
A4
81 (10%)
90 (25%)
125 (95%) 112 (79%)
74 (4%)
83 (13%)
C1
82 (12%)
82 (12%)
112 (79%) 110 (75%) 112 (79%) 110 (75%)
C2
86 (18%)
92 (30%) 136 (>98%) 135 (>98%)
74 (4%)
82 (12%)
C3
46 (<2%)
90 (25%)
95 (37%)
79 (8%)
78 (7%)
C4
76 (5%)
86 18%
120 (91%) 115 (84%)
76 (5%)
76 (5%)
108 (70%)
Overall, academic engagement increased to an average of 86.9% (SD =12.3) of the intervals during
Treatment phase 1. All participants also displayed a significant increase in academic engagement when
treatment strategies were re-introduced after the reversal to baseline phase was concluded. The overall
mean of academic engagement increased to 83.1% (SD =18.6) of the intervals during Treatment phase 2.
Results regarding the overall maintenance of treatment effects indicate that increased levels of academic
engagement were sustained over 15- and 30-day periods. Overall academic engagement was observed an
average of 75.3% (SD =17.5) and 80.0% (SD =20.1) of the intervals during the 15- and 30-day follow-up
phases, respectively.
In addition to increased academic engagement by all participants, two interesting trends emerged.
First, participants assigned to a treatment strategy that was primarily antecedent-based displayed higher
rates of academic engagement then those assigned to a treatment strategy that was primarily consequentbased, regardless of function of behavior. Participants receiving a primarily antecedent-based treatment
strategy displayed academic engagement an average of 91.6% (SD =9.3) and 86.7% (SD =19.9) of
the intervals during Treatment phases 1 and 2, respectively, whereas, participants receiving a primarily
consequent-based treatment strategy displayed academic engagement an average of 82.5% (SD =13.3)
and 79.6% (SD =16.9) of the intervals during the two treatment phases. Second, participants whose
disruptive behavior was functionally related to task-avoidance displayed substantially higher levels of
academic engagement when assigned to an antecedent-based treatment strategy, than when assigned to
a consequent-based treatment strategy. Participants assigned to an antecedent-based treatment strategy
whose disruptive behavior was functionally related to task-avoidance displayed academic engagement
an average of 96.9% (SD =4.3) and 96.0% (SD =3.5) during Treatment phases 1 and 2 respectively,
whereas, participants assigned to a consequent-based treatment strategy whose behavior was functionally related to task-avoidance displayed academic engagement an average of 75.9% (SD =13.3) and
68.9% (SD =16.4) during the two treatment phases. Although statistical analyses were not conducted,
visual inspection of the mean percentages of academic engagement (and disruptive behavior) clearly
Functional Assessment-based Interventions
25
demonstrate that antecedent-based treatment strategies were substantially more effective for participants
whose behavior was functionally related to task-avoidance.
When conducting a similar study with second grade students, Restori et al. (in review) found that
antecedent-based treatment strategies were significantly more effective with students whose disruptive
behavior was functionally related to attention seeking. These results indicate that treatment strategies
that are primarily antecedent-based may be more effective for older children whose disruptive behavior
is functionally related to task-avoidance, whereas, antecedent-based treatment approaches may be more
effective for younger children whose disruptive behavior is functionally related to attention-seeking.
Disruptive behavior
Most participants displayed stable patterns of disruptive behavior during the initial baseline
sessions. All participants met the pre-established criteria regarding disruptive behavior for inclusion
into the study. Overall, participants displayed disruptive behavior an average of 59.5% (SD=20.4) of the
intervals during the initial baseline phase and 38.3% (SD=23.0) during the reversal to baseline phase.
Commensurate to the previously reported increases in academic engagement, all participants showed a
significant decrease in disruptive behavior during both treatment phases. Overall, the mean of disruptive behavior decreased to 6.8% (SD=7.6) of the intervals during Treatment phase 1 and 6.5% (SD=8.0)
during Treatment phase 2. Overall disruptive behavior decreased to an average of 15.6% (SD=23.0) and
10.8% (SD=16.0) of the intervals during the 15- and 30-day follow-up phases, respectively.
Similar to academic engagement, all participants demonstrated a significant reduction in disruptive behavior. In addition, the two previously described trends pertaining to academic engagement were
commensurately observed with disruptive behavior. First, participants assigned to a primarily antecedent-based treatment strategy displayed lower levels of disruptive behavior then those assigned to
a treatment strategy that was primarily consequent-based regardless of function of behavior. Participants receiving an antecedent-based treatment strategy displayed disruptive behavior an average of 4.3%
(SD=6.0) and 5.4% (SD=9.9) of the intervals during Treatment phases 1 and 2, respectively, whereas,
participants receiving a consequent-based treatment strategy displayed disruptive behavior an average of
9.1% (SD=8.4) and 7.5% (SD=5.8) of the intervals during the two treatment phases. Second, participants
whose disruptive behavior was functionally related to task-avoidance displayed lower levels of disruptive behavior when assigned to a primarily antecedent-based treatment strategy, than when assigned
to a primarily consequent-based treatment strategy. Participants assigned to an antecedent-based treatment strategy whose disruptive behavior was functionally related to task-avoidance displayed disruptive behavior an average of 0.89% (SD=2.3) and 0.85% (SD=1.2) during Treatment phases 1 and 2
respectively, whereas, participants assigned to a consequent-based treatment strategy whose behavior
was functionally related to task-avoidance displayed disruptive behavior an average of 13.1% (SD=9.7)
and 9.0% (SD=4.6) during both treatment phases.
Discussion
The current study had two primary objectives. First, was to explore the practicality of conducting
functional assessments within general education settings with typically developing children who displayed
excessive rates of disruptive behavior and poor academic engagement as a function of either attention
seeking or task avoidance. If it can be demonstrated that functional assessments can be conducted effectively and with minimal intrusion within a general education classroom setting, school psychologists and
teachers would be better able to meet the growing demand of students experiencing disruptive patterns
26
The California School Psychologist, 2007, Vol. 12
of behavior. Second, was to compare treatment strategies that were primarily antecedent-based with
treatment strategies that were primarily consequent-based. Since teachers often use consequent-based
treatment strategies for addressing the needs of children exhibiting behavioral difficulties, determining
the effectiveness and efficiency of antecedent-based treatment strategies would likely have practical
implications for teachers working with children demonstrating excessive patterns of disruptive behavior
within a general education classroom setting.
Regarding the first objective, teachers and a school psychologist participated in a collaborative approach in conducting an FBA and developing and implementing a behavior plan for each of
the participants. The school psychologist was primarily responsible for gathering data that would be
used to generate a hypothesis statement regarding the function of behavior. As previously stated, this
was accomplished via systematic student observations, teacher interviews, and the SSRS-T. The school
psychologist and teacher then: (a) analyzed the data, (b) reviewed information obtained from the teacher
interview and SSRS-T, (c) generated a hypothesis statement regarding the function of behavior, and (d)
developed a behavior plan. Once the behavior plan was developed, teachers and participants were trained
in intervention procedures. The teachers were primarily responsible for implementing the behavior plans
and completing component analysis checklists. The school psychologist was primarily responsible for
gathering student outcome and treatment integrity data. Results of this consultation-based, collaborative
approach for conducting FBAs within a general education classroom was successful in identifying the
variables maintaining disruptive behavior for students whose behavior was functionally related to either
task-avoidance or attention-seeking.
Regarding the second objective, treatment strategies that were primarily antecedent-based were
more effective than treatment strategies that were primarily consequent-based for reducing disruptive
behavior and increasing academic engagement for all participants. Closer inspection of the data indicate that antecedent-based treatment strategies were particularly more effective than consequent-based
treatment strategies for reducing disruptive behavior and increasing academic engagement when disruptive behavior was functionally related to task avoidance. These results are of particular interest when
compared to a similar study conducted by Restori and colleagues (in review) with second grade students,
wherein antecedent-based treatment strategies were particularly more effective for students whose disruptive behavior was found to be functionally related to attention seeking. Although the conclusions that can
be drawn from these two studies are speculative due to the small sample size of each, the age differences
of the participants in each study may provide the most plausible explanation for the differences found.
That is, an antecedent-based treatment strategy such as self-monitoring may have been more effective
with the younger, second grade students whose disruptive behavior was functionally related to attention
seeking because self-monitoring provided them with sufficient attention from their teacher’s resulting
in the observed increase in academic engagement and reduction in disruptive behavior. Whereas, an
antecedent-based intervention such as task-modification may have been more successful with fifth grade
students whose disruptive behavior was functionally related to task avoidance where the academic
demands were more rigorous because it provided the necessary support, which subsequently produced
the observed reduction in disruptive behavior and increase in academic engagement. Results of both
studies indicate that treatment strategies that were primarily antecedent-based were more effective than
consequent-based treatment strategies for second and fifth grade students whose disruptive behavior
was related to either task-avoidance or attention-seeking. Anecdotally, teachers reported that interventions that were primarily antecedent-based were easier to implement and monitor, that consequent-based
treatment strategies were exhausting, and that students were eager to participate in antecedent-based
Functional Assessment-based Interventions
27
treatment strategies such as self-monitoring and task-modification. This would indicate that antecedentbased treatment strategies were not only as effective as consequent-based treatment strategies, but less
labor intensive and efficient.
Limitations of the Current Study
The current study has several important limitations that must be noted. First, due to the small sample
of participants used in this study, results regarding external validity should be interpreted cautiously. The
issue of external validity was partially addressed by employing two participants (replication) for each of
the four possible functions of behavior and intervention strategy combinations. Second, all of the participants in the study were male, therefore, results of treatment outcomes must also be interpreted cautiously
regarding their application to females. Although the extent to which these results may apply to females
is a limitation of the study, it is important to note that boys are more likely to engage in externalizing
behaviors (i.e., disruptive behavior) at a rate of approximately 4:1 (Walker et al., 2004). Third, statements
regarding the generalizability of the participants’ behavior are limited, since participants were observed
once per day for one 15-minute session and only during a reading or reading related lesson. Although
the daily 15-minute observations provide a “snapshot” of behavior, the generalization of behavior across
settings (e.g., classrooms, playground, home), lessons (e.g., math, social studies), and situations (e.g.,
transitions, independent seatwork, lessons) cannot be assumed. Fourth, functional analyses were not
conducted therefore, the hypothesized function of disruptive behavior was not experimentally validated. Although functional assessments yield convergent sources of information regarding the function
of behavior, a functional analysis is needed to validate the hypothesis regarding function of behavior.
Although functional analyses were not conducted and the function of behavior was not validated, use
of an ABAB single-case design and the success of the interventions for all participants indicate that the
correct function of behavior was identified. Fifth, results regarding function of behavior should be interpreted cautiously since the participants demonstrated disruptive behavior that was functionally related to
either task-avoidance or attention-seeking. It is very likely that some students will engage in disruptive
behavior that serves a dual function, is undifferentiated, or is functionally related to some other variable
(e.g., sensory reinforcement). Sixth, the distinction between teacher and peer attention was not made. It
is probable that students from different age groups will be differentially affected by either teacher and/or
peer attention. Seventh, comparing intervention strategies that are primarily antecedent- or consequentbased may reveal which intervention strategies are better, that is, are more effective and/or easier for
teachers to implement, however, such a comparison may not demonstrate which approach is best.
Implications for Research and Practice
Results and limitations of the current study indicate that a number of issues will need to be addressed
through future research. First, the use of functional assessments in applied settings such as special and
general education classrooms with typically developing children and children either at-risk or identified
as EBD is of utmost importance. Sasso and his colleagues (2001) raised a number of important issues
regarding functional assessment/analysis research that has been conducted in applied settings. Some of
these concerns include the use of functional analysis within applied settings, validating the procedures
and techniques used when conducting functional assessment research in applied settings, and developing
a clear methodology for how each of these techniques and procedures contribute to the development of
a hypothesis regarding function of behavior. Each of these concerns must be addressed if a clear, empirically validated science for using applied behavior analysis methodology within applied settings is to
28
The California School Psychologist, 2007, Vol. 12
grow.
Second, the field of applied behavior analysis has set a “high bar” regarding how behavior assessment and interventions should be researched and applied. In the context of university research settings
where personnel and resources may be more readily available, such standards may be justifiable, attainable, and practical, however, within public school classroom settings, such standards may not be reasonable. Therefore, those who conduct research in applied behavior analysis and those who apply it in “real
world” settings will need to find a common ground from which to build a research base and practice.
Johnston, Foxx, Jacobson, Green, and Mulick (2006) have identified two “movements” that have had a
significant impact within the behaviorist field. One movement consists of the applied behavior analysts
(ABA) who adhere to stringent experimental rigor, high standards with regard to internal validity, and
has a well-documented research base. The second and newer movement is comprised of the positive
behavior support (PBS) researchers and practitioners that apply variations of applied behavior analysis,
usually within school settings. PBS researchers and practitioners are willing to relinquish some of the
high research standards set forth by ABA in the interest of using the behavioral technology with children
in school settings. Scott and his colleagues (2004) maintain that it is unrealistic to expect practitioners
working within public school settings to adhere to the rigorous experimental standards set forth by ABA.
Thus, those who research functional assessment in applied settings must continue to build a research
base to establish a minimum standard for conducting functional assessment research in applied settings
and its use in practice.
Third, future research using functional assessment procedures in applied settings should consider
developing assessment procedures that may be employed across a variety of settings, lessons, and situations to further establish their external validity and generalizability. Functional assessment procedures
should be modified to be less intrusive and restrictive to ensure success within applied settings. For
example, classroom teachers may find a functional assessment observation form to be more practical
than a partial-interval, time-sampling procedure for gathering information and data pertaining to the
function of behavior. Functional assessment observation forms yield important information regarding
the description, frequency, duration, time, and hypothesized function of maladaptive behavior.
Fourth, develop treatment plans that utilize antecedent- and/or consequent-based treatment strategies across a variety of settings, lessons, and situations to further establish their external validity and
generalizability. Treatment packages should be modified to be less intrusive and restrictive to ensure
success within special and general education classrooms. Treatment strategies such as self-monitoring
can be modified to be practical for use within classroom settings. For example, the current study modified the self-monitoring procedure described by Shapiro and Cole (1992) to fit the general education
curriculum. The previously described observation and self-monitoring procedures can be modified and
used across a variety of settings, lessons, and situations within general education settings.
Fifth, future research should classify students whose disruptive behavior is functionally related to
attention-seeking, as seeking either teacher or peer attention. The current study did not make the distinction between peer and teacher attention, however, it is likely that some students prefer peer attention,
others prefer teacher attention, and others are reinforced by both teacher and peer attention. Consideration
of the type of attention-seeking (i.e., peer or teacher attention) among students from different age groups
is likely to have a substantial impact on the selection of both assessment and treatment procedures.
Sixth, future research focusing on treatment strategies that are primarily antecedent-based should
focus on the following four aspects: (a) modifying established antecedent-based treatment strategies
such as self-monitoring and task-modification for use within general education settings, (b) developing
Functional Assessment-based Interventions
29
new and reliable antecedent-based treatment strategies to prevent the occurrence of problem behaviors
within general education settings, (c) providing social validation for the use of antecedent-based treatment strategies within general education settings, and (d) dissemination of antecedent-based treatment
strategies to classroom teachers in order to expand their repertoire for implementing effective classroom
management. As previously stated, antecedent-based treatment strategies should be part of a comprehensive approach for meeting the challenges posed by students with behavioral disorders.
Conclusion
This investigation provided evidence that functional assessments can be used within a general education classroom with typically developing students that demonstrate behavior problems. Results of the
study indicate that all participants demonstrated decreased disruptive behavior and increased academic
engagement. Results of the current study also indicated that antecedent-based treatment strategies were
more effective than the more typically used consequent-based treatment strategies. Finally, results of
the study indicate that school psychologists and teachers may be able meet the academic and behavioral
needs of students in general education classrooms using a collaborative approach.
References
Becker, W., Madsen, C., Arnold, C., & Thomas, D. (1967). The contingent use of teacher attention and praise in
reducing classroom behavior problems. The Journal of Special Education, 1, 55-67.
Clarke, S., Dunlap, G., Foster-Johnson, L., Wilson, D., White, R., & Vera, A. (1995). Improving the conduct of
students with behavioral disorders by incorporating student interests into curricular activities. Behavioral Disorders, 20, 221-237.
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied Behavior Analysis (2nd ed.). Upper Saddle River, NJ:
Pearson Prentice Hall.
Dunlap, G., & Kern, L. (1996). Modifying instructional activities to promote desirable behavior: A conceptual and
practical framework. School Psychology Quarterly, 11, 297-312.
Dunlap, G., Kern, L., dePerczel, M., Clarke, S., Wilson, D., Childs, K. E., White, R., & Falk, G. D. (1993). Functional analysis of classroom variables for students with emotional and behavioral disorders. Behavioral Disorders, 18, 275-291.
Dunlap, G., White, R., Vera, A., Wilson, D., & Panacek, L. (1996). The effects of multi-component, assessmentbased curricular modifications of the classroom behavior of children with emotional and behavioral disorders.
Journal of Behavior Education, 6, 481-500.
Fuchs, D. & Fuchs, L. (1989). Exploring effective and efficient prereferral interventions: A component analysis of
behavioral consultation. School Psychology Review, 18, 260-283.
Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and prereferral intervention. School
Psychology Review, 18, 37-50.
Gresham, F. M., & Elliott, S. N. (1990). Social Skills Rating System. Circle Pines, MN: American Guidance
Service.
Gresham, F. M., Watson, T. S., & Skinner, C. (2001). Functional behavior assessment: Principles, procedures, and
future directions. School Psychology Review, 30, 156-172.
Horner, R. D. (1994). Functional assessment contributions and future directions. Journal of Applied Behavior Analysis, 27, 401-404.
Horner, R. H., & Sugai, G. (2000). School-wide behavior support: An emerging initiative (special issue). Journal of
Positive Behavioral Interventions, 2, 231-232.
Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richmond, G. S. (1982). Toward a functional analysis of
self-injury. Analysis and Intervention in Development Disabilities, 2, 1-20.
Johnston, J. M., Foxx, R. M., Jacobson, J. W., Green, G., & Mulick, J. A. (2006). Positive behavior support and
applied behavior analysis. The Behavior Analyst, 29, 51-74.
Johnston, J. M., & Pennypacker, H. S. (1993). Strategies and tactics of behavioral research (2nd ed.). Hillsdale, NJ:
Lawrence Erlbaum.
30
The California School Psychologist, 2007, Vol. 12
Kamps, D., Wendland, M., & Culpepper, M. (2006). Active teacher participation in functional behavior assessment
for students with emotional and behavioral disorders risks in general education classrooms. Behavioral Disorders, 31, 128-146.
Kern, L., & Clemens, N. H. (2007). Antecedent strategies to promote appropriate classroom behavior. Psychology
in the Schools, 44, 65-75.
Kern, L., Delaney, B., Clarke, S., Dunlap, G., & Childs, K. E. (2001). Improving the classroom behavior of students
with emotional and behavioral disorders using individualized curricular modifications. Journal of Emotional
and Behavioral Disorders, 9, 239-247.
Kern, L., Hilt, A. M., & Gresham, F. M. (2004). An evaluation of the functional behavior assessment process used
with students with or at risk for emotional and behavioral disorders. Education and Treatment of Children, 27,
440-452.
Lane, K. L., Robertson, E. J., & Graham-Bailey, M. A. L. (2006). An examination of school-wide interventions with
primary level efforts conducted in secondary schools: Methodological considerations. In T. E. Scruggs & M. A.
Mastropieri (Eds.), Applications of research methodology: Advances in learning and behavioral disabilities (vol.
19). Oxford, UK: Elsevier.
Lane, K. L., Umbreit, J., & Beebe-Frankenberger, M. (1999). A review of functional assessment research with
students with or at-risk for emotional and behavioral disorders. Journal of Positive Behavioral Interventions, 1,
101-111.
Lane, K. L., Weisenbach, J. L., Little, M. A., Phillips, A., & Wehby, J. (2006). Illustrations of function-based interventions implemented by general education teachers: Building capacity at the school site. Education and Treatment of Children, 29, 549-671.
Lennox, D. B., Miltenberger, R. G., Spengler, P., & Erfanian, N. (1988). Decelerative treatment practices with
persons who have mental retardation: A review of five years of literature. American Journal of Mental Retardation, 92, 492-501.
Lewis, C., & Sugai, G. (1996). Functional assessment of problem behavior: A pilot investigation of the comparative and interactive effects of teacher and peer social attention on students in general education settings. School
Psychology Quarterly, 11, 1-19.
Marcus, B. A., & Vollmer, T. R. (1996). Combining non-contingent reinforcement and differential reinforcement
schedules as treatment for aberrant behavior. Journal of Applied Behavior Analysis, 29, 43-51.
Martens, B. K., Peterson, R. l., Witt, J. C., & Cirone, S. (1986). Teacher perceptions of school-based interventions.
Exceptional Children, 53, 213-223.
Newcomer, L. L., & Lewis, T. J. (2004). Functional behavior assessment: An investigation of behavior reliability and
effectiveness of function-based interventions. Journal of Emotional and Behavioral Disorders, 12, 168-181.
Olweus, D. (1979). Stability of aggressive reaction patterns in males: A review. Psychological Bulletin, 86, 852875.
Restori, A. F., Gresham, F. M., Chang, T., Lee, H. B., Laija-Rodriquez, W. (2007). Functional assessment-based
interventions: Outcomes of antecedent- and consequent-based tactics in general education classrooms. Manuscript submitted for publication.
Sasso, G. M., Conroy, M. A., Stichter, J., & Fox, J. J. (2001). Slowing down the bandwagon: The misapplication
of functional assessment for students with emotional and behavioral disorders. Behavioral Disorders, 26, 282296.
Scott, T. M., Bucalos, A., Liaupsin, C., Nelson, C. M., Jolivette, K., & DeShea, L. (2004). Using functional behavior
assessment in general education settings: Making a case for effectiveness and efficiency. Behavioral Disorders,
29, 189-201.
Shapiro, E. S., & Cole, C. L. (1992). Self-monitoring. In T. H. Ollendick & M. Hersen (Eds.). Handbook of child
and adolescent assessment (2nd ed). pp. 124-139). New York: Pergmon.
Umbreit, J. (1996). Functional analysis of disruptive behavior in an inclusive classroom. Journal of Early Intervention, 20, 18-29.
Umbreit, J., & Blair, K. S. (1997). Using structural analysis to facilitate treatment of aggression and noncompliance
in a young child at-risk for behavioral disorders. Behavioral Disorders, 22, 75-86.
Umbreit, J., Lane, K. L., & Dejud, C. (2004). Improving classroom behavior by modifying task difficulty: Effects of
increasing the difficulty of too-easy tasks. Journal of Positive Behavior Interventions, 6, 13-20.
Walker, H. M., Ramsey, E., & Gresham, F. M. (2004). Antisocial behavior in school: Evidenced-based practices (2nd
ed.). Belmont, CA: Wadsworth/Thomson Learning.
Walker, H. M., & Severson, H. H. (1991). Systematic Screening for Behavior Disorders (SSBD): User’s guide and
technical manual. Longmont, CO: Sopris West.
The California School Psychologist, Vol. 12, pp. 31 –45, 2007
Copyright 2007 California Association of School Psychologists
31
The Use of Response to Intervention Practices for Behavior:
An Examination of the Validity of a Screening Instrument
Paul Muyskens &
Doug Marston
Minneapolis Public Schools
Amy L. Reschly
University of Georgia
Behavioral difficulties of school-aged students are typically dealt with in a reactive, rather than preventative manner. This article examines a proactive approach, consistent with the Response-to-Intervention model, using a screening measure designed to identify students at risk for behavior difficulties
and targeting these students for early intervention. Participants in this study were 22,056 kindergarten
through 8th grade students in the Minneapolis Public Schools. Teachers completed a 12-item behavior
screener for each student in the fall. The scores on this measure were significantly correlated with
suspensions, achievement scores and attendance data over the course of the school year. Potential
implications for behavioral planning and interventions are discussed.
KEYWORDS: Behavior, Behavior Screening, Response to Intervention
In a recent report, the National Academy of Sciences Committee on Minority Representation in
Special Education noted, “There is substantial evidence with regard to both behavior and achievement
that early identification and intervention is more effective than later identification and intervention.” (p.
6, Donovan & Cross, 2002). While the logic of preventing or addressing problems before they become
severe is both obvious and compelling, historically, schools have been reactive, rather than proactive
institutions (Walker et al., 1996). In terms of addressing academic or behavioral concerns, this reactivity
is often referred to as a “wait-to-fail” model. Criticism of this approach is well-documented and there
are increasing calls in the literature for screening and/or early intervention for academic and behavior
problems (e.g., Donovan & Cross, 2002; O’Shaughnessy, Lane, Gresham, & Beebe-Frankenberger,
2003; Walker et al., 1996; Reschly & Ysseldyke, 2002).
Despite the link between student behavior and academic performance (Alexander, Entwisle,
& Dauber, 1993) and the association between behavior and long-term social outcomes for students
(Kaplan, Peck, & Kaplan, 1997), the knowledge base for systematic screening and tracking of emotional
and behavior problems lags behind that of academics, particularly reading (Donovan & Cross, 2002). In
addition to interfering with other students’ learning, problem behaviors can be violent and/or destructive.
For example, in 1999-2000, 29% of principals reported that student bullying occurred daily or weekly,
while 19% of principals reported acts of disrespect to teachers with the same frequency (DeVoe et al.,
2004). Many educators report difficulty managing student behavior on a day-to-day basis and both
parents and the general public are concerned about discipline and safety in our schools. For example,
when Americans were asked about the biggest problems facing education, concerns over school discipline/control were tied for 2nd as the most frequent response (behind school funding and tied with overcrowding; Gallup, 2004); and 1/3 of parents worry about their children’s physical safety at school, a
concern that peaks with the parents of middle school-age children (Gallup, 2002).
Please send correspondence to Paul Muyskens, at Minneapolis Public Schools; 807 NE Broadway; Minneapolis, MN,
USA; 55413-2398 or e-mail [email protected]
32
The California School Psychologist, 2007, Vol. 12
Although there is evidence for the efficacy of proactive, school-wide behavior support models
(e.g., Sugai et al., 2000), schools typically rely on disciplinary action as the primary means of
managing student misbehavior. Zero tolerance policies, the elimination of social promotion, and “get
tough” rules are the current trends. Unfortunately, there is little evidence for the effectiveness of these
approaches for improving school safety or long-term student outcomes (Skiba & Peterson, 1999). In
addition, the result of disciplinary policies based upon clear standards or zero tolerance is typically not
instructional, but rather exclusionary. Students and schools can ill afford exclusionary policies that
result in lost opportunities for instruction, an outcome that is associated with lower student and schoollevel achievement (Skiba & Rausch, 2004) and may be related to increased dropout rates, or students
dropping out at a younger age (Bock, Tapscott, & Savner, 1998; DeRidder, 1990).
Another issue in managing student behavior primarily through disciplinary approaches is that the
application of these practices varies by school characteristics, principal attitudes, and even by teachers
within the same school (Rausch & Skiba, 2004; Skiba & Rausch, 2006). Whenever there is uneven
or inconsistent application of an educational practice or policy, harmful disparities may result. In the
case of discipline, there is evidence that the use of harsh disciplinary practices varies by socioeconomic
status and race, negatively affecting students of lower socioeconomic backgrounds and African-American students. As noted by Skiba and Rausch (2006), “Over thirty years of consistent data concerning
African American over-representation in suspension and expulsion indicates that disciplinary school
exclusion may carry inherent risks for creating or exacerbating racial and socioeconomic disadvantage.” (p. 93).
Concerns over fairness and bias in managing and evaluating student behavior spill over into the
special education arena. On a national level, African-American students are disproportionately represented in special education programs for Emotional Disturbance and Mental Retardation, and AmericanIndian students are disproportionately represented in programs for Learning Disabilities, while Asian
Americans, Hispanics, and Whites are underrepresented (Donovan & Cross, 2002; MacMillan & Reschly,
1998). While the causes of over- and under- representation are complex and the debate controversial,
there is concern about bias in interpretation of students’ behavior and the seemingly haphazard nature of
the referral procedure. The identification and referral process for emotional and behavioral concerns has
been described as, “unsystematic, idiosyncratic, and late in the development of a behavioral problem”
(Donovan & Cross, 2002, p.296), while the President’s Commission on Excellence in Special Education
(U.S. Department of Education, Office of Special Education and Rehabilitative Services, 2002, p. 26)
noted that, “Minority children are much more likely to be placed in the emotional disturbance category
because of behavioral characteristics associated with the cultural context in which a child is raised. A
major factor is the role of teacher referral.”
Minneapolis Public Schools
Problem-Solving/Response to Intervention Model
A Response-to-Intervention (RtI), or problem-solving, approach has been put forth as a potential
alternative to the traditional refer-test-place process for special education (e.g., Gresham, 2002; Grimes,
2002; Marston, 2002; Reschly & Tilly, 1999; Shinn, Good, & Parker, 1999). Problem-solving has been
described as, “…a self-correcting methodology to identify, analyze, and intervene with difficulties at
the individual, group, and system levels” (Reschly, 2005). As described by Fuchs and Fuchs (2006), the
first step in an RtI model is to identify students at-risk for not responding to interventions. This activity
should take place early in the school year, and involves screening all students on a tool that is tied to
Behavioral Screening
33
subsequent performance. An RtI or problem-solving model is a means of counteracting the wait-to-fail
approach because once students are identified as at-risk, their progress is monitored and if progress is
not made interventions are provided. Students’ responses, or outcomes to interventions, are then used to
determine the need for more intensive interventions and services. When personnel and services are organized around intervention, rather than assessment for placement, the time and resources for screening
and early intervention activities become available.
While most recent attention and controversy about RtI concerns the use of this model in identifying
students with learning disabilities (e.g., Bradley, Danielson, & Hallahan, 2002; Naglieri, 2005; Reschly,
2005), large-scale applications (Iowa, Minneapolis Public Schools) use this methodology to address both
academic and behavioral concerns. In addition, there is some evidence that the utilization of a model
like RtI reduces minority overrepresentation in special education (Barbour, 2006), possibly due to the
use of direct (e.g., Curriculum-Based Measurement; structured behavioral observation) rather than indirect measures (e.g., IQ) of student performance.
Minneapolis Public Schools has used a Problem-Solving Model (PSM) as an alternative to traditional
special education eligibility criteria since 1993. The steps in the PSM include defining the problem and
establishing the current level of performance, generating possible solutions to the problem, implementing
the best solution, and reviewing the results of the intervention (see Deno, 2002). As implemented in
Minneapolis, students in need of intervention cycle through this process first within the general education classroom, then with increasing intensity at the building assistance team level and, if necessary, in
the special education evaluation phase. Further information regarding the rationale of the PSM or its
implementation in Minneapolis Public Schools may be found in Marston, Muyskens, Lau, and Canter
(2003) and Lau, Sieler, Muyskens, Canter, VanKueren and Marston (2006). The expansion of the role
of the PSM in the Minneapolis Public Schools was facilitated by a voluntary compliance agreement
with the Office of Civil Rights. The purpose of this agreement was to address the disproportionate representation of some minority groups in special education programs by implementing screening and early
interventions for students having difficulty with academics and/or school behavior. Implementation of
behavioral screening for kindergarten through 8th grades was a part of this agreement.
Although sites across the country vary in their implementation of the PSM, a hallmark of the Minneapolis model is the use of both academic and behavioral screening measures. In this model, students
go from screening directly into a series of activities or queries related to the design of an intervention. More specifically, high scores on the screening measures indicate a need for further investigation,
and school staff is subsequently directed to complete the Classroom Intervention Worksheet for these
students. Subsequent completion of the Classroom Intervention Worksheet prompts teachers to initiate
communications with parents, students, other staff members and a review of records. Further, teachers
are asked to describe current levels of performance, identify student strengths, review health information, and specify potential intervention strategies. This information is used to design a classroom-based
intervention that is implemented by the teacher. Students who do not demonstrate improved behaviors
in response to this intervention may be referred to the problem-solving team. This team implements a
problem-solving process which is used to design a more intensive intervention, along with the collection
of more frequent progress-monitoring data. Students who do not respond to these interventions may
then be referred for special education evaluation (see Marston et al., 2003).
There is a knowledge base for screening students in core academic areas (Donovan & Cross, 2002).
Minneapolis Public Schools utilizes students’ performance on the District-wide achievement test (the
Northwest Achievement Levels Test, or NALT), Curriculum-Based Measurement probes, and informal
34
The California School Psychologist, 2007, Vol. 12
reading inventories. In many cases, scores on these measures are tied, or benchmarked, to passing scores
on the state of Minnesota high-stakes assessment, the Minnesota Comprehensive Assessment – Series
II (MCA-IIs). As mentioned previously, however, the methodology to systematically screen and track
students with behavioral concerns lags behind that which is available in academic areas. To address
the need for behavioral screening in the PSM, staff in the Minneapolis Public Schools developed a new
screening measure, the Behavior Screening Checklist (BSC).
The determination that a new behavioral screening measure was needed was largely done through
a process of elimination. There were several school-related variables that were plausible indicators of
behavioral problems, such as office referral and suspension data, teacher report card ratings, and observations. However, there were problems with using these variables for screening purposes. For example,
office referral and suspension data were typically too reactionary for a screening measure (i.e., data are
not collected until after a problem is severe), and the data are often not compiled systematically. Report
card ratings were quite global in nature (e.g., satisfactory, satisfactory plus) and the use of observations
for screening purposes would have been very time-intensive and expensive. In selecting a measure for
screening, it was important that the data not only have evidence for technical adequacy and validity but
also amenable to quick and systematic collection from all students. Although rating scale reports can be
limited by the context or relationship with the child and/or informant and are generally of little utility
for determining etiology or intervention planning (McConaughy 1993), rating scales are efficient, relatively objective and reliable, allow for a collection of information about a broad range of observations
and teacher interactions, and indicate risk status (Feil, Severson, & Walker, 2002). Finally, behavioral
rating scale data can be used to estimate symptom severity across times, observers, and settings within a
problem-solving model (Busse, 2005).
After a review of the literature and existing instruments, it was determined that existing comprehensive rating systems such as the Behavior Assessment System for Children (BASC) – Second Edition
(Reynolds & Kamphaus, 2004) or Child Behavior Checklist system (Achenbach, 2002) were too long
for our purpose of screening all students. For example, the BASC – Second Edition Teacher Rating
Scale takes about 10 to 15 minutes to complete. It is recommended that teachers complete it on one occasion with minimal distractions (BASC; Reynolds and Kamphaus, 2004). This time commitment does not
include scoring, and thus, would be impractical for use as a class- or school-wide screening instrument.
The BSC was developed as a general purpose measure in consultation with district teachers and
behavioral experts. A key in the development of our screener was weighing the need for brevity while
still tapping a range of important classroom behaviors with adequate psychometric properties. The
primary purpose of the BSC is to help teachers identify students early in the year who have behavioral
issues who may benefit from general education interventions within the PSM. In this study, we sought
to examine the technical adequacy of this measure in terms of, a) the inter-rater and internal consistency
reliability and b) the predictive validity of the BSC with subsequent behavioral and academic difficulties. Measures of achievement are included in our examination of validity because of the interconnected
nature of student behavior and academic performance, and because of the paucity of objective measures
of behavior.
Method
Participants
The subjects of this study included 22,056 kindergarten through grade 8 students in the Minneapolis Public Schools (MPS), a large urban district located in Minnesota. Of these students 49.9% were
Behavioral Screening
35
female and 51.1% were male. The racial/ethnic distribution of the sample was: 40% African American,
28% White American, 16.6% Hispanic American, 11.8% Asian American, and 3.6% Native American.
Approximately 68% of the sample received free or reduced lunch services.
Measures
Behavior Screening Checklist. When first implemented in the District during the 1998-99 academic
year, the BSC was comprised of 10 items; items regarding property destruction and out of place behavior
were added during the first year based on teacher and staff feedback. The 12 items included in the
BSC were grouped into 3 categories: Classroom Behaviors, Externalizing Behaviors, and Socialization
(See Appendix A). The Classroom Behaviors section included four items: evaluating student attention,
following directions, completing work, and class involvement. The Externalizing Behaviors section
included: physical behavior toward others, verbal behavior toward others, physical behavior toward
property, and out of place. The last section of the BSC, Socialization, also had four items: adult interactions, peer interactions, coping with change, and projected self-image. On each item staff was asked to
rate the student on a scale of 1 to 5, with 1 representing the most appropriate behavior related to that area
and a 5 associated with significant concerns. When introducing the BSC to teachers it was emphasized
that this tool is a screening measure, and should be thought of as a prompt to think of all students using
the same questions. If questions arose about a particular rating, teachers were encouraged to go with
their first impression.
Thirty-six was used as the BSC cut-score for those students in need of additional classroom interventions. This score was initially chosen as the cut score because it was the point in the distribution
at which approximately 5% of the student population fell at or above this score. This is similar to the
level of students considered to be in need of tertiary prevention or intense individualized interventions in
some systems of Positive Behavior Support (Sugai & Horner, 2002) or Response-to-Intervention Models
(Batsche, 2005). It was also thought that this targeted a manageable number of students for additional
intervention. For these students teachers were directed to begin interventions related to the problem of
concern within their classrooms, and to determine their responses to these interventions.
Northwest Achievement Levels Test. The Northwest Achievement Levels Test, or NALT, is a standardized multiple-choice test that was administered to all students in the district in grades 2 through
7, and 9 in the areas of reading and mathematics. From the item bank, district personnel chose items
that corresponded to the district’s curriculum standards, allowing for measurement of students’ knowledge and skills with respect to these standards. The NALT is a levels-test with national growth norms
and, therefore, student performance may be linked from level-to-level and grade-to-grade. Further, it is
possible to measure growth over time at the student, classroom, school, and district levels. The norms
are based on a sample of over 1 million students. A student’s performance is based on two factors: the
number of correct answers and the difficulty of the questions. Each student’s normal curve equivalent
(NCE) score was used in the analysis. NCE scores for reading ranged from 1 to 99 with a mean of 48.7
and a standard deviation of 21.8. For math the NCE scores ranged from 1 to 99 with a mean of 54.6 and
a standard deviation of 22.0.
Student behavior: Suspensions and attendance. The total number of suspensions from school during
the entire academic year was used as one criterion measure of student behavior. Suspension scores for
students in our sample ranged from 0 to 21 with a mean of 0.35 and a standard deviation of 1.24. Student
attendance was calculated by dividing the number of days absent from school by the total number of
Further information may be found at http://www.nwea.org/
36
The California School Psychologist, 2007, Vol. 12
days enrolled (days present and absent). The percentage of days absent in the sample ranged from 0%
to 77.4% with a mean of 4.9% and a standard deviation of 5.8%.
Procedures
BSC data were collected in the fall. Teachers completed the BSC for all students in their classrooms by October 1st of the academic year. Data were collected district-wide as part of the PSM process
described in the introduction. Teachers were asked to complete screeners for all students, typically all
at one time. While they were encouraged to go with their first impression, expanded scoring definitions
were available. Other criterion measures were collected from District records.
Results
Reliability Analyses
Inter-rater reliability. Inter-rater reliability and agreement were used to examine this form of reliability. At the time the BSC was first implemented in the District, a collaborative teaching project was
in place in which two teachers were assigned to every third grade classroom. These teachers shared
instructional responsibilities for their classes, which presented an optimal situation for checking interrater agreement of the behavior screener. Research staff recruited six pairs of collaborative teachers to
complete a behavior screener with entire classes of students (2 raters for 143 students). These analyses
were run with the 10-item version of the BSC. An inter-rater reliability coefficient was calculated for
each pair as well as the percent agreement for those below and above the district cut-score identifying
students in need of additional classroom interventions.
Inter-rater reliability coefficients were calculated for six pairs of 3rd grade teachers who taught 143
students in six classrooms. Sample sizes for those ratings ranged from 17 to 28 students per classroom.
All coefficients were significant at or below .01. Coefficients ranged from .659 to .965, with a mean
rating of .825 (see Table 1). Inter-rater agreement between teachers for those students below and at or
above the cut-score was 91%.
TABLE 1. Inter-rater reliability coefficients for six pairs of teachers
Teacher Group
Number of Students
Pair 1
25
Pair 2
25
Pair 3
27
Pair 4
17
Pair 5
28
Pair 6
21
Average Correlation
Reliability Coefficient
.865
.965
.847
.659
.819
.796
.825
Internal Consistency Reliability
Chronbach’s alpha was used to examine the internal consistency reliability of the BSC. Chronbach’s
alpha is reported for the original 10-item version of the BSC (N = 143) and the current 12-item BSC.
Thirty-six is the cut score used with the 12-item BSC. A cut score for the 10-item BSC that corresponds with
the same portion of the distribution was 31.
Behavioral Screening
37
The distribution of the sample used in analysis of the 12-item BSC by grade may be found in Table 2.
The alpha coefficient for the original 10-item BSC was .93. On the more recent 12-item version alpha
coefficients ranged from .92 to .95 for grades kindergarten to grade 8 (kindergarten: .93; 1st grade: .92;
2nd grade: .93; 3rd grade; .93, 4th grade; .93, 5th grade: .93; 6th grade: .94; 7th grade: .95; and 8th grade:
.94).
Descriptive Analyses
Descriptive data for the entire sample of students are presented in Tables 2 and 3. Scores on the
BSC ranged from 12 (which represents a rating of “1” on each item) to 60 (a rating of “5” on each item).
The mean total score on the Behavior Screener Checklist was 20.37 with a standard deviation of 8.40.
The distribution of scores was positively skewed, with most students earning scores in the low-end of
the scale, indicating few behavioral concerns (See Table 2). Approximately 74% of the students scored
at or below 24 on the BSC, while 5.2% scored above 36, the cut-off score for initiation of classroom
interventions.
TABLE 2. BSC Descriptive Statistics by Grade
Grade
Mean
N
Std. Deviation
Minimum
Maximum
Kdg
1
2
3
4
5
6
7
8
Total
19.68
19.29
19.84
19.53
20.17
19.43
21.87
22.33
21.82
20.37
2607
2623
2710
2431
2443
2521
2192
2197
2332
22056
7.69
7.55
7.83
8.01
8.01
7.62
9.01
9.95
9.37
8.40
12
12
12
12
12
12
12
12
12
12
60
56
57
59
59
57
56
60
60
60
Note: Distribution Characteristics of the total Sample Skewness = 1.297 (SE = .016); Kurtosis = 1.711 (SE = .033)
The means, standard deviations and sample sizes at each grade level are presented in Table 2. It is
apparent that there is a gradual increase in scores on the BSC as students grow older, with the highest
scores recorded for students in the middle school years. Behavior screening scores also vary by gender
and ethnicity (Table 3). Male students scored higher than female students. BSC scores are somewhat
higher for African-American and Native-American students. Relatively lower scores were obtained for
Asian-American and White-American students.
Predictive validity analyses
Predictive criterion validity of the BSC was studied by using separate correlation analyses for
students in K to 5 (N = 14,335) and grades 6 to 8 (N = 6,721), corresponding with the change in school
level from elementary to middle school. It was anticipated that BSC scores would vary as a function of
school level, with more reported difficulties for older students. Fall BSC data were correlated with criterion measures (NALT, suspensions, attendance) collected in the spring of the same academic year.
38
The California School Psychologist, 2007, Vol. 12
TABLE 3. BSC Descriptive Statistics by Grade
Variable
Mean
N
SD
Minimum
Maximum
Gender
Female
18.71
10995
7.32
12
Male
22.02
11061
9.06
12
Total
20.37
22056
8.40
12
60
60
60
Ethnicity
Native American
22.58
783
8.82
12
African American
23.45
8828
9.29
12
Asian American
17.37
2609
6.29
12
Hispanic
19.06
3664
7.15
12
White
17.73
6172
6.77
12
Total
20.37
22056
8.40
12
59
60
60
58
59
60
TABLE 4. Correlations between BCS and criterion variables (K-5 sample)
Number of
Suspensions
Number of
Absences
NALT Reading
NCE
Behavior Screener
0.28*
0.19*
-0.39*
N
15335
15335
6135
Number of Suspensions
0.18*
-0.20*
N
15335
6135
Number of Absences
-0.13*
N
6135
NALT Reading NCE
N
NALT Math
NCE
-0.42*
6117
-0.19*
6117
-0.18*
6117
0.80*
6092
*p < .001
TABLE 5. Correlation between BSC and Criterion Variables (6-8 sample)
Number of
Suspensions
Number of
Absences
NALT Reading
NCE
Behavior Screener
0.51*
0.36*
-0.45*
N
6721
6721
3657
Number of Suspensions
0.46*
-0.30*
N
6721
3657
Number of Absences
-0.20*
N
3657
NALT Reading NCE
N
*p < .001
NALT Math
NCE
-0.48*
3357
-0.34*
3357
-0.26*
3357
0.84*
3320
Behavioral Screening
39
Because of the skewed distribution of BSC scores and the ordinal nature of the scale, a Spearman’s rho correlation was used for the measure of predictive validity for both the K-5 and 6-8 samples.
The total score on the BSC administered in the fall was correlated with the end of the year number of
suspensions, percentage of days absent from school, and district achievement scores in reading and
math from spring test administration. For both groups the correlations between all of these variables
were significant at the .01 level (Tables 4 and 5). Correlations between the BSC and criterion variables
were higher in all cases for the grade 6-8 group. There were moderate correlations between fall BSC
scores and the number of suspensions and reading and math achievement in the spring. The correlation
between the BSC and suspensions was higher for the grade 6-8 group (.28 and .51, respectively, K-5
and grades 6-8). Correlations between the achievement measure and the BSC were also better for the
grade 6-8 group (an average correlation of -.40 for K to grade 5; -.46 for the upper grade level group).
A visual analysis of the predictive nature of the BSC was also conducted. A cross-tabs procedure
was used to display scores on the BSC with the number of suspensions, achievement scores in reading
and math, and total days absent from school. This analysis, presented in Figure 1, was conducted
because we were interested in examining how higher scores on the fall BSC interacted with the related
Figure 1. Percentage of students suspended, median reading and math NALT NCE scores, and average
percentage of days absent for groups of students scoring at or above selected BSC total scores.*
70
60
50
% Suspended
40
% Days Absent
30
Mean Reading
NALT NCE
20
10
GT=52
GT=48
GT=44
GT=40
GT=36
GT=32
GT=28
GT=24
GT=20
GT=16
0
Mean Math
NALT NCE
GT=12
Percentile or Standard Score
80
BSC3 Rating
* Values on Y-axis represent three different scales associated with suspensions, achievement, and attendance.
40
The California School Psychologist, 2007, Vol. 12
variables. Note that the vertical axis in Figure 1, which ranges from 0 to 80, represents three scales:
percentage of students suspended; mean NCE scores in reading and math for each BSC group, and
average percentage of days absent from each BSC group. There appears to be a linear relationship
between fall BSC scores and the likelihood of being suspended during the year. While only 13.6%
of the total sample was suspended, 60.3% of those students scoring 36 or above on the fall behavior
screener were suspended.
The relationship between the BSC scores and achievement scores appears to be greater at the lower
end of the scale, but continues in an inverse manner as the behavior scores rise. The student group as a
whole achieved over 6 NCE’s higher in reading and math than those scoring 16 or above on the BSC,
and 17 to 19 NCE’s higher in reading and math than those scoring 36 or above on the BSC. This difference can be largely accounted for by the average NALT NCE score for reading being above 60 for those
students scoring 12 on the BSC (mean score = 61.0 for reading and 67.6 for math (N=2,007 and 1,949,
respectively). The mean score for those students scoring 16 on the BSC was 51.1 for reading (N=449)
and 57.7 for math (N=485). The relationship between the fall BSC scores and percentage of days absent
from school was not strong. There is a gradual increase in percentage of absences as the BSC scores rise,
before a plateau around a score of 48.
Discussion
Student behavior figures prominently in our schools, with links to academic performance and longterm social outcomes for students. The most frequent or common approach to managing student behavior
is reactionary, rather than proactive, in nature. The use of harsh disciplinary practices – zero tolerance
policies – has increased, despite the absence of evidence for the effectiveness of these approaches for
improving student behavior or school climate. Indeed, these practices may negatively affect student
achievement and school completion, and there is evidence of socioeconomic and racial disparities in the
disciplinary management of students in our nations’ schools. The logic of preventing or intervening early
with problems, whether academic or behavioral, before these problems become severe and sometimes
intractable, is obvious; however, this requires a major shift in thinking and practice. The knowledge base
for screening and tracking emotional and behavioral problems is not well-established. The focus of this
study was to examine a classroom behavior screener, the BSC, developed for use in the PSM utilized by
the Minneapolis Public Schools. The purpose of the BSC was to identify students with behavior needs
to inform general education interventions as part of the PSM.
One of the first questions in the development and use of a new measure relates to the technical
adequacy. The inter-rater reliability coefficients examined in this study were technically adequate, as
was the inter-rater agreement for teacher ratings of those below and at or above the cut score signaling
the need for additional intervention. It should also be noted these coefficients tended to be higher than
those often obtained for cross-informant ratings of student behavior. In a meta-analysis of 119 studies,
Achenbach, McConaughy, and Howell reported that the average correlation between respondents of
similar roles, such as 2 teachers, was .60, while the correlation among respondents of different roles
was considerably lower (reported in McConaughy, 1993b). The internal consistency coefficients were
extremely high and also documented technical adequacy in the area of reliability.
There are concerns over bias and disparity in both disciplinary actions and special education placements. The pattern of scores on the BSC is similar to that found in other research: African American,
Native American, middle school, and male students received higher scores on the BSC. Screening with
an instrument like the BSC will not address concerns over misinterpretation of student behavior due to
Behavioral Screening
41
cultural or racial differences; however, screening with a focus on early intervention is a step in the right
direction and may help attenuate some of the poor outcomes associated with harsh disciplinary policies
(e.g., loss of instructional time, dropout, special education placements). Donovan and Cross (2002)
argued the overall impact of early intervention models will be to reduce the numbers of students who
fail, which in turn meets the educational needs of students of color.
BSC predictive validity was evidenced by moderate correlations between fall ratings and behavioral
performance over the course of the school year and academic assessments administered in the spring.
In addition, the results showed a clear and consistent relationship between scores on the BSC and the
likelihood of suspension. Those students scoring 36 or above in the fall have a 50% chance of being
suspended over the course of the school year.
This study provided some evidence to support the use of the BSC as an efficient, reliable, and
valid screening tool to aid teachers and other school staff in identifying students likely to encounter
significant behavioral difficulties, particularly those at-risk for suspensions. As noted by Gersten and
Dimino (2006), one of the failures of the prereferral model of the late 1980’s was that, “Student Study
Teams typically made intervention recommendations on the basis of the classroom teacher’s description
of the student’s academic or behavioral performance. Many of the descriptions were anecdotal; few
were data based.” (Gersten & Dimino, 2006, p. 101). The ability to use the BSC to identify students in
need of interventions may help teachers to examine their students’ behaviors earlier in the school year,
and provides a measure which prompts teachers to rate all students’ behaviors on objective items. This
should also lessen the bias introduced by teacher nomination, and will hopefully provide for intervention
before a cycle of frustration or failure is created.
In practical terms, the provision of teachers with a tool which in less than a minute can help them
identify students likely to present significant behavioral or academic concerns over the course of the
school year seems worthy. On average those students scoring at or above 36 on the BSC were over 4
times as likely to be suspended as the total sample, were likely to be absent almost twice as much, and
scored around one-half standard deviation lower on measures of achievement. This knowledge can be
used to target students, classrooms or programs for intervention programs before these problems arise.
There is a tension between maintaining scientific rigor while conducting research in real-world
settings. This study is no exception. The strengths of this study – the design and implementation of
a screening measure for use in a PSM and subsequent research with a large, diverse school population
– are also weaknesses. There was little time between development and implementation of the BSC as a
district-wide K-8 screener; analyses of its adequacy were post-hoc. Future study should address refining
the BSC, including examination of individual items, the need for and utility of age-specific versions, and
the establishment of norms and cut-scores for screening purposes. In addition, validity studies relating
the BSC to other forms of behavioral assessment are needed as further evidence of its validity. Furthermore, although behavior rating scales in general are ideal for screening purposes, due to their relative
efficiency, objectivity, and reliability (Feil, Severson, & Walker, 2002), these scales do not provide information regarding etiology or intervention design (McCanaughy, 1993). This is true of the BSC as well.
The purpose of this screener was to identify students in need of early intervention; it does not provide
information to teachers on how to intervene nor is it a sensitive measure of how a specific behavior
changes due to the intervention. Future research must address the critical link between screening and
intervention planning and implementation within the PSM.
There are a range of demands on the educational system, including increased accountability, No
Child Left Behind, budgeting concerns, and discipline. In the midst of these demands, however, it is
42
The California School Psychologist, 2007, Vol. 12
clear that the traditional wait-to-fail model used for academic and behavioral difficulties does not meet
the needs of our students or the system as a whole. While wait-to-fail is the past and present, screening
and early interventions are the future. In their call for a “paradigm shift,” Reschly and Ysseldyke (2002)
argue that data-based problem solving is crucial to system reform. A response-to-intervention model
promotes the early identification of students’ academic and behavioral difficulties and provides interventions and monitoring with increasing intensity until effective interventions and environments are
found for students. Although further research is needed, the behavior screener described is a model for
addressing significant behavioral issues in a proactive way that is linked to problem solving and better
outcomes for at risk children and youth.
Appendix A
Behavior Screening Checklist III
Student Name:
Student ID#:
Date:
Rate the student on the following continuum:
I. Area: Classroom Behaviors
Attention:
1
2
Consistently attends to
classroom activities
3
4
Sometimes follows along
with classroom activities
5
Rarely follows along
with classroom activities
Follows Directions:
1
2
Consistently follows rules
3
4
Sometimes follows rules
5
Rarely follows rules
Completing Work:
1
2
Consistently completes
work independently
3
4
Sometimes completes
work independently
5
Rarely completes
work independently
Class Involvement:
1
2
Participates Well
3
4
Sometimes participates
5
Rarely participates
II. Area: Externalizing Behaviors
Physical Behavior Toward Others:
1
2
3
4
Physically appropriate
Occasionally physically appropriate
5
Rarely physically
appropriate
Verbal Behavior:
1
2
3
4
Uses appropriate Sometimes uses appropriate
verbal behavior
verbal behavior
5
Rarely uses appropriate
verbal behavior
Behavioral Screening
43
Physical Behavior Toward Materials or Property:
1
2
3
4
Is consistently respectful
Is sometimes respectful
of Materials or Property
of Materials or Property
Out of Place:
1
2
Remains in assigned area
5
Is rarely respectful
of Materials or Property
3
4
Sometimes remains
in assigned area
5
Rarely is in
assigned area
Coping with Change:
1
2
Handles change
appropriately
3
4
Occasionally handles change appropriately
5
Rarely handles
change appropriately
Adult Interactions:
1
2
Seeks positive
relationships
3
4
Sometimes seeks
positive relationships
5
Rarely seeks
positive relationships
Peer Interactions:
1
2
Seeks positive
relationships
3
4
Sometimes seeks
positive relationships
5
Rarely seeks
positive relationships
III. Area: Socialization
Scores: Classroom Behaviors
Externalizing Behaviors
Socialization
© Special School District #1, 2007 – Minneapolis Public Schools
Total
9/07
References
Achenbach, T. M. (2002). Achenbach System of Empirically Based Assessment. Retrieved June 13, 2005 from
ASEBA website, www.aseba.org/index/html.
Alexander, K. L., Entwisle, D. R., & Dauber, S. L. (1993). First-grade classroom behavior: Its short-and long-term
consequences for school performance. Child Development, 64(3), 801-813.
Barbour, C. B. (2006). Engaging RTI: Notes from the field. Presentation at the University of Cincinnati, May 12,
2006.
Batsche, G. M. (2005, October 4). Problem-Solving and Response to Intervention: Maximizing Opportunities for
Student Progression. Paper presented at the LaGrange Area Department of Special Education. Retrieved May
30, 2006, from http://www.ladse.org/prof%20dev.htm
Bock, S. J., Tapscott, K. E., & Savner, J. L. (1998). Suspension and expulsion: Effective management for students?
Intervention in School and Clinic, 34(1), 50-52.
Bradley, R., Danielson, L., & Hallahan, D. P. (2002). (Eds). Identification of learning disabilities: Research to practice. Mahwah, New Jersey: Lawrence Erlbaum
Busse, R. T. (2005). Rating Scale Applications within the Problem-Solving Model. In R. Brown-Chidsey (Ed.) Assessment for Intervention: A Problem-Solving Approach (pp. 200-218). New York: Guilford.
Casat, C. D., Norton, & Whitsel (1999). Identification of elementary school children at risk for disruptive behavioral disturbance: Validation of a combined screening method. Journal of the American Academy of Child and
Adolescent Psychiatry, 38, 1246-1253.
Deno, S. L. (2002). Problem solving as “Best Practice.” In A. Thomas & J. Grimes (Eds.), Best Practices in School
Psychology IV (pp.37-55). Bethesda, MD: NASP Publications.
44
The California School Psychologist, 2007, Vol. 12
DeRidder, L. M. (1990). The impact of school suspensions and expulsions on dropping out Educational Horizons,
68, 153-157.
DeVoe, J. F., Peter, K., Kaufman, P., Miller, A., Noonan, M., Snyder, T. D., & Baum, K. (2004). Indicators of School
Crime and Safety: 2004 (NCES 2005-002/NCJ 205290). U.S. Department of Education and Justice. Washington, DC: U.S. Government Printing Office.
Donovan, M. S., & Cross, C. T. (Eds.). (2002). Minority students in special and gifted education. Washington DC:
National Academy Press.
Feil, E. G., Severson, H. H., & Walker, H. M. (2002). Early screening and intervention to prevent the development of aggressive, destructive behavior patterns among at-risk children. In M. Shinn, H. Walker, & G. Stoner
(Eds.), Interventions for Academic and Behavior Problems II: Preventive and Remedial Approaches (pp 143166. Bethesda, MD: NASP Publications.
Fuchs, D., & Fuchs, L. S. (2006). Introduction to Response to Intervention: What, why, and how valid is it?
Reading Research Quarterly 41(1), 93-99.
Gallup Poll News Service (September 17, 2002). Parents concerned about school safety. Available on-line: www.
gallup.com
Gallup Poll News Service (October 26, 2004). Americans on local schools: “Show us the money”. Available on-line:
www.gallup.com
Gersten, R., & Dimino, J. A. (2006). RTI (Response to Intervention): Rethinking special education for students with
reading difficulties (yet again). Reading Research Quarterly 41(1), 99-108.
Ikeda, M. J., Grimes, J., Tilly, W. D., Allison, R., Kurns, S., & Stumme, J. (2002). Implementing an interventionbased approach to service delivery: A case example. In M. Shinn, H. Walker, & G. Stoner (Eds.), Interventions
for Academic and Behavior Problems II: Preventive and Remedial Approaches (pp 53-69. Bethesda, MD:
NASP Publications.
Kaplan, D. S., Peck, B. M., & Kaplan, H. (1997). Decomposing the academic failure-dropout relationship: A longitudinal analysis. Journal of Educational Research, 90(9), 331-343.
Lau, M. Y., Sieler, J. D., Muyskens, P., Canter, A., VanKeuren, B., & Marston, D. (2006). Perspectives on the use of
the problem-solving model from the viewpoint of a school psychologist, administrator, and teacher from a large
Midwest urban school district. Psychology in the Schools, 43(1), 117-127.
Marston, D., Muyskens, P, Lau, M, & Canter, A. (2003). Problem-solving model for decision making with highincidence disabilities: The Minneapolis experience. Learning Disabilities Research and Practice, 18(3), 187200.
Marston, D. (2002). A functional and intervention-based approach to establishing discrepancy for students with
learning disabilities. In R. Bradley, D. P. Hallahan, & L. Danielson (Eds.), Identification of learning disabilities:
Research to practice. NJ: Lawrence Erlbaum Associates, Inc..
McClelland, M. M., Morrison, F. J., & Holmes, D. (2000). Children at risk for early academic problems: The role
of learning-related social skills. Early Childhood Research Quarterly, 15(3), 307-329.
McConoughy, S. H. (1993a). Evaluating behavioral and emotional disorders with the CBLC, TRF, and YSR crossinformant scales. Journal of Emotional and Behavioral Disorders, 1, 40-52.
McConaughy, S. H. (1993b). Advances in empirically based assessment of children’s behavioral and emotional
problems. School Psychology Review, 22(2), 285-307.
McEvoy, A., & Welker, R, (2000). Antisocial behavior, academic failure, and school climate: A critical review.
Journal of Emotional and Behavioral Disorders, 8, 130-140.
Naglieri, J. (2005). “Current perspectives on behavioral interventions” lacks balance. NASP Communique, 33(5).
O’Neil, R., Welsh, M., Parke, R. D., Wang, S., & Strand, C. (1997). A longitude assessment of the academic correlates of early peer acceptance and rejection. Journal of Clinical Child Psychology, 26, 290-303.
O’Shaughnessy, T. E., Lane, K. L., Gresham, F. M., & Beebe-Frankenberger, M. E. (2003). Children placed at-risk
for learning and behavioral difficulties: Implementing a school-wide system of early identification and intervention. Remedial and Special Education, 24(1), 27-35.
Rausch, M. K. & Skiba, R. (2004). Unplanned outcomes: Suspensions and expulsions in Indiana. Education Policy
Briefs, 2(2). Available On-line: http://ceep.indiana.edu/ChildrenLeftBehind/
Reschly, D. J. (2005). Viewpoint. Child outcomes and school psychology service delivery: Response to Naglieri.
NASP Communique, 33(5).
Reschly, D. J., & Tilly, W. D. (1999). Reform trends and system design alternatives. In D. J. Reschly, W. D. Tilly,
& J. P. Grimes (Eds). Special Education in Transition: Functional assessment and noncategorial programming.
Longmont, CO: Sopris West.
Behavioral Screening
45
Reschly, D. & Ysseldyke, J. E. (2002). Paradigm shift: The past is not the future. In A. Thomas & J. Grimes (Eds.),
Best practices in School psychology IV (pp 3-20). Bethesda, MD: National Association of School Psychologists.
Reynolds, C. R., & Kamphaus, R. W. (2004). Manual: Behavior Assessment System for Children. Circle Pines, MN:
American Guidance Service.
Skiba, R. & Peterson, R. (1999). The dark side of zero tolerance: Can punishment lead to safe schools? Phi Delta
Kappan, 80(5), 372-382.
Skiba, R., & Rausch, M. K. (2006). School disciplinary systems: Alternatives to suspension and expulsion. In G.
Bear & K. Minke (Eds). Children’s Needs III: Understanding and addressing the developmental needs of children (pp. 87-102). National Association of School Psychologists.
Sugai, G, & Horner, R. H. (2002). Introduction to the special series on Positive Behavior Support in Schools.
Journal of Emotional and Behavioral Disorders, 10, 130-135.
Sugai, G., Horner, R. H., Dunlap, G., Hieneman, M., Lewis, T. J., Nelson, C. M., Scott, T., Liaupsin, C., Sailor, W.,
Turnbull, A. P., Turnbull, H. R. III, Wickham, D., Reuf, M., & Wilcox, B. (2000). Applying positive behavioral
support and functional behavioral assessment in the schools. Journal of positive behavioral interventions, 2,
131-143.
U.S. Department of Education, Office of Special Education and Rehabilitative Services (2002). A new era: Revitalizing special education for children and their families. Washington DC, U.S. Department of Education.
Walker, H. M., Horner, R. B., Sugai, G., Bullis, M., Sprague, J.R., Bricker, D., & Kaufman, M. J. (1996). Integrated approaches to preventing antisocial behavior patterns among school-age children and youth. Journal of
Emotional & Behavioral Disorders, 4(4), 194- 210.
Woodward, L. J., & Fergusson, D. M. (2000). Childhood peer relationship problems and later risks of educational
under-achievement and unemployment. Journal of Child Psychology and Psychiatry and Allied Disciplines,
41(2), 191-201.
The California School Psychologist, Vol. 12, pp. 47 – 58, 2007
Copyright 2007 California Association of School Psychologists
47
Comprehensive Assessment of Emotional Disturbance:
A Cross-Validation Approach
Emily S. Fisher,
Katie E. Doyon and
Enrique Saldaña
Loyola Marymount University
Megan Redding Allen
University of California, Santa Barbara
Assessing a student for emotional disturbance is a serious and complex task given the stigma of the
label and the ambiguities of the federal definition. One way that school psychologists can be more
confident in their assessment results is to cross validate data from different sources using the RIOT
approach (Review, Interview, Observe, Test). Because each data collection process has advantages
and limitations, using all four processes together allows for comprehensive assessment for emotional
disturbance. Additionally, school psychologists should strongly consider a student’s strengths, cultural
factors, and the interaction between the student and the environment in order to interpret assessment
findings. This approach serves to tailor interventions regardless of diagnosis.
Assessing a student for emotional disturbance (ED) is a complex task both because of the ambiguities of the diagnosis in the educational code and because of the seriousness of assigning this classification
to a student. While it is tempting to want to use standardized assessments to make a definitive diagnosis
of ED, other forms of data collection are equally important as they allow for the cross-validation of information from various sources. Leung (1993) first wrote about a method of comprehensive assessment
using the acronym of RIOT (Review, Interview, Observe, Test). Leung acknowledged that each technique has flaws and advocated using information from all four data sources to support conclusions about
diagnoses. By incorporating information from the cumulative folder, interviews with parents, teachers,
and the student, and observations in the classroom and alternative settings along with data from instruments and tests, school psychologists can better justify their conclusions and present information in a
truly comprehensive manner that allows parents and school personnel to have confidence in the results.
At the beginning of an ED assessment, most often, school personnel have already identified social,
emotional, and/or behavioral problems as a primary concern and the reason for referral is to determine
the extent to which such problems are contributing to the student’s overall school functioning. While the
school psychologist will collect copious amounts of data about the student’s functioning over the course
of the assessment, it is imperative that the school psychologist also consider the ecological context in
which the student’s behaviors occur (Wright, Gurman, & The California Association of School Psychologists/Diagnostic Center, Southern California Positive Intervention Task Force, 2001). This allows the
school psychologist to understand the reciprocal relationship between the student and the environment
(Landau & Swerdlik, 2005) and to examine whether adequate interventions were implemented during
the pre-referral process. Additionally, by conceptualizing the problems from an ecological perspective,
school psychologists are better prepared to make recommendations for interventions at the conclusion of
the assessment, regardless of the ultimate diagnosis.
Please send correspondence to Emily S. Fisher, PhD, School of Education, University Hall, Suite 1500, Loyola Marymount University, 1 LMU Drive, Los Angeles, California, 90045. Email: [email protected]
48
The California School Psychologist, 2007, Vol. 12
Review of Records
There is virtually no recent research literature on reviewing cumulative records; however, the diagnosis of emotional and behavioral problems should begin with an understanding of the student’s prior
school experience. A review of records provides the school psychologist guidance about what information needs to be gathered from other assessment procedures and about interventions that have been
attempted to help the student be more successful in school. For example, if the school psychologist finds
major changes in school functioning of a 10th grade student occurred during 7th grade, he or she will want
to elicit perspectives on these changes during the parent and student interviews. Additionally, it would
be helpful for the school psychologist to speak with the student’s 7th grade teacher for more information
and to find out what interventions, if any, were implemented during that year. Similarly, a review of
records can inform the school psychologist about questions to ask during interviews, when and where to
observe the student, and which tests and instruments might be most appropriate.
In the case of an ED assessment, a record review is crucial to determine if the student has had
emotional, social, and/or behavioral problems “over a long period of time” as required by the federal
definition of emotional disturbance (Friend, 2008, p. 203). Because interviews rely on retrospective
reports of the onset of problems, the student’s cumulative records provide a less biased report of the
student’s prior functioning in school. Report card comments can often provide some evidence of social/
emotional functioning over time.
In addition to onset, there is other key information to attend to in the cumulative record, and it is
as important to note a lack of findings (i.e., lack of evidence of ED) as it is to note significant findings
(i.e., evidence of ED). In addition to examining grades and test scores, school psychologists should look
for any early warning signs of emotional problems such as difficult transitions to school and between
schools, teacher comments on social skills, discipline records, frequent visits to the nurse’s office, counseling referrals, and abnormalities in attendance (e.g., excessive absences or tardiness). It is also important to note non-normative transitions such as changing schools mid-year. A review of records should
also document attempts at intervention and the outcomes of such interventions. Most often, there will
have been some type of pre-referral team meeting to discuss the student and develop a plan for action,
including interventions. Rather than just documenting the interventions from the pre-referral meeting
papers, the school psychologist should investigate whether the interventions were actually implemented
with integrity and what worked as well as what did not. The school psychologist might consider examining interventions from a Response-to-Intervention approach, which provides the student with increasingly intensive interventions (Fairbanks, Sugai, Guardino, & Lathrop, 2007). A diagnosis of ED should
not be considered if appropriate interventions have not been tried.
Leung (1993) highlighted the limitations of information gathered through a review of a student’s
records, including “dated materials, incomplete records, [and] skewed opinions” (p. 6). When making
inferences about what information records reveal, the school psychologist should cross-validate this
information with other assessment procedures before drawing conclusions. So, if it is noted that the
student has frequent absences, tardiness, and trips to the nurse’s office, the school psychologist might
form the hypothesis that the student may be experiencing significant anxiety or fear about school. In this
case, interviews and standardized measures could be used to test if this is a correct hypothesis.
Comprehensive Assessment of Emotional Disturbance
49
Interviews
Interviewing is a commonly used and important tool in the assessment of students for ED. An interview can be defined as an interpersonal encounter to obtain information about a person’s symptoms and
behaviors, while providing the opportunity for observation of verbal and nonverbal behaviors (Aklin &
Turner, 2006). During a comprehensive assessment, interviews should be conducted with the student,
his or her parents, and the school staff working directly with the student. Collecting information from
these varied perspectives through the interviewing process is important in making eligibility decisions.
The two types of interviews generally used in an assessment are unstructured or open interviews and
semi-structured or structured interviews. In an unstructured interview, the school psychologist determines what questions will be asked. School psychologists can collect information on both strengths and
challenges the student is having, and tailor questions based on interviewee responses. While this type
of interview allows the assessment to fit the individual needs of the interviewee, there are problems with
variance and they tend to be unreliable, reflecting more the psychologist’s perception than a reliable and
accurate picture of the student’s functioning (Aklin & Turner; 2006; Kamphaus & Frick, 2005).
Semi-structured and structured interviews consist of a set of questions that are asked to the student,
parent, or teacher. A stem question is generally provided and then if it is answered affirmatively, followup questions are asked based on the response. Structured interviews are more rigid in terms of the
questions asked and provide explicit scoring criteria, while semi-structured interviews generally provide
sets of questions allowing for some flexibility to ask follow-up questions. Both show better reliability
and validity than unstructured interviews, and these types of interviews have been found to increase the
accuracy of diagnosis for individuals of diverse backgrounds because they rely more on standardized
criteria rather than interpretation (Aklin & Turner, 2006; Kamphaus & Frick, 2005). One example of a
semi-structured interview that may be appropriate for use for an ED assessment is the Child Assessment
Schedule (CAS; Hodges, Kline, Stern, Cytryn, & McKnew, 1982), which has been shown to have good
reliability and validity (Hodges, Cools, & McKnew, 1989; Hodges & Saunders, 1989).
Structured and semi-structured interviews have some limitations that the school psychologist should
consider. First, interviews can last for 60-90 minutes, generally longer than an open interview. Additionally, there is some evidence that the student’s self report on diagnostic interviews is unreliable for
students younger than 9 years (Hodges & Zeman, 1993).
By using interviews, especially a semi-structured interview supplemented with other important
questions, the school psychologist can better understand the parameters of a student’s emotions and
behaviors that are not generally assessed by a behavior rating scale or classroom observations. For
example, information can be obtained about the duration of the student’s behavioral difficulties, the age
at which the problems began to emerge, the level of impairment that is associated with the symptoms,
and when symptoms occur (Merrell, 2003). An interview can assess additional psychosocial stressors
that may contribute to a student’s emotional difficulties, allowing the symptoms to be understood within
the context. Interviewing also allows the school psychologist to establish rapport, trust, and security
with the student, family, and teacher, which may be crucial to the eventual implementation of an intervention plan.
Observations
Observing a student is “one of the most direct and objective” (Merrell, 2003, p. 51) ways for the
school psychologist to see a student’s interactions and behaviors in various naturalistic settings (e.g.,
50
The California School Psychologist, 2007, Vol. 12
classroom and playground) across time. There are definite limitations to observations, such as observer
bias and observer influence on a student’s behavior (Leung, 1993) and the time commitment that observations require is great, but observations can provide valuable information about a student’s current performance. Observations also allow the school psychologist to better understand the interaction between
the student and the environment, which helps determine the antecedents, consequences, and functions of
behaviors and to plan interventions. The school psychologist might want to conduct observations before
conducting the student interview or formal testing. This not only reduces the likelihood that the student’s
behavior will be influenced by the presence of the observer, but also allows the school psychologist to
form hypotheses about the student that should guide the choice of testing instruments.
When conducting an ED assessment, it is important for the school psychologist to conduct both
unstructured and structured observations of a student. Unstructured observations typically take the form
of a running log of a student’s behaviors during the observation period. The school psychologist might
note such things as type of response to teacher directions, on-task behavior, responding to or ignoring
other students, level of participation, and response to redirection or discipline. This type of observation
should inform the school psychologist about what behaviors are problematic, what type of structured
observation should be used to collect data, what type of assessment battery is appropriate, and what types
of interventions should be tried.
Structured observations typically take the form of collecting and coding data on certain observable behaviors during a defined period of time. Volpe, DiPerna, Hintze, and Shapiro (2005) provide a
comprehensive review of structured observation coding systems, including characteristics, psychometric
properties, strengths, limitations, and recommendations for each system. Based on this review, the
Direct Observation Form (DOF; Achenbach & Rescorla, 2001) is supported for observing both externalizing and internalizing problems across group settings. This system “requires that the observer write a
narrative and observe on- and off-task behavior simultaneously” (Volpe et al., 2005, p. 467) and then
rate the student’s behavior on items that generally correspond to the Child Behavior Checklist (Achenbach & Rescorla, 2001). The DOF also allows for social comparison data to be collected, which helps
to determine if the student’s behavior is significantly different from the behaviors of peers in the same
settings (Merrell, 2003).
It is important for the school psychologist to observe the student in different settings at different
times. Based on the cumulative record review and the teacher interview, the school psychologist should
identify a classroom time when the student is experiencing more difficulties and a classroom time when
the student is experiencing fewer difficulties. This allows the school psychologist to examine the reciprocal relationship between the student and the classroom environment (Landau & Swerdlik, 2005). The
information obtained from these observations can help the school psychologist make recommendations
to support a student’s success in the classroom. Additionally, observing a student in a non-academic
setting, such as recess or lunch, will provide information on the student’s behavior (e.g., peer interactions) in an unstructured setting.
Because observations are time consuming, school psychologists should consider how to make
observations serve multiple purposes. One way to do this is to use the structure of a functional behavior
assessment (FBA) to interpret qualitative and quantitative observation data collected. FBA requires
collecting information on when and where behaviors occur most often, what is happening before the
behavior occurs, and what is reinforcing the behavior (Watson & Steege, 2003). By collecting these
data, the school psychologist can develop hypotheses about the function of the behavior and environmental factors that need to be changed (Wright et al., 2001). If significant external factors are present,
Comprehensive Assessment of Emotional Disturbance
51
then environmental changes should be considered rather than a diagnosis of ED.
In addition to observing in the classroom and other school settings, school psychologists should
attend to observational data collected through testing sessions. Because testing sessions are structured
and tests are administered in a standardized manner, this environment allows the school psychologist to
compare observations of a student to other students with whom the school psychologist has tested, as well
as how the student reacts to factors unique to the testing environment, such as one-to-one interactions
and few distractions (McConaughy, 2005). If school psychologists want to quantify observational data
during testing, the Test Observation Form (McConaughy, 2005) provides a norm-referenced observation
system. This system requires that the school psychologist keep a running log of a student’s behavior
during testing and then rate the behavior on items that generally correspond to other rating forms in the
Achenbach System of Empirically Based Assessment (ASEBA; Achenbach & Rescorla, 2001).
Tests
“Testing,” in the RIOT model, is loosely defined as the collection of data through various instruments, including cognitive assessments, self-report measures of behavior, behavior rating scales, and
projective assessments. Most of these provide school psychologists with information about a student’s
strengths and challenges as compared to other students of the same age. When using standardized instruments, it is important to consider potential sources of bias in administration, scoring, and interpreting
test results (Sattler, 2001). For example, Skiba, Knesting and Bush, (2002) found Caucasian teachers,
compared to teachers of color, gave Mexican-American and African-American students disproportionately higher ratings for problem behaviors on behavior rating scales. Although these ratings do not
necessarily indicate bias inherent to the test, they do suggest that behavior rating scales are not immune
to informant bias (Skiba et. al., 2002).
Cognitive Assessment
Over 60% of school psychologists use intelligence tests as part of most ED assessments (Shapiro &
Heick, 2004). At a very basic level, cognitive assessment allows the school psychologist to determine
the extent to which students may be experiencing intellectual or sensory difficulties that impact their
ability to learn. In more traditional districts that routinely use cognitive assessments, they may help
rule out learning disabilities and mental retardation (Teeter & Korducki, 1998). However, in districts
that employ a Response-to-Intervention model, cognitive assessments are not a necessary part of the
ED assessment as intervention data, observations, and work samples can provide this information. If
cognitive assessments are used, the school psychologist can examine data on cognitive strengths and
weaknesses to better understand the interaction of the individual characteristics of the student with the
classroom environment (Wright et al., 2001). For example, if a student has average cognitive abilities
but slower processing speed, the student might be having inappropriate behaviors in the classroom due to
frustration with timed tasks such as tests or embarrassment about being called on before having time to
process the question. Thus, adequate academic and behavioral interventions would need to be employed
before considering the student for a diagnosis of ED.
Issues of bias in intelligence tests pervade the literature, and perhaps the most significant source of
bias concerns the use of these assessments with ethnic minority students. This may be especially true for
African American students, who, on average, score one standard deviation below Caucasian students on
standardized cognitive measures (Chung-yan & Cronshaw, 2002; Kwate, 2001). Given the overrepresentation of minority students labeled as ED, school psychologists should strongly consider using other
52
The California School Psychologist, 2007, Vol. 12
sources of data to determine eligibility.
Behavior Rating Scales
Behavior rating scales are the most commonly used assessment modality of childhood psychopathology (Achenbach & McConaughy, 1987) due to their many advantages. Rating scales are easy to
administer and score, cost-efficient, based on normative data, and organized by grouping problems into
larger scales (McConaughy & Ritter, 2002). They also provide information on a large range of problems
and allow for systematic comparison across informants (McConaughy & Ritter, 2002).
For an ED assessment, behavior rating scales are typically completed by teachers, parents, and
other adults who know the student well (e.g., administrator or day-care provider). Parent reports are
especially important as they can provide information about the severity, duration, and frequency of
behaviors (Kovacs, 1986). Similarly, teacher reports are beneficial, especially in elementary school,
where teachers spend a large part of the day with the student and are able to observe the student over the
course of the day (Epkins, 1995).
Two broad social-emotional rating scales that are commonly used are the Behavior Assessment
System for Children – Second Edition (BASC-2; Reynolds & Kamphaus, 2004), which has a parent
rating form and a teacher rating form, and the Achenbach System of Empirically Based Assessment
(ASEBA; Achenbach & Rescorla, 2001), which contains the Child Behavior Checklist (CBCL; to be
completed by parents) and the Teacher Report Form (TRF; to be completed by teachers). Both systems
have good reliability and validity, and are based on large normative samples of students from backgrounds representative of the population of the United States. Both systems are appropriate for use in an
ED assessment, as they provide information in internalizing and externalizing domains. Once the school
psychologist determines the areas of most concern based on these broad-based systems, he or she can
conduct more targeted assessment of specific problem areas. For example, if both parents and teachers
rate the student high for anxiety, the school psychologist may want to explore specific symptoms of
anxiety with the student through interviews, self-report measures, or a more narrow-band rating scale,
such as the Multidimensional Anxiety Scale for Children (MASC; March, Parker, Sullivan, Stallings, &
Conners, 1997).
Despite their many advantages, behavior rating scales have limitations that should be considered by
the school psychologist. There may be gender differences in how teachers and parents rate a student’s
temperament (Else-Quest, Hyde, Goldsmith, & Van Hulle, 2006). For example, what is considered
“cries easily” for boys might be different than for girls. Also, as previously mentioned, characteristics
of the rater can impact ratings (Skiba et al., 2002). Additionally, while parents have been found to be
good at reporting overt behaviors such as conduct problems (Dollinger, 1992), they tend to underreport
depressive symptoms (Angold et al., 1987; Jensen, Rubio-Stipec, & Canino, 1999). Therefore, the
school psychologist should use other sources of data, such as observations and interviews, to ensure that
their conclusions about a student’s functioning are supported.
Self-Report Measures
Self-report measures share similar advantages with behavior rating scales, and are especially important in assessing feelings and behaviors that are difficult to observe directly. The two systems previously
discussed, the BASC-2 and ASEBA have self-report forms. Reliability of self-reports increases with age
(Edelbrock, Costello, Dulcan, Kalas, & Conover, 1985). Consequently, it is important to put more weight
on parent and teacher reports for younger students.
Comprehensive Assessment of Emotional Disturbance
53
Merrell (2003) identified a number of different response biases that can affect the results of selfreport measures. The first, acquiescence, refers to the tendency among children to answer dichotomous
questions (those requiring a true/false or yes/no response) consistently in one direction. This can be
particularly problematic when test items are unclear. The second, social desirability, refers to either a
conscious or unconscious tendency to respond to test items in a manner that makes the student appear
favorable to others. The third, faking, refers to deliberate actions taken by the student to create either
favorable or negative impressions of him or herself. Because of the errors that occur through response
bias, it is important that the school psychologist cross-validate findings with other sources of information.
Projective Tests
Projective tests, such as sentence completion tasks, storytelling techniques, and drawings, are thought
to access information about a student’s internal experience through the use of ambiguous stimuli. In a
recent study by Hojnoski, Morrison, Brown, and Matthews (2006), over half of the school psychologists surveyed indicated using projective tests in their practice, including using them to make eligibility
decisions. However, since their inception, projective tests have been plagued by controversy, generally
due to their lack of adequate reliability and validity (Miller & Nickerson, 2006). Because of the major
concerns with the psychometric properties of projective techniques with students, their use in determining ED eligibility is not recommended (Miller & Nickerson, 2006; Smith & Dumont, 1995). Instead,
school psychologists should consider using the tests as a way to build rapport and generate hypotheses
rather than using them to draw conclusions about a student’s social and emotional functioning (Garb,
Wood, Lillienfeld, & Nezworksi, 2002; Miller & Nickerson, 2006).
Sentence completion tasks are the most commonly used projective technique by school psychologists
(Hojnoski et al., 2006). They come in a variety of forms that focus on different areas of psychological
functioning and have different purposes (Rabin & Zltogorski, 1981) and are generally quick to administer. The open-ended nature of the tests may facilitate students’ ability to express their attitudes and
feelings because they allow for a wide variety of responses (Holaday, Smith, & Sherry, 2000). Rogers,
Bishop, and Lane (2003) suggest that sentence completion tasks can be used as a quick screening of
feelings toward self and others. However, sentence completion tests are often administered in a nonsystematic way, are not formally scored, and are rarely individualized based on the presenting problem
(Holaday et al., 2000). If school psychologists are going to use them, they should consider administering
more specific sentence completion tests, such as the Sentence Completion Test for Depression, for which
there is evidence of reliability and validity in assessing depressive thinking (Barton, Morley, Bloxham,
Kitson, & Platts, 2005).
Over a quarter of school psychologists who responded to the survey reported using drawing techniques in their practice (Hojnoski et al., 2006). Drawing tasks, such as the Draw-A-Person: Screening
Procedure for Emotional Disturbance (DAP:SPED; Naglieri, McNeish, & Bardos, 1991), are quick and
require little verbal skills, which can help in working with younger students or students with lower verbal
abilities (Matto, 2002). The DAP:SPED, which is normed on a large standardization sample, has been
found to be able to distinguish students with emotional disturbances from typically developing peers
(Matto, Naglieri, & Clausen, 2005) and has been found to be appropriate for use with African-American,
Hispanic, and Caucasian students (Matto & Naglieri, 2005). While the DAP:SPED has been shown to be
an adequate assessment of internalizing behaviors, it is not as useful for assessing externalizing behaviors (Matto, 2002). In order to assess externalizing behaviors, observations, behavior rating scales, and
54
The California School Psychologist, 2007, Vol. 12
interviews provide more useful data.
Storytelling techniques, such as the Roberts-Second Edition (Roberts-2; Roberts & Gruber, 2005)
and the Thematic Apperception Technique (TAT; Murray, 1943) are reported to be used by 16-30% of
school psychologists (Hojnoski et al., 2006) and are thought to be non-threatening and fun for children
(Kamphaus & Frick, 2005). In general, storytelling techniques can inform the school psychologist about
how the student constructs his or her world, the quality of interactions between the student and others,
and how the student resolves conflicts. While the TAT is generally interpreted by examining recurrent
themes, the Roberts-2 has explicit scoring procedures, resulting in adequate reliability. It should be
noted that the developers of the Roberts-2 describe this instrument as a means to assess social cognitive
understanding (Roberts & Gruber, 2005). Limitations of storytelling techniques include their heavy
reliance on the student’s verbal abilities and questionable reliability and validity resulting from inconsistent administration, scoring, and interpretation (Kamphaus & Frick, 2005). Like all projective tests,
storytelling techniques should not be used as a sole means of determining eligibility and might be better
used to build rapport with students.
Considering Strengths
Historically, psychoeducational assessments focus on a student’s deficits (Epstein, 1999), and this
seems to be particularly true in the case of ED assessment. It is as if the assessor has to point out that
the student is failing in all areas to make the case for ED, but this is unnecessary. The federal definition
requires that a student only meet one of the criteria to be considered ED, although a student often meets
several criteria. In addition, it is to the benefit of everyone, especially the student and family, to not
only include but to highlight a student’s strengths because this provides hope to everyone and aids in the
development of interventions (Epstein & Sharma, 1998).
Information about a student’s strengths should be gleaned from all of the aforementioned datacollection processes. From the cumulative record review, the school psychologist might highlight positive comments from teachers and growth in an academic area (e.g., high math ability). During observations, the school psychologist might note a student’s positive attributes such as an ability to persist
during a difficult task, offers to help other students, attention to the teacher, or sense of humor. During
interviews with the teacher, parents, and student, the school psychologist should ask more than one
question about a student’s strengths or interests, and should follow up with other questions to more fully
understand the student’s positive attributes. These strengths and interests should inform interventions
to ensure success.
There has been a movement to develop strength-based assessments, and research indicates that these
are an important component of a comprehensive ED assessment (Reid, Epstein, Pastor, & Ryser, 2000).
Specifically, the Behavior and Emotional Rating Scale-Second Edition (BERS-2), which has teacher,
parent, and youth rating scales, can be used to quantify a student’s strengths as an overall score (strength
index) and in the following domains: interpersonal strength, family involvement, intrapersonal strength,
school functioning, affective strength, and career and vocational strengths (Epstein, Mooney, Ryser, &
Pierce, 2004). Reid et al. (2000) found that the Behavior and Emotional Rating Scale could discriminate
between typical students and those with emotional and behavioral disorders (EBD), with typical students
scoring significantly higher in all domains. In addition to the BERS-2, which solely assesses a student’s
strengths, the BASC-2 has Adaptive Scales, which measure strengths in addition to the Clinical Scales
that measure problem behaviors (Reynolds & Kamphaus, 2004).
Comprehensive Assessment of Emotional Disturbance
55
Cultural Considerations
Emotions, and thus behaviors and social interactions, vary across cultures, and as such, emotional
responses should be evaluated based on the student’s cultural norms (Mesquita and Walker, 2003). School
psychologists must balance understanding cultural norms with understanding the degree of influence
these norms might have on an individual student. When conducting culturally competent assessment for
ED, it is important to consider assessment data with awareness of the societal and historical forces that
continue to affect minority students (Skiba et al., 2002). Additionally, because measuring emotions is
rather subjective, their assessment is more influenced by cultural differences (Spielberger, 2006).
Given the growing diversity of the United States (as of the year 2000, about one third of the population was of non-European background; Chen, Downing and Peckham-Hardin, 2002), school psychologists must understand the complexities of conducting culturally competent assessment of ED, especially
since there are a disproportionate number of minority students diagnosed as such (U.S. Department
of Education, 2003). Engaging in culturally competent assessment does not mean that culture should
account for everything, nor should the school psychologist discount its impact altogether (Cartledge,
Kea and Simmons-Reed, 2002); rather, culturally competent assessment involves an overall commitment to data collection procedures that do not contribute to the over-identification of minority students
for special education (Skiba et al., 2002). Culturally competent assessment is much broader than simply
examining test bias (Harry & Klingner, 2006; Skiba et al., 2002; Skiba et al., 2006), and at the core of
the issue is ensuring that cultural factors and/or lack of educational opportunity are not contributing
significantly to the student’s school difficulties. Because overrepresentation is a multi-faceted issue,
which includes considering physical facilities, curriculum, expectations, and motivation, to name a few
(Skiba et al., 2002), school psychologists need to be well-informed about national and local initiatives to
examine overrepresentation. While an in-depth analysis of this topic is beyond the scope of this article,
readers are referred to the following sources: “Cultural and Linguistic Competency and Disproportionate
Representation” (Osher et al., 2004) and Why are so Many Minority Students in Special Education?:
Understanding Race & Disability in Schools (Harry & Klingner, 2006).
Conclusion
While ED assessments are complex, and the methods used to gather data have significant limitations, school psychologists can gain confidence in the conclusions they draw if they do not over-rely on
any one assessment approach and consistently cross-validate their findings using the RIOT model. In
addition to collecting data through the assessment methods described in this article, it is critical that the
school psychologist have an appreciation of the student’s strengths and an awareness of the complexity
of culturally competent assessment to reduce overrepresentation of minority students classified as ED.
By approaching the data collection from an ecological perspective, the school psychologist can present
a more complete picture of the student within the school environment and make appropriate recommendations for interventions that focus on the environmental contributions to and functions of the student’s
behavioral and emotional responses.
The school psychologist should use data gathered from the assessment to make recommendations
for interventions and supports to help the student be more successful in school regardless of whether the
student qualifies for special education services as ED. Recommendations should go far beyond simply
suggesting an appropriate placement. From conducting observations, the school psychologist should
be able to make recommendations about environmental and instructional strategies to help the student
56
The California School Psychologist, 2007, Vol. 12
experience greater success, as well as suggest ways to support the student’s behavior. From interviews
and strength-based approaches, the school psychologist is in a position to recommend strategies to capitalize on things the student enjoys and build on the student’s strengths. The school psychologist should
also recommend ways to help remediate areas of weakness and to build greater academic and social
competence.
References
Achenbach, T. M. & McConaughy, S. H. (1987). Empirically based assessment of child and adolescent psychopathology: Practical implications. Newbury Park, CA: Sage.
Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT:
University of Vermont, Research Center for Children, Youth, & Families.
Aklin, W. M., & Turner, S. M. (2006). Toward understanding ethical and cultural factors in the interviewing
process. Psychotherapy: Theory, Research, Practice, and Training, 43, 50-64. Angold, A., Weissman, M. M., John, K., Merikangas, K. R., Prusoff, B. A. Wickramaratne, P., Gammon, G. D., &
Warner, V. (1987). Parent and child reports of depressive symptoms in children at low and high risk of depression. Journal of Child Psychology and Psychiatry, 28, 901–915.
Barton, S., Morley, S., Bloxham, G., Kitson, C., & Platts, S. (2005). Sentence Completion Test for Depression
(SCD): An idiographic measure of depressive thinking. British Journal of Clinical Psychology, 44, 29-46.
Cartledge, G., Kea, C., & Simmons-Reed, E. (2002). Serving culturally diverse children with serious emotional
disturbance and their families. Journal of Child and Family Studies, 11, 113-126.
Chen, D., Downing J. E., Peckham-Hardin, K.D. (2002). Working with families of diverse cultural and linguistic
backgrounds. In J. M. Lucyshyn, R. H. Horner, & G. Dunlap (Eds.), Families and positive behavior support:
Addressing problem behavior in family contexts (pp. 133-151). Baltimore, MD: Paul H. Brookes Publishing
Co.
Chung-yan, G. A. & Cronshaw, S. F. (2002). A critical re-examination and analysis of cognitive abilities using the
Thorndike model of fairness. Journal of Occupational and Organizational Psychology, 75(4), 489-509.
Dollinger, S. J. (1992). Idiographic and nomothetic child assessment: Convergence between the California Child
Q-Sort and the Personality Inventory for Children. Personality and Individual Differences, 13, 585–590.
Edelbrock, C, Costello, A. J., Dulcan, M. K., Kalas, R., & Conover, N. C. (1985). Age differences in the reliability
of the psychiatric interview of the child. Child Development, 56, 265-275.
Else-Quest, N. M., Hyde, J. S., Goldsmith, H. H., & Van Hulle, C. A. (2006). Gender differences in temperament: A
meta-analysis. Psychological Bulletin, 132, 33-72.
Epkins, C. C. (1995). Teachers’ ratings of inpatient children’s depression, anxiety, and aggression: A preliminary
comparison between inpatient-facility and community-based teachers’ ratings and their correspondence with
children’s self-report. Journal of Clinical Child Psychology, 24, 63–70.
Epstein, M. H. (1999). The development and validation of a scale to assess the emotional and behavioral strengths
of children and adolescents. Remedial and Special Education, 20, 258-262.
Epstein, M. H., Mooney, P., Ryser, G., & Pierce, C. D. (2004). Validity and reliability of the Behavior and Emotional
Rating Scale (2nd Edition): Youth Rating Scale. Research on Social Work Practice, 14, 358-367.
Epstein, M. H., & Sharma, H. M. (1998). Behavioral and Emotional Rating Scale: A strength based approach to
assessment. Austin, TX: PRO-ED.
Fairbanks, S., Sugai, G., Guardino, D., & Lathrop, M. (2007). Response to intervention: Examining classroom
behavior support in second grade. Exceptional Children, 73, 288-310.
Friend, M. (2008). Special education: contemporary perspectives for school professionals. Boston: Pearson.
Garb, H. N., Wood, J. M., Lilienfeld, S. O. Nezworski, M. T. (2002). Effective use of projective techniques in
clinical practice: Let the data help with selection and interpretation. Professional Psychology: Research and
Practice, 33(5) pp. 454-463.
Harry, B., & Klingner, J. (2006). Why are so many minority students in special education?: Understanding race &
disability in schools. New York: Teachers College Press.
Hodges, K., Cools, J., & McKnew (1989). Test-retest reliability of a clinical research interview for children: The
Child Assessment Schedule. Psychological Assessment, 1, 317-322.
Hodges, K., Kline, J., Stern, L., Cytryn, L., & McKnew, D. (1982). The development of a child assessment interview
for research and clinical use. Journal of Abnormal Child Psychology, 10, 173-189.
Hodges, K., & Saunders, W. (1989). Internal consistency of a diagnostic interview for children: The Child Assessment Schedule. Journal of Abnormal Child Psychology, 17, 691-701.
Comprehensive Assessment of Emotional Disturbance
57
Hodges, K., & Zeman, J. (1993). Interviewing. In T. H. Ollendick & M. Hersen (Eds.), Handbook of child and
adolescent assessment (pp. 65-81). Needham Heights, MA: Allyn & Bacon. Hojnoski, R. L., Morrison, R., Brown, M., & Matthews, W. J. (2006). Projective test use among school psychologists: A survey and critique. Journal of Psychoeducational Assessment, 24, 145-159.
Holaday, M., Smith, D. A., & Sherry, A. (2000). Sentence Completion Tests: A review of the literature and results of a
survey of members of the Society of Personality Assessment. Journal of Personality Assessment, 74, 371–383.
Jensen, P. S., Rubio-Stipec, M., & Canino, G. (1999). Parent and child contributions to diagnosis of mental disorder:
Are both informants always necessary? Journal of the American Academy of Child & Adolescent Psychiatry,
38, 1569-1579.
Kamphaus, R. W., & Frick, P. J. (2005). Clinical assessment of child and adolescent personality and behavior (2nd
ed.). New York: Springer Science & Business Media, Inc.
Kovacs, M. (1986). A developmental perspective on the methods and measures in the assessment of depressive
disorders: The clinical interview. In M. Rutter, C. E. Izard, & P. B. Read (Eds.), Depression in young people:
Developmental and clinical perspective (pp. 435-465). New York: Guilford.
Kwate, N. O. A. (2001). Intelligence of misorientation? Eurocentrisim in the WISC-III. Journal of Black Psychology,
27, 221-236.
Landau, S., & Swerdlik (2005). What you see is what you get: A Commentary on school-based direct observation
systems. School Psychology Review, 34, 529-536.
Leung, B. P. (1993, November). Back to basics: Assessment is a R.I.O.T.! National Association of School Psychologists Communiqué, 22, 1-6.
March, J. S., Parker, J. D. A., Sullivan, K., Stallings, P., & Conners, C. K. (1997). The Multidimensional Anxiety
Scale for Children (MASC): Factor structure, reliability, and validity. Journal of American Academy of Child
and Adolescent Psychiatry, 36, 554-565.
Matto, H. C. (2002). Investigating the validity of the Draw-A-Person: Screening procedure for emotional disturbance: A measurement validation study with high risk youth. Psychological Assessment, 14, 221-225.
Matto, H. C., & Naglieri, J. A. (2005). Validity of the Draw-A-Person: Screening Procedures for Emotional Disturbance (DAP:SPED) in strengths-based assessment. Research on Social Work Practice, 15, 41-46.
Matto, H. C., Naglieri, J. A., & Clausen, C. (2005). Validity of the Draw-A-Person: Screening procedure for emotional
disturbance (DAP:SPED) in strengths-based assessment. Research on Social Work Practice, 15, 41-46.
McConaughy, S. H. (2005). Direct observational assessment during test sessions and child clinical interviews.
School Psychology Review, 34, 490-506.
McConaughy, S. H., & Ritter, D. (2002). Best practices in multidimensional assessment of emotional or behavioral
disorders. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology (4th ed., pp. 1303–1336).
Bethesda, MD: National Association of School Psychologists.
Merrell, K. W. (2003). Behavioral, social, and emotional assessment of children and adolescents (2nd ed.) Mahwah,
NJ: Lawrence Erlbaum Associates.
Mesquita, B., & Walker, R. (2003). Cultural differences in emotions: A context for interpreting emotional experiences. Behaviour Research and Therapy, 41, 777-793.
Miller, D. N., & Nickerson A. B. (2006). Projective assessment and school psychology: Contemporary validity
issues and implications for practice. The California School Psychologist, 11, 73-84.
Murray, H. A. (1943). Thematic Apperception Test. Cambridge, MA: Harvard University Press.
Naglieri, J. A., McNeish, T. J., & Bardos, A. N. (1991). Draw-A-Person: Screening procedure for emotional disturbance. Austin, TX: ProEd.
Osher, D., Cartledge, G., Oswald, D., Sutherland, K. S., Artiles, A. J., & Coutinho, M. (2004). Cultural and linguistic
competency and disproportionate representation. In R. B. Rutherford, M. M. Quinn, & S. R. Mathur (Eds.),
Handbook of research in emotional and behavioral disorders (pp. 54-77). New York: Guilford Press.
Rabin, A. I. & Zltogorski, Z. (1981). Completion methods: Word association, sentence, and story completion. In
A. I. Rabin (Ed.), Assessment with projective techniques: A concise introduction (pp. 126-149). New York:
Springer.
Reid, R., Epstein, M. H., Pastor, D. A., & Ryser, G. R. (2000). Strengths-based assessment differences across
students with LD and EBD. Remedial and Special Education, 21, 346-355.
Reynolds, C.R., & Kamphaus, R.W. (2004). Behavior Assessment System for Children, Second Edition (BASC-2).
Circle Pines, MN: AGS.
Roberts, G. E, & Gruber, C. (2005). Roberts-2. Los Angeles, CA: Western Psychological Services.
Rogers, K., Bishop, J., & Lane, R. (2003). Considerations for the use of Sentence Completion Tests. Journal of
Contemporary Psychotherapy, 33, 235-242.
58
The California School Psychologist, 2007, Vol. 12
Sattler, J. M. (2001). Assessment of children: Cognitive applications (4th ed.). San Diego, CA: Jerome M. Sattler,
Publisher.
Shapiro, E. S., & Heick, P. F. (2004). School psychologist assessment practices in the evaluation of students referred
for social/behavioral/emotional problems. Psychology in the Schools, 41, 551-561.
Skiba, R., Knesting, K., & Bush, L. (2002). Culturally competent assessment: More than nonbiased tests. Journal
of Child and Family Studies, 11, 61-78.
Skiba, R., Simmons, A., Ritter, S., Kohler, K., Henderson, M., & Wu, T. (2006). The context of minority disproportionality: Practitioner perspectives on special education referral. Teacher College Record, 108, 1424-1459.
Smith, D. & Dumont, F. (1995). A cautionary study: Unwarranted interpretations of the Draw-A-Person Test. Professional Psychology: Research and Practice, 26, 298-303
Spielberger, C. D. (2006). Cross-cultural assessment of emotional states and personality traits. European Psychologist, 11, 297-303.
Teeter, P. A., & Korducki, R. (1998). Assessment of emotionally disturbed children with the WISC-III. In A. Prifitera
& D. H. Saklofske (Eds.), WISC-III clinical use and interpretation: Scientist-practitioner perspectives (pp. 119138). San Diego, CA: Academic Press.
U.S. Department of Education (2003). Twenty-fifth annual report to Congress on the implementation of Individuals
with Disabilities Education Act. Washington, DC: Author.
Volpe, R. J., DiPerna, J. C., Hintze, J. M., & Shapiro, E. S. (2005). Observing students in classroom settings: A
review of seven coding schemes. School Psychology Review, 34, 454-474.
Watson, T. S. & Steege M. W. (2003). Conducting school-based functional behavioral assessments: A practitioner’s
guide. New York: The Guilford Press
Wright, D. B., Gurman, H. B., & The California Association of School Psychologists/Diagnostic Center, Southern
California Positive Intervention Task Force (2001). Positive interventions for serious problem behaviors: Best
practices in implementing the positive behavioral intervention regulations. Sacramento, CA: California Department of Education.
The California School Psychologist, Vol. 12, pp. 59 – 71, 2007
Copyright 2007 California Association of School Psychologists
59
Mental Health Intervention Teams:
A Collaborative Model to Promote Positive Behavioral
Support for Youth with Emotional or Behavioral Disorders
Katina M. Lambros
Child & Adolescent Services Research Center,
Rady Children’s Hospital, San Diego
San Diego State University
Shirley K. Culver &
Aidee Angulo
Mental Health Resource Center,
San Diego Unified School District
Pamela Hosmer
Special Education Programs Division,
San Diego Unified School District
This paper describes an innovative intervention model for promoting mental health and positive social
adjustment for youth with emotional or behavioral disorders (EBD) in San Diego. More specifically,
it highlights a unique partnership between several program divisions within the San Diego Unified
School District (SDUSD), namely, the Mental Health Resource Center (MHRC) and the Emotional
Disturbance Program (ED) and also includes research and evaluation consultation from the Child and
Adolescent Services Research Center (CASRC). This collaborative service model was developed to
expand and standardize evidence-based interventions for students in self-contained special education
ED classrooms in order to improve their academic and social outcomes.
Keywords: School-based Services, Mental Health Intervention Emotional or Behavioral Disorders,
Evidence-based Practices, Positive Behavioral Support.
The provision of appropriate educational and mental health services for youth with emotional or
behavioral disorders (EBD) is a challenging endeavor for school systems (Landrum, Tankersley &
Kauffman, 2003). While there has been substantial progress in the school-based services literature
outlining positive behavioral support and evidence-based intervention for youth with EBD (Colvin, 2004;
Lane, Gresham & O’Shaughnessy, 2002; Sprague & Walker, 2000; Webster-Stratton, Reid & Hammond,
2004), the extent to which these services are available in all classrooms remains unknown (Hunter,
Hoagwood, Evans, Weist et al., 2005). Hunter and colleagues identified the following characteristics of
effective school-based mental health programs, noting they are difficult to achieve: implementing and
sustaining collaboration and training across school staff (i.e., teachers, para-educators, psychologists,
etc.) and community providers (i.e., mental health clinicians), overcoming fiscal constraints, homeschool collaboration, and progress monitoring program effectiveness.
Please send correspondence to Katina M. Lambros, PhD, Research Scientist, Child and Adolescent Services Research
Center (CASRC), Children’s Hospital, San Diego, 3020 Children’s Way MC 5033, San Diego, California, 92123. Email:
[email protected]
60
The California School Psychologist, 2007, Vol. 12
Prevalence Rates and Characteristics of Youth with EBD
Prevalence rates of EBD range from 6-10% of school-age youth (Kaufman, 2005), yet federal data
indicate that less than 1% of students in the U.S. are identified under the special education handicapping
code of Emotionally Disturbed (ED); and in the state of California, even fewer (0.31%) are served under
this category (Hallahan & Kauffman, 2006). Inherent problems with the ED definition, lack of culturally appropriate assessment tools, as well as hesitation to negatively label students are potential reasons
contributing to the under-identification (Kauffman, 2005). Despite this low identification rate, in 2002,
25,984 students in California alone were classified as needing special education services to address their
emotional and behavioral needs (U.S. Department of Education, 2005). Given this sizable number of
students, appropriate and accessible mental health supports are warranted.
Youth with EBD often demonstrate complex behaviors and co-occurring disorders (Kauffman,
2005). Youth served in ED special education have higher rates of mental health disorders than youth
served by primary care, juvenile justice and mental health sectors (Garland, Hough, McCabe, Yeh, Wood,
& Aarons, 2001). Yet national data indicate only 49% of students served as ED received mental health
services and 55% had behavior management plans (U.S. DOE, 2005), a statistic revealing that nearly
half did not receive necessary support.
Students with EBD often have recurrent contact with juvenile justice, are likely to live in single parent
or foster homes, and are frequently economically disadvantaged (Coutinho, Oswald, Best & Forness,
2004; Hallahan & Kaufman, 2006). Males outnumber females within this category by 5 to 1 or more
(Kauffman, 2005). ED is also among the five largest disability categories for all racial/ethnic groups
except Asian/Pacific Islander, with African American students 2.25 and Native American students 1.30
times more likely than all other racial/ethnic groups combined to receive special education under ED and
(U.S. DOE, 2005). Students with EBD also display significant deficits in academic achievement. Due to
frequent off-task, disruptive, and defiant behavior, these students spend less time academically engaged
and often fail to master basic academic skills (Gunter & Denny, 1998; Hinshaw, 1992). Students with
EBD typically perform a year or more below grade level, and compared to students in all other disability
categories, fail more courses, have higher absences, more grade retentions (Kauffman, 2005; Wagner,
1995) and unfortunately, have the highest drop-out rates (U.S. DOE, 2005).
Given the serious mental health and academic needs of the EBD population, they are more often
educated outside the regular classroom than other students with disabilities. In fact, a large portion
(30.7%) of students with EBD is educated outside the regular classroom for more than 60% of the school
day (U.S. DOE, 2005).
Need for an Integrated Educational and Mental Health Treatment Model
There is considerable pressure for schools to address both the mental health and educational needs
of youth with EBD. According to a consensus statement by the School Mental Health Alliance (Hunter
et al., 2005) “health, and especially mental health, is a fundamental cornerstone for ensuring that all
youth have an equal opportunity to succeed at school and that no child is left behind” (p. 8). Along a
similar note, the No Child Left Behind Act recommends “student access to quality mental health care by
developing innovative programs to link the local school system with the mental health system” (p. 427)
(Office of Elementary and Secondary Education, 2002). Lastly, the Individuals with Disabilities Education Act (IDEA) requires that state and local educational agencies equip school personnel with skills to
appropriately address serious behaviors and student mental health (IDEA, 2004).
Because of their daily access to children and families, school personnel are well positioned to
School-Based Services for Youth with EBD
61
address significant academic and mental health issues; however, school personnel alone cannot provide
all needed services. The EBD population often requires services from multiple community agencies
(Farmer & Farmer, 1999) that are ideally, integrated and coordinated with one another as well as with
school services (Zanglis, Furlong, & Casas, 2000).
San Diego Unified Schools Mental Health Resource Center
The Mental Health Resource Center (MHRC) as part of the Parent, Community and Student Engagement Branch of SDUSD (established in 2001) is funded by county monies and a Safe Schools Healthy
Students grant. The MHRC provides mental health prevention and intervention to reduce violence and
substance abuse, decrease emotional symptoms, improve behavior, and raise student achievement and
attendance. MHRC provides assessment, case management, and treatment for students at all grade levels
in both regular and special education. Key components of the MHRC include early screening, accessibility of mental health service, coordination with community mental health providers, and parental
involvement. Its multi-disciplinary and multicultural staff consists of clinical psychologists, licensed
mental health clinicians, social workers, school counselors, behavioral rehabilitation specialists, and
psychiatrists. Lastly, the MHRC coordinates, operates and oversees a multitude of programs and uses
several evidence-based interventions. A list of MHRC programs is provided on Table 1.
SDUSD Special Education Programs for Youth with EBD and District Demographics
Over 800 SDUSD students require supports and services under the ED category. SDUSD ED classes
are located in 45 comprehensive school sites (elementary, middle and high school). Additionally, 40 ED
classes are located in more restrictive alternative settings (i.e., non-public day and residential treatment
centers). Recommended class size is 10 students for primary classes, and 12 for upper, middle and senior
classes. In 2006, the ethnicity of the students classified as having ED was 29% Latino, 34% Caucasian,
3% Asian/Pacific Islander, 33% African American and 1% American Indian. In the same year, 42% of
the district students were Latino, 26% Caucasian, 17% Asian/Pacific Islander, 14.5 % African American
and .5% American Indian.
It is important to note the diversity of the SDUSD, as it provides the context for all specialized
programs. There is a rising culture of economic poverty in San Diego’s inner-city schools, as the city’s
border location contributes to drug trafficking and a growing population of migrant workers from Mexico
and Central/South America. Influxes of Asian and East African refugees contribute to language barriers,
ethnic tensions, and unemployment. Twenty-eight percent of the district students are English learners
and 55% receive free/reduced lunch.
MHRC and ED Program Collaboration
A partnership between the MHRC and the ED program was established to address the need for
added mental health support in classrooms by restructuring the way district services are delivered.
Through a series of planning meetings, a continuum of services was developed. The primary goal was to
deliver intensive services to students and their families in their local neighborhoods in order to prevent
more restrictive placement (out of neighborhood) of students with EBD. Additional objectives included
reducing suspensions and increasing academic achievement among students with EBD, as well as
improving teacher retention in ED classrooms. By providing teachers with a larger repertoire of therapeutic interventions, it was hypothesized that teachers would be better equipped to address the unique
Coordinates assignment & oversight of community mental health contracted EPSDT
providers delivering individual, group & family services at 60 schools.
MHRC provides MST services to families of students at ALBA Schools and Clark
Middle. MST is an evidenced-based practice which addresses delinquency, gang
activity, physical assault, & school truancy.
MHRC provides assessment, individual & family counseling to students with
significant attendance problems.
School sites can purchase mental health staff to provide services, consultation,
& staff training for working effectively with challenging students & families.
3.EPSDT (Medi-Cal)
School Site Provider
4.Multi-Systemic
Therapy (MST)
5. School Attendance
Review Board
(SARB)
6.School Site Support
1.AB2726
MHRC has contracted with Children’s Mental Health to provide individual & family
therapy to students who need mental health services through the IEP process.
MHRC provides mental health supports to young children
through consultation, assessment, referral, & parent training.
2.Early Childhood
Special Education
Mental health clinicians located on ALBA sites provide immediate access to services
(assessment, evaluation, & treatment). Over 1,200 students screened for mental
health disorders, over 700 students received group, individual, or family therapy, &
over 350 received transition (reintegration back to neighborhood school) counseling.
Program Description
1.Alternative Learning
for Behavior and
Attitude (ALBA)
General Education
TABLE 1. MHRC Programs
(Henggeler et al., 1998)
Multisystemic Therapy
(Commitee for Children,
1998; Grossman et al., 1997)
Second Step
(Botvin, 1998)
Botvin Life Skills
(Henggeler et al., 1998)
Multisystemic Therapy
Evidence-Based
Interventions
62
The California School Psychologist, 2007, Vol. 12
Marcy School & New Dawn full-day rehabilitation program provide
comprehensive treatment services to special needs elementary, middle,
& high school students.
IOP pilot project attaches a mental health team to one ED elementary or middle
school classroom for an entire semester and then moves to a new class the following
semester, leaving behind case management services in first classroom. The IOP team
consists of a mental health therapist, a rehabilitation technician, a case manager,
a part-time psychiatrist and targets three key environments/people: a) classroom
teacher & support staff, b) parent/caregiver, and c) students.
Intervention components teach appropriate social & behavioral skills through positive behavioral support.
MHRC provides treatment services to DHH
students w/ emotional/behavioral needs.
LCI supports general & special education students in shelters, foster youth
facilities, psychiatric hospitals & day treatment programs. MHRC staff
collaborates with outside agencies providing treatment & ensures success during
student transitions.
MHIT provides services to all elementary & middle schools that have SDC-ED
classrooms on site. MHIT Teams help to structure classrooms, develop point
systems/token economies, assist school teams in developing, implementing, and
monitoring behavior support and behavior intervention plans (BSP/BIP), conduct
group therapy for anger management, social skills, etc. (once or twice per week), and
provide individual therapy (30 -60 minutes per week) that is dependant upon student
and family needs.
Assists with consultation, hiring mental health staff & program supervision (TRACE,
Diagnostic Learning Center)
2.Day Treatment
3.Intensive Outpatient
Program (IOP)
4.Deaf & Hard of
Hearing (DHH)
5.Licensed Childrens
Institutions
6.Mental Health
Intervention team
(MHIT)
7.Special Education
Program Support
(Chibnall & Abbruzzese,
2004)
Pilot Project:
The Parent Project
(Kumpfer et al., 2007)
Strengthening Families
(Webster-Stratton et al., 2004)
The Incredible Years
(Kumpfer et al., 2007)
Strengthening Families
(Commitee for Children,
1998; Grossman et al., 1997)
Second Step
(Kumpfer et al., 2007)
Strengthening Families
(Commitee for Children,
1998; Grossman et al., 1997)
Second Step
School-Based Services for Youth with EBD
63
64
The California School Psychologist, 2007, Vol. 12
challenges of this population, resulting in fewer disciplinary actions that place students outside of the
class (i.e., principal’s office, counseling department). A final goal was to improve student achievement
and behavior by engaging and empowering parents to be active members of intervention teams.
According to these program objectives, the partnership established a continuum of integrated educational/mental health programs for students classified as ED, which included direct service to students,
consultation with teachers on meeting the mental health and behavioral needs of students with EBD, and
parent training. This initiative was implemented by the Mental Health Intervention Teams (MHIT).
Mental Health Intervention Teams (MHIT)
The MHIT program provides services to all elementary and middle schools on regular school
campuses that have self-contained ED classrooms onsite. Program components include classroom
behavioral interventions, consultation services, case management, traditional individual and group
psychotherapy, and family outreach and parenting groups.
MHIT consists of 6 teams (1 mental health clinician and 1 rehabilitation specialist) to serve ED
classrooms. The employment qualifications required for a mental health clinician include: a) a master’s
degree in psychology, counseling, social work, or related field; b) licensure as a marriage and family
therapist, clinical social worker, or clinical psychologist; and c) four years of post-license experience in
counseling and youth/family crisis intervention. For the rehabilitation specialist position, a college degree
or license is not essential, however, three years of behavior modification experience (training, experience
and/or education) with emotionally disturbed or conduct-disordered youth in a mental health setting,
preferably in inpatient hospitalization, intensive day treatment, or residential treatment, is required. A
clinical psychologist supports all MHITs and provides neuropsychiatric assessment on complex cases as
well as case consultation. Lastly, a psychiatrist is also available to provide medication management and
consultation for youth.
Each MHIT has a caseload of 6-8 ED classes (a mix of elementary & middle schools). While these
teams support the ED classrooms in a variety of ways, the primary focus is to provide service at three
main levels:
Classroom/Teacher: The MHIT provides behavior and classroom management strategies to teachers
and para-professionals. Teams help to structure classrooms, develop point systems/token economies, and
assist school teams in developing, implementing, and monitoring function-based behavior support and
behavior intervention plans (BSP/BIP).
Individual Child/Youth: The MHIT conducts group therapy for anger management, social skills, etc.
(once or twice per week), and provides individual therapy (30 -60 minutes per week) for those students
and families needing more intensive treatment.
Parent Outreach: The MHIT provides outreach to parents of students with ED. Substantial time
each week is spent calling and visiting family homes to build trust and recruit caregivers to attend
weekly parenting groups using empirically supported curricula (see Evidence-Based Intervention section
below). MHIT staff also provides parent education on various topics and if necessary, refers them to
adult mental health resources in the community.
Evidence-Based Intervention Components Implemented by MHIT
The MHIT staff was trained in one or more of the following interventions: a) The Incredible Years,
b) Strengthening Families, and c) Parent Project. The MHRC sent several MHIT staff to training in one
or more of the above interventions or arranged for trainings to be conducted locally in San Diego. The
School-Based Services for Youth with EBD
65
MHRC plans to roll-out further training on these interventions as well as “refresher/booster” trainings
during Summer 2007 to ensure all MHIT staff are trained in the three programs by the 2007-08 academic
year.
The Incredible Years (IY): Elementary Classrooms: The Incredible Years: Parent, Teacher, and Child
Training Series is a comprehensive curriculum to promote social competence and prevent, reduce, and
treat aggression and related behavior problems in children ages 3-10. The parent intervention is ideally
delivered in 2-hour, weekly parent group sessions lasting 20 weeks. The child component is designed as
a “pull out” treatment program for small groups of children exhibiting conduct problems, and is to be
delivered in 2-hour weekly group sessions lasting 20-22 weeks. The teacher training program is focused
on strengthening classroom management strategies, promoting children’s prosocial behavior and school
readiness, and reducing aggression and non-compliance. This component can be used to train a variety
of school staff (i.e., teacher, aides, psychologists, school counselors).
All IY intervention components have been evaluated and positive findings have been replicated by
independent investigators on different ethnic populations and age groups (Webster-Stratton & Hammond,
1997; Webster-Stratton, Reid & Hammond, 2001; Webster-Stratton et al., 2004). Participation in the IY
was associated with improvements among culturally diverse, socio-economically disadvantaged populations with mental health problems, including young children diagnosed with Obsessive Compulsive
Disorder (ODD), Conduct Disorder (CD) and/or Attention Deficit Hyperactivity Disorder (ADHD). The
IY has been modified for Spanish speaking families, which is appropriate for the linguistic diversity
present in SDUSD.
The Strengthening Families Program (SFP). Middle School Classrooms: The Strengthening Families Program is a 14-session parenting and family skills training program designed to increase resilience
and reduce risk among youth 10-14 years of age through skill-building, improved parenting practices, and
strengthened relationships between children and parents (Kumpfer, Alvarado & Tait, 2007; Molgaard,
Kumpfer & Spoth, 1994). This SFP involves groups of 4-12 parents in a Parent Skills Training group
conducted during the first hour of each weekly session and a separate Children’s Skills Training group to
be held concurrently. In the second hour the families are split into two multifamily Family Skills Training
groups that are facilitated by two MHIT group leaders. Families are taught and encouraged to practice
observation, monitoring, therapeutic play, communication, and positive discipline skills.
SFP has been associated with improvements in parent competencies, adolescent substance-related
risk, and school engagement, as well as long-term academic success (Spoth, Randall & Shin, in press).
The SFP has also been adapted for multiethnic populations, including economically disadvantaged and
urban youth (Kumpfer, Alvaredo, Smith & Bellamy, 2002).
The Parent Project Pilot: Middle School Classrooms: The Parent Project was created for parents
of adolescents, 11-19 years of age, with difficult or unmanageable behaviors. Because several of the
EBD students presented significant delinquent and highly destructive behaviors (i.e., substance use, gang
involvement, practice of the occult, running away, violence toward others and suicide), the MHIT staff
developed lessons that directly tackled these serious behaviors. Using a structured, self-help support
group model, parents learn and practice specific prevention and intervention strategies to address each
of the above mentioned behaviors. The PP lasts 10-16 weeks and includes two intervention units. Unit
I, “Laying the Foundation for Change,” consists of six activity-based instructional units. Typically,
each unit is delivered via a weekly three-hour session. Unit II, “Changing Behavior and Rebuilding
Family Relationships,” includes 10 topic-focused parent support group sessions, which are delivered
via weekly two-hour blocks. Sessions provide parents with emotional support and include an activity-
66
The California School Psychologist, 2007, Vol. 12
based parenting skills component. The PP has been featured by the Office of Juvenile Justice and
Delinquency Prevention as a promising community and school program to decrease serious delinquent
behavior (Chibnall & Abbruzzese, 2004), however; this program has not been sufficiently researched
and is presently considered a pilot project.
MHIT and School Staff Interface
At each school with an onsite ED classroom a designated liaison assists the MHIT. More specifically, this liaison can be any school staff member on campus (i.e., school counselor, district resource
teacher, vice principal, school psychologist, etc.) who agrees to orient the MHIT with the school, coordinate shared office space, and attend weekly/monthly meetings with the MHIT, ED classroom staff, and
support personnel.
At the beginning of the school year, the MHIT conducts direct observation and interviews (both
semi-structured and informal) with ED teachers and school staff to determine the level of support that is
needed and builds relationships in that classroom. MHIT clinicians reported that in some classrooms, ED
teachers demonstrated exceptional classroom organization, behavior management skills, and effective
instructional practices. These classrooms generally required minimal classroom support, with occasional individual child crisis intervention services or consultation for particularly difficult students and
moderate (i.e., weekly) concentration on parent/family outreach. Other ED classrooms require significant and ongoing support in developing classroom reinforcement systems, teaching positive behavioral
support strategies, and adjusting instructional practices to be more appropriate for students. Such classes
often required intense individual child services and significant parent outreach, resulting in high involvement (i.e., daily) of the MHIT with that particular class. In essence, the MHIT developed an individualized program of support for each classroom on their caseload.
MHITs adjust support services to accommodate differing levels of student mainstreaming into
general education, which varies by school. In the majority of self-contained ED classes, students remain
with the same teacher and peer cohort for most of the school day, with some students mainstreaming
as appropriate. However, approximately 30% of middle school programs do not have a “core” ED
classroom and instead, students are fully mainstreamed into regular classes with itinerant support. In
these schools, the MHIT attempted to maintain contact with several general education teachers and the
special education teacher regarding student progress and/or behavioral support plans. But it was difficult
to provide consultation services to each of the general education teachers and to provide direct services
in the individual classrooms. Additionally, as youth are in different general education classes, it was a
challenge to find common class periods to conduct group therapy sessions.
MHIT Interface with School Psychologists:
Most of the MHITs communicate several times per week with the school psychologist on site.
The MHIT staff provides the school psychologist with assessment and/or treatment information on the
students they serve such as, their history of mental health support services, informal and formal behavioral observation data, behavioral rating scales results, and group or individual therapy progress updates.
MHITs and school psychologists work together to jointly support ED teachers and classroom aides with
implementation of behavioral support services (i.e., especially on days where one or the other is not
working at the school site), monitor teacher implementation of intervention strategies, make referrals
for additional mental health services (i.e., AB3632, etc.), and co-facilitate child therapy groups. The
sustained behavioral and mental health support to the ED classes frees the school psychologist to provide
School-Based Services for Youth with EBD
67
more services to general education students (i.e., Student Study Team, 504 process, prevention efforts,
behavioral consultation, etc.), which may prevent ED referrals. Further collaboration occurs regarding
parent outreach, as school psychologists and MHIT staff work to engage parents of ED students to
become more active in their child’s academic and behavioral interventions. Both MHITs and psychologists attend all Individual Education Plan (IEP) meetings, as well as weekly and monthly meetings with
ED teachers and school administration.
The MHITs may also interact with one of four psychologists who are assigned (full-time) to a single
school with multiple ED classes. In these instances, the cross-coordination and communication between
the MHIT and ED school psychologist occurs several times a day as they work together in multiple
ways to support students with ED and their families. The ED program intends to hire four additional ED
project school psychologists for the 2007-08 year.
Research and Community Partnership: MHRC/SDUSD and CASRC
The Child and Adolescent Services Research Center (CASRC) at Rady Children’s Hospital-San
Diego is a NIMH funded center comprised of a multi-disciplinary consortium of investigators as well
as community representatives from the public system of care (i.e., mental health, child welfare, juvenile
justice, education, alcohol/drug, primary care). CASRC conducts mental health services research that
spans clinical epidemiology studies linked to evidence-based practice, effectiveness and quality of care
studies, and implementation studies. CASRC has a long standing relationship with the MHRC and has
conducted program evaluation and provided consultation on evidence-based interventions for youth with
a range of academic and behavioral needs. Members of CASRC and the MHRC meet monthly to discuss
school-based services and research collaborations. This relationship provides a unique and valuable
opportunity to study large-scale specialty mental health care taking place in the real world context of
schools.
Preliminary Evaluation of MHIT
Process Evaluation: In 2005-06, the MHITs served 37 ED classrooms on comprehensive sites. This
start-up year consisted of a) building relationships and integrating with educational and administrative staff, b) identifying evidence-based interventions for elementary and middle school populations, c)
deciding the appropriate amount of time spent in each class, d) assessing the correct balance of elementary and middle school ED classes on team caseloads, e) determining levels of clinical supervision, f)
scheduling formal training for MHIT members on chosen interventions and g) provision of consultation
services to classroom teachers.
For 2006-07, the MHIT provided services to students in 38 ED classes on comprehensive school
campuses. This second year involved a) provision of direct services to students and parents, b) continuing
consultation to classroom teachers, c) hiring additional MHIT staff, d) training new MHIT staff on the
IY and SFP, e) training all staff on the new PP intervention, f) working to build relationships at newly
assigned schools or with new ED teachers, and g) working with school administrative staff on discipline
and suspension policy.
Outcome Evaluation: Members of CASRC, MHRC, and MHIT are trying to secure grant funding
for the outcome evaluation component of the MHIT program. Evaluation meetings have focused on how
to best capture outcome variables; a process which remains quite challenging as MHIT services vary in
intensity within and across the individual stakeholders: students, parents, and teachers. It is important to
note, that limited evaluation data have been collected to date (as focus has been on program development
68
The California School Psychologist, 2007, Vol. 12
and implementation), however, an evaluation plan has been drafted. The outcome evaluation is designed
to collect information from each stakeholder:
Classroom/Teacher: Classroom teacher acceptability and satisfaction with the MHIT model was
assessed with a 15-item survey. Additionally, this survey was administered to school staff who support
the ED class (i.e., school counselors, administrators, psychologists). All school staff completed the
survey anonymously. A survey response rate of 34% was obtained. The MHIT staff (the clinician and
rehabilitation specialist) completed a longer 30-item survey for each assigned ED class. The MHIT
survey response rate was substantially higher at 82%. Table 2 contains a portion of the results which
addresses the quality of the MHIT and school staff relationship. The overwhelming majority of ED
teachers and school staff report having a positive relationship with their MHIT, maintaining effective
team communication, and feeling supported by the model. MHIT staff report was also positive, although
included more variable responses.
Several open ended survey questions asked “what impact did the MHIT have in your classroom?”
Teacher responses included “MHIT helped me a great deal with classroom management, they put a
great system in place and I followed it” as well as “the anger management and peer interaction groups
the MHIT conducted were great…problem behavior decreased in frequency while learning increased.”
Surveys for the 2006-07 year were administered.
ED teacher retention was also tracked. In the year prior to the MHIT program (2004-05), there
existed a 50% ED teacher turnover rate, with only 19 of the original 38 elementary and middle school
teachers remaining in the program. Following the first year of MHIT, 29 of 37 ED teachers remained
(78% retention rate), with 8 teachers leaving for the following reasons: moving out of state, moving to
general education or RSP program, or receiving promotions.
Future evaluation efforts will focus on formal implementation of treatment integrity measures incorporated by the IY and SFP developers for the classroom interventions. This will be a staged process, with
independent observers assessing the treatment integrity of the MHIT staff as they train the classroom
teachers and school staff and model strategies with students. Subsequently, the school psychologists
and MHIT will jointly monitor the treatment fidelity of the classroom teachers as they use intervention
strategies. Lastly, informal measures assessing classroom organization, structure, and climate will also
be collected.
Individual Child/Youth: Individual student evaluation will consist of educational indicators (i.e.,
grades, attendance, academic achievement, IEP goal attainment, impairment ratings, educational placement maintenance, etc.) and will be downloaded from the Standards, Assessment, and Accountability
department in SDUSD at the end of each academic year, beginning with the 2006-07 year. In addition,
single subject methodologies will be employed to chart and monitor individual student behavioral progress.
Parent Outreach: During 2006-07, MHITs conducted parenting groups and reported variable attendance ranging from 35-65%. Increased attendance was reported for groups that provided childcare
and food. Barriers to parent participation included transportation to the school. Future evaluation of
parenting groups will include administration of IY and SFP measures (i.e., parenting scale, parent satisfaction questionnaire etc.) to parents/caregivers to assess behavioral support skills and program satisfaction. Independent observers will also assess treatment integrity of the MHIT staff as they train parents
on intervention strategies.
Lastly, at all service levels, qualitative research methods will be used to answer questions that quantitative data may be unable to answer. Qualitative methods (i.e., focus groups, key informant inter-
School-Based Services for Youth with EBD
69
views) describe complex phenomena such as the experiences and interpretation of events by people with
different stakes and roles. Additionally, these approaches can describe complex settings (schools, classrooms, etc.) and interactions (families, teachers, students) (Sofaer, 1999). It is intended that evaluation
results will provide ongoing feedback to the program staff as well as inform the research community on
the feasibility and effectiveness of implementing evidence-based interventions within the framework of
the MHIT service delivery model.
MHIT Implications and Future Directions
The MHIT is a collaborative service delivery model using school-based mental health teams to
implement evidence-based interventions to promote positive social adjustment for youth with EBD and
their families as well as support classroom teachers. This model is aligned with research suggesting that
integrating mental health intervention within schools and classroom settings can improve school climate
and attitudes about mental health as well as support teachers who serve children and adolescents with
EBD (Bruns, Walrath, Glass-Seigel, & Weist, 2004).
By joining educational staff and clinical providers in the classroom to treat students with EBD, the
MHIT has addressed a long standing barrier in the provision of mental health services – lack of infrastructure to support mental health programs (Hunter et al., 2005). MHIT staff does not service youth in
isolation, but is integrated into the existing special education class unit and is part of the support staff
working closely with school psychologists and other personnel. Thus, mental health services are weaved
into the daily classroom curricula rather than fragmented.
TABLE 2. MHIT Program Survey Results
Survey Question
School Staff
MHIT Staff
Surveys (N=17) Surveys* (N=61)
How would you describe the relationship
between you and your Teacher or MHIT
team member?
Very Good
Good
Fair
Poor
15 (88%)
1 (6%)
1 (6%)
0 (0%)
33 (54%)
23 (37%)
4 (7%)
1 (2%)
Do you feel supported by your Teacher or
MHIT team member?
Yes
Sometimes
No
16 (94%)
1 (6%)
0 (0%)
42 (69%)
17 (28%)
2 (3%)
Do you feel you communicate/collaborate
effectively with your Teacher or MHIT
team member?
All the Time
Most of the Time
Could Improve
There are Difficulties
Not at All
14 (82%)
2 (12%)
0 (0%)
1 (6%)
0 (0%)
28 (46%)
20 (32%)
9 (15%)
4 (7%)
0 (0%)
Do you have established weekly or
monthly meetings involving the teachers
and MHIT team members?
Yes
No
13 (76%)
4 (24%)
47 (77%)
14 (23%)
* The MHIT staff completed more than one survey
70
The California School Psychologist, 2007, Vol. 12
The MHIT model may offer districts a template for combining fiscal and personnel resources from
separate departments to provide expanded services. Hunter and colleagues (2005) posit that one of the
most salient obstacles to implementing mental health interventions in schools is a lack of funding to
support and sustain them. By integrating program components and combining fiscal resources to develop
the MHIT, a more efficient use of resources may be possible resulting in increased services to students
and teachers beyond what is likely when mental health and educational programs work in isolation.
One strength of the MHIT model is the parenting programs. Parents of children with EBD are
typically difficult to engage due to countless negative interactions they have had with school personnel
regarding their child’s behavior. Yet, by co-locating mental health services within special education
classes, the MHIT model may provide better access to mental health services for students/families
who would not otherwise seek individual or family counseling. In addition, the emphasis on developing parenting skills that foster prosocial behavior in youth rather than criticize parenting skills may
encourage parental engagement in the groups. Finally, the parent lessons also promote parent-to-parent
support systems that can last beyond the class sessions.
Lastly, the integration of clinical mental health providers and school personnel provides a truly
multidisciplinary approach, allowing cross-discipline training to occur. School staff learn about psychological symptomotology and diagnoses, while MHITs become familiar with effective instructional practice and IEP goals and objectives. Additionally, the involvement of district administrative staff and school
services researchers has enhanced this MHIT model by providing a practitioner-researcher collaborative
to working with youth having EBD.
References
Bruns, E.K. Walrath, C., Glass-Siegel, M., & Weist, M.D. (2004). School-based mental health services in Baltimore:
Association with school climate and special education referrals. Behavior Modification, 28, 491-512.
Botvin, G. (1998). Life skills training-middle school. White Plains, NY: Princeton Health Press.
Chibnall, S. & Abbruzzese, K. (2004). A community approach to reducing risk factors. Journal of the Office of
Juvenile Justice and Delinquency Prevention, IX(1), September, 30-31.
Colvin, G. (2004). Managing the cycle of acting-out behavior in the classroom. Eugene, OR: Behavior Associates.
Committee for Children. (1998). Second Step Violence Prevention Intervention: Steps for Successful Implementation. Retrieved May 2004 from http://cfchildren.org/comboimp.odf.
Coutinho, M., Oswald, D., Best, A., & Forness, S. (2004). Gender and socio-demographic factors in the disproportionate identification of minority students as emotionally disturbed. Behavioral Disorders, 27, 109-125.
Farmer, E.M., & Farmer, T.W. (1999). The role of schools in outcomes for youth: Implications for children’s mental
health services research. Journal of Child and Family Studies, 8, 377-396.
Garland, A.F., Hough, R.L., McCabe, K.M., Yeh, M., Wood, P.A., & Aarons, G.A. (2001). Prevalence of psychiatric disorders in youth across five sectors of care. Journal of the American Academy of Child and Adolescent
Psychiatry, 40(4), 409-418.
Grossman, D.C., Neckerman, H.J., Koepsell, T.D., Liu, P.Y., Asher, K.N., Beland, K., et al. (1997). Effectiveness of
violence prevention curriculum among children in elementary school: A randomized controlled trial. Journal of
the American Medical Association, 277(20), 1605-1612.
Gunter, P.L., & Denny, R.K. (1998). Trends and issues in research regarding academic instruction of students with
emotional and behavioral disorders. Behavioral Disorders, 24(1), 44-50.
Hallahan, D, & Kauffman, J. (2006). Exceptional learners: Introduction to special education, (10th ed.). Boston:
Allyn & Bacon.
Henggeler, S.W., Schoenwald, S.K., Borduin, C.M., Rowland, M.D., & Cunningham, P.B. (1998). Multisystemic
treatment of antisocial behavior in children and adolescents. New York, NY: Guilford Press.
Hinshaw, S.P. (1992). Academic underachievement, attention deficits, and aggression: Comorbidity and implications for intervention [Special issue]. Journal of Consulting and Clinical Psychology, 60(6), 893-903.
School-Based Services for Youth with EBD
71
Hunter, L., Hoagwood, K., Evans, S., Weist, M., Smith, C., Paternite, C., Horner, R., Osher, D., Jensen, P., &
the School Mental Health Alliance (2005). Working Together to Promote Academic Performance, Social and
Emotional Learning, and Mental Health for All Children. New York: Center for the Advancement of Children’s
Mental Health at Columbia University.
Individuals with Disabilities Education Improvement Act Amendments of 2004, 20 U.S.C. § 1462 (h) et seq.
Kauffman, J.M. (2005). Characteristics of emotional and behavioral disorders of children and youth (8th ed.). Upper
Saddle River, NJ: Merrill Prentice-Hall.
Kumpfer C., Alvarado R., Smith, P. & Bellamy, N. (2002). Cultural sensitivity and adaptation in family-based
prevention interventions. Prevention Science, 3(2):241-246.
Kumpfer, K.L., Alvarado, R. & Tait, C. (2007). The Strengthening Families Program: An evidence-based, multicultural family skills training program. In: Tolan, P, Szapocznik, J., & Sambrano, S (eds). Preventing youth
substance abuse: Science-based programs for children and adolescents. Washington, DC, US: American
Psychological Association: 59-181.
Landrum, T.J., Tankersley, M., & Kauffman, J.M. (2003). What’s special about special education for student with
emotional and behavioral disorders? The Journal of Special Education, 37, 148-156.
Lane, K.L., Gresham, F.M., & O’Shaughnessy, T. (Eds.). (2002). Interventions for children with or at risk for
emotional and behavioral disorders. Needham: Allyn & Bacon.
Molgaard, V., Kumpfer, K. L. & Spoth, R. (1994). The Iowa Strengthening Families Program for Pre and Early
Teens. Ames, IA: Iowa State University
Office of Elementary and Secondary Education. (2002). No Child Left Behind: A desktop reference. Washington,
DC: U.S. Department of Education.
Sofaer, S. (1999). Qualitative methods: What are they and why use them? Health Services Research, 34(5), 11011118.
Spoth, R., Randall, G. K., & Shin, C. (In press). Experimental support for a model of partnership-based family
intervention effects on long-term academic success. School Psychology Quarterly .
Sprague, J., & Walker, H. (2000). Early identification and intervention for youth with antisocial and violent behavior.
Exceptional Children, 66, 367-379
U.S. Department of Education. (2005). Twenty-sixth annual report to Congress on the implementation of the Individuals with Disabilities Education Act, vol. 2. Washington, DC: Author.
Wagner, M.M. (1995). Outcomes for youths with serious emotional disturbance in secondary school and early adulthood. The Future of Children, 5(2), 90-112.
Webster-Stratton, C. & Hammond M. (1997). Treating children with early-onset conduct problems: A comparison
of child and parent training interventions. Journal of Consulting and Clinical Psychology, 65, 93-109.
Webster-Stratton, C., Reid, M. J., & Hammond, M. (2001). Social skills and problem solving training for children
with early-onset conduct problems: who benefits? Journal of Child Psychology and Psychiatry, 42(7), 943952.
Webster-Stratton, C., & Reid, M. J. & Hammond, M. (2004). Treating children with early-onset conduct problems: Intervention outcomes for Parent, Child, and Teacher Training. Journal of Clinical Child and Adolescent
Psychology, 33(1), 105-124.
Zanglis, I., Furlong, M., & Casas, J. (2000). Case study of a community mental health collaborative: Impact on
identification of youths with emotional or behavioral disorders. Behavioral Disorders, 25, 359-371.
The California School Psychologist, Vol. 12, pp. 73 – 91, 2007
Copyright 2007 California Association of School Psychologists
73
Diagnosis of Attention-Deficit/Hyperactivity
Disorder (AD/HD) in Childhood:
A Review of the Literature
Stephen E. Brock &
Amanda Clinton
California State University, Sacramento
This article examines recent literature related to the diagnosis of Attention-deficit/Hyperactivity
Disorder (AD/HD) in childhood. First, the article discusses diagnostic criteria presented in the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2000). Next,
it explores the diagnostic procedures for AD/HD recommended in current publications. Results of
this comprehensive literature review indicate that rating scales, interviews, laboratory/psychological
testing, and observations are the most frequently recommended AD/HD diagnostic techniques. The
implications of these findings for school psychologists are discussed.
KEYWORDS: ADHD, Attention Deficit, Diagnosis
Although medical science recognized inattention, impulsivity, and hyperactivity as problematic
among children as long as 200 years ago (Anastopoulos & Shelton, 2001), no single method for reliably diagnosing Attention-deficit/Hyperactivity Disorder (AD/HD) has yet to be identified. This lack
of a specific diagnostic procedure makes AD/HD identification complicated (Brock, 1999; Detweiler,
Hicks, & Hicks, 1999). Furthermore, differential diagnosis can be confounded by a variety of other
disorders that can co-exist with AD/HD or cause symptoms similar to those observed in AD/HD (Levy,
Hay, Bennett, & McStephen, 2004; Power, Costigan, Eiraldi, & Leff, 2004). To address these issues, an
accurate diagnosis requires multiple diagnostic procedures, such as obtaining information from different
sources (Barkley, 1998; Hoff, Doepke, & Landau, 2002), behavioral observations (Miranda, Presentacion, & Soriano, 2002), and psychological assessment (Detwiler et al., 1999).
Before proceeding further, it is important to acknowledge that the role of the school psychologist in
the diagnosis of AD/HD is somewhat controversial as this disorder is not a special education eligibility
category in either state or federal regulations. Consequently, from the first author’s experiences, it has
been observed that some districts do not allow their school psychologists to make AD/HD diagnoses.
Such limitations are problematic given that (a) both state and federal regulations clearly specify that
students with AD/HD may be eligible for special education services or Section 504 protections (CEC
§56339; Davila, Williams, & MacDonald, 1991), and (b) school districts are required to identify special
needs and must conduct assessments in all areas of suspected disability (34 C.F.R. §§ 104.32, 104.35,
300.304). Thus, there appear to be important legal reasons for school psychologists at least being well
informed about the important elements of the AD/HD diagnosis. Further emphasizing the need for such
knowledge is the California Education Code section specifying that all school personnel be trained to
develop greater awareness of AD/HD [CEC §56339(d)].
Consistent with the need for school psychologists to be better informed regarding AD/HD, the
purpose of this article is to review relevant diagnostic issues and information, beginning with a discussion of the American Psychiatric Association’s (APA) diagnostic criteria and, subsequently, presenting
the results of a comprehensive literature review that identified the frequently recommended techniques
Address correspondence to Stephen E. Brock, Department of Special Education, Rehabilitation, School
Psychology, and Deaf Studies; California State University, Sacramento; 6000 J Street; Sacramento, CA 95819-6079.
Email: [email protected]
74
The California School Psychologist, 2007, Vol. 12
for diagnosing AD/HD. This latter part of the paper is intended to provide school psychologists with a
resource that documents the agreed upon elements of a comprehensive AD/HD diagnostic assessment.
While it is clear there is no specific method or set of methods for diagnosing AD/HD, this literature
review reveals significant agreement regarding recommended diagnostic practices. In addition to guiding
the school psychologist’s practice, from the first author’s applied experiences, this knowledge will be
helpful in those instances wherein a school assessment team is presented with an “AD/HD” diagnosis
that is judged to be questionable because it fails to meet standard diagnostic practices.
AD/HD Diagnostic Criteria
AD/HD is included in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition,
Text Revision (DSM-IV-TR; APA, 2000). The diagnosis of AD/HD is found within the section titled,
“Disorders Usually First Diagnosed in Infancy, Childhood, or Adolescence” (pp. 39-134). Common
among all disorders in this section is the presence of symptoms prior to an individual’s 18th birthday. AD/
HD is further subsumed within the category of “Attention-Deficit and Disruptive Behavior Disorders”
(pp. 85-103). Conduct Disorder, Oppositional Defiant Disorder, and Disruptive Behavior Disorders, Not
Otherwise Specified are also included within this broader category.
The primary AD/HD symptoms are a “persistent pattern of inattention and/or hyperactivity-impulsivity that is more frequently displayed and more severe than is typically observed in individuals at a
comparable level of development” (APA, 2000, p. 85). The DSM-IV-TR classifies AD/HD into three
subtypes: (a) Predominantly Hyperactive-Impulsive Type; (b) Predominantly Inattentive Type; and (c)
Attention-deficit/Hyperactivity Disorder, Combined Type, which includes both hyperactive-impulsive
and inattentive symptomology. Although the specific behavioral symptoms presented in the DSM-IV-TR
are relatively self-explanatory, other diagnostic requirements appear to require further elaboration and
are discussed below.
Although not currently specified in diagnostic criteria, leaders in this field have argued that the
Predominantly Inattentive Type of AD/HD can be diagnosed significantly later than AD/HD Hyperactive-Impulsive Type and AD/HD Combined Type (e.g., Barkley, 1998). According to the DSM-IV-TR
“Academic deficits and school-related problems tend to be most pronounced by the types marked by
inattention …, whereas peer rejection and, to a lesser extent, accidental injury are most salient in the
types marked by hyperactivity and impulsivity” (APA, 2000, p. 88). Due to differences such as these,
Barkley (1997a; 1998) has suggested AD/HD Inattentive Type may be a separate and distinct disorder,
rather than a subtype of AD/HD.
Symptom Onset
Even though a clinical diagnosis of AD/HD may come later, symptoms of inattention and/or hyperactivity-impulsivity that compromise daily functioning should be observed prior to a child’s 7th birthday.
However, this does not mean that a diagnosis must be made prior to the age of 7. In fact, it can be made
in adulthood if careful verification of symptom onset prior to 7 years of age is made. To document age of
onset, review of records and clinical interviews with parents and teachers may be required.
It is important to note that, since toddlers and preschoolers are typically very active, it is recommended that developmentally appropriate behaviors be taken into consideration and diagnosis of AD/HD
be made with caution in the preschool years. A diagnosis of AD/HD prior to age 7 should be made only
in cases where children are significantly more active than their developmental peers, so as to avoid the
tendency to over-diagnose the disorder in preschoolers (Loughran, 2003). Conversely, if symptom onset
Diagnosis of AD/HD in Childhood
75
occurs subsequent to 7 years of age (particularly in cases of hyperactivity) the cause may be something
other than AD/HD. For example, the cause of “AD/HD-like” behaviors in the older child or adolescent
might include substance abuse (Burke, Loeber, & Lahey, 2001), learning disabilities, or physical illness
(Root & Resnick, 2003).
Symptom Duration
According to the DSM-IV-TR, the pattern of symptoms for each subtype of AD/HD must be present
for a minimum of six months prior to diagnosis. Symptom duration is critical when differentiating
AD/HD from other disorders and normative developmental transitions, including school adjustment or
temporary family stress. To document duration of symptoms, review of records and clinical interviews
with parents and teachers may be required.
Symptom duration of at least six months is particularly important in assessment and diagnosis of
AD/HD in preschool children. This is because it has been estimated that from 15% to 29% of this
population is rated as inattentive and overactive by their parents at some point prior to age 5 (Gimpel &
Kuhn, 1998; Loughran, 2003). However, in the majority of these cases concerns remit within 12 months
(Campbell, 1990). In other words, significant inattention and hyperactivity/impulsivity in the 3- to 4year-old child is not necessarily indicative of a pattern of AD/HD, but may be considered developmentally appropriate behavior (Barkley, 1998).
Multiple Settings
According to the DSM-IV-TR, the impairments resulting from the symptoms of hyperactivity/impulsivity and/or inattention must be observed in at least two distinct settings, such as home, school, work,
and social situations (APA, 2000). For example, symptoms could be present at home and school, or at
school and after-school programs. The presence of symptomology in only one setting would not be sufficient for a diagnosis. Typically, the severity of symptoms varies between settings. For example, inattentiveness and hyperactivity/impulsivity are typically more problematic in situations of limited interest or
novelty or those that require sustained mental effort. Traditional classroom lectures and lengthy repetitive
tasks typically fall into this category. To document that symptoms are problematic in multiple settings,
information from direct observations or the use of rating scales by various informants (e.g., parents and
teachers) may be appropriate.
Clinical Significance
The DSM-IV-TR specifies that “clear evidence of social, academic, or occupational functioning”
(APA, 2000, p. 93) impairments must be considered before a diagnosis of AD/HD is made. Specifically,
for a child or adolescent to receive a diagnosis, their symptoms must negatively impact their academic
performance, social interactions, and family relationships. For example, if an inattentive and/or hyperactive/impulsive child is able to obtain passing grades, follow classroom and playground rules, and
develop appropriate peer relationships, clinically significant impairments may not exist and an AD/HD
diagnostic assessment (let alone the diagnosis itself) would not be appropriate. To document clinically
significant impairments, review of records and clinical interviews with parents and teachers may be
required.
76
The California School Psychologist, 2007, Vol. 12
Developmental Level
Symptoms of inattentiveness and hyperactivity are often observed among children with low intellectual functioning when they are placed in inappropriate educational settings (APA, 2000). Inattentive
and hyperactive behavior that results from inappropriate school placement must be distinguished from
true AD/HD. In cases of children who have been diagnosed with mental retardation, “an additional
diagnosis of AD/HD should be made only if the symptoms of inattention or hyperactivity are excessive
for the child’s mental age” (APA, 2000, p. 91). For example, if a 12-year-old with a developmental functioning level of 6 years demonstrates behavior typically observed among 6-year-olds, this criterion for
a diagnosis of AD/HD would not be met. In this case the behavior is commensurate with developmental
level. However, if this same 12-year-old demonstrated inattentiveness and/or hyperactivity similar to that
observed among 4-year-olds, a diagnosis of AD/HD may be appropriate. To document that symptoms
are developmentally inappropriate, norm referenced psycho-educational assessments may be required.
Differential Diagnosis
Finally, AD/HD diagnostic requirements require that other conditions with similar symptoms be
ruled out before an AD/HD diagnosis is made (APA, 2000). In addition, the differential diagnosis of
this disorder requires that age-appropriate behaviors among younger children (e.g., jumping, running,
and yelling among preschoolers), mental retardation, under-stimulating environments, oppositional
behavior, learning disorders, and other mental disorders (e.g., Pervasive Developmental Disorders;
Psychotic Disorder; and Other Substance-Related Disorder, Not Otherwise Specified) be considered
and ruled out as primary causes of the observed behaviors before the diagnosis of AD/HD is made. This
requirement highlights the fact that a variety of conditions may generate AD/HD-like behaviors and that
the diagnostic evaluation must include evaluation tools designed to consider these alternative explanations for AD/HD behaviors (Brock, 1999). Thus, even in those instances where the school psychologist
is not making the AD/HD diagnosis, a comprehensive psycho-educational evaluation may be necessary
to differentiate AD/HD from other similar disorders.
Components of the AD/HD Diagnosis
To identify frequently recommended components of an AD/HD diagnosis, the authors conducted a
comprehensive review of the literature published within the past 15 years. Making use of the PsycINFO
database, 48 articles, books, or book chapters, published since 1990 were identified based upon the use
of the following words in the publication’s title: “attention deficit” and “diagnose” or “diagnosis” or
“assessment.” Next the authors evaluated whether the source was specifically intended to give practical guidance to health and/or mental health care professionals regarding how to diagnose AD/HD in
childhood (i.e., did the publication have as one of its goals instructing readers on how to diagnose this
disorder). From these selection procedures, 42 sources were identified as being appropriate for this
literature review. Analysis of these sources revealed that a variety of different diagnostic procedures are
recommended. However, most of these can be classified into one of several specific categories. Table
1 provides a summary of these diagnostic procedure categories and the sources that advocate their use.
Table 2 provides an overview of the questions addressed by each procedure and offers examples of
Similarly, intellectually gifted children may demonstrate inattentiveness when they are placed in under-stimulating academic environments (APA, 2000). In the case of gifted children, obtaining historical information from
several informants may help define their ability to regulate behavior across settings and help to determine if
behaviors are isolated to the school or if they meet the multiple setting criteria.
Diagnosis of AD/HD in Childhood
77
TABLE 1. AD/HD Diagnostic Procedures Recommended in Recent Publications
RS
INT
L/T
DO
ME
SR
1. Am. Acad. of Pediatrics (2000)
3
3
3
3
2. Anastopoulos & Shelton (2001)
3
3
3
3
3
3. Atkins & Pelham (1991)
3
3
4. Barkley (1990)
3
3
3
3
3
3
5. Barkley (1991)
3
3
3
3
6. Barkely (1997b)
3
3
3
3
7. Barkley (1998)
3
3
3
3
3
8. Barkley (2006)
3
3
3
3
3
9. Brown (2000)
3
3
3
3
10. Burcham & DeMers (1995)
3
3
3
3
11. Casat et al. (2001)
3
3
3
3
3
12. Cipkala-Gaffin (1998)
3
3
3
3
3
13. Detweiler et al. (1999)
3
3
3
3
14. DuPaul et al. (1991)
3
3
3
3
15. DuPaul & Stoner (1994)
3
3
3
16. Guevremont & Barkley (1992)
3
3
3
3
17. Guevremont et al. (1990)
3
3
3
3
18. Hardy et al. (2004)
3
3
3
19. Hechtman (2000)
3
3
3
3
20. Hinshaw (1994)
3
3
3
3
21. Leach & Brewer (2005)
3
3
3
3
22. Learner et al. (1995)
3
3
3
23. Martin (2003)
3
3
24. Meyer (1999)
3
3
3
3
3
3
3
3
3
3
3
PA
3
3
78
The California School Psychologist, 2007, Vol. 12
25. Nahlik (2004)
3
3
3
26. Oesterheld et al. (2003)
3
3
3
27. Pelham et al. (2005)
3
3
3
28. Quinlan (2000)
3
3
3
3
29. Quinn (1997)
3
3
3
3
30. Rapport (1993)
3
3
3
3
31. Robin (1998)
3
3
3
3
3
32. Root & Resnick (2003)
3
3
3
3
3
3
33. Schaughency & Rothlind (1991)
3
3
3
34. Searight et al. (1995)
3
3
3
35. Shelton & Barkley (1994)
3
3
3
3
36. Shelton & Barkley (1995)
3
3
3
3
37. Silver (1999)
3
3
3
38. Silver (2004)
3
3
3
39. Slomka (1998)
3
3
3
3
40. Swanson & Smith (1996)
3
3
3
3
3
41. Wolraich et al. (2005)
3
3
3
3
100%
98%
90%
68%
34%
24%
Totals
3
3
3
7%
specific techniques. As can be seen, the four nearly universally recommended procedures are rating
scales, interviews, laboratory/psychological assessments, and direct behavioral observations. Thus, these
procedures will be discussed first. Subsequently, a discussion of other less frequently recommended
procedures, including medical evaluations, school record reviews, and peer ratings is provided.
Rating Scales
Rating scales are the most widely advocated procedure for evaluating children with AD/HD, with
100% of the sources reviewed for this paper endorsing their use. As stated by Hinshaw (1994), “…rating
scales are an indispensable element of the assessment of children with suspected attention deficits and
hyperactivity” (p. 32). In some cases, rating scales can effectively differentiate children with AD/HD
from others without this disorder (Barkley, 1998). In addition, they are time and cost effective (Anastopoulos & Shelton, 2001). While for the school psychologist many other data sources are readily accessible, for the diagnostician not based in a school setting, rating scales are generally the only practical
Diagnosis of AD/HD in Childhood
79
TABLE 2. Summary of Recommended AD/HD Diagnostic Procedures
Procedure
Diagnostician
Sample Techniques
Sample Questions
Rating Scales
Mental Health
Professional
Conners’ (1997)
Rating Scales
Are AD/HD
symptoms present?
Educational
Specialist
Behavior
Assessment
System for
Children 2
(Reynolds &
Kamphaus, 2004)
How deviant are
symptoms for the
norm?
Structured
interview
techniques
(e.g., Diagnostic
Interview Schedule
for Children [Shaffer
et al., 1996])
Are AD/HD
symptoms present?
Parent, Teacher,
Child Interviews
Mental Health
Professional
Medical
Professional
Unstructured
interview
techniques
Semi-structured
interviews (e.g.,
developmental &
health history)
Are there comorbid
conditions?
When was the
onset of AD/HD
symptoms?
How long have
symptoms been
present?
Is the environment a
factor?
Is there a family
history of AD/HD?
Is the
developmental/
medical history
significant?
Are there comorbid
conditions?
Direct Behavioral
Observations
Mental Health
Professional
Observation of test
taking behavior
Educational
Specialist
Classroom and playground
observations
Does the child
display AD/HD
behavioral
symptoms?
Does the child
display AD/HD
symptoms?
80
The California School Psychologist, 2007, Vol. 12
Psychological
Pscho-Educational
Tests
Mental Health
Professional
Educational
Specialist
Executive
functioning tests
(e.g., Test of
Everyday Attention
for children [Manly
et al., 1999])
Intelligence testing
Medical Evaluation
Neuroimaging
What is the child’s
ability level?
Achievement testing
What is the child’s
academic level?
Pediatrician
Medical interview
General Practitioner
Psychiatrist
Physical
examination
Are symptoms
secondary to a
medical condition?
Medical Specialist
PET
Other medical tests
as indicated
MRI
fMRI
SPECT
School Records
Does the child
demonstrate
neuro-psychological
or processing
deficits?
Is there abnormal
brain functioning
during tasks that
require planning,
attention, and
impulse control?
Mental Health
Professional
Cumulative file
review
Is there a history of
AD/HD symptoms?
Educational
Specialist
School work sample
review
When was the onset
of AD/HD
symptoms?
way to obtain information from classroom teachers, since more time-consuming clinical interviews and
observations may be difficult to arrange (Wender, 2004).
Rating scales provide a structured format for documenting the presence and degree of AD/HD
symptoms using a normative frame of reference (Anastopoulos & Shelton, 2001; Burcham & DeMers,
1995; Landau & Burcham, 1995; Nahlik, 2004). Additionally, rating scales also allow the diagnostician
to obtain information used in assessment of treatment effectiveness (Landau & Burcham, 1996). In other
words, rating scales may be administered during the diagnostic process and again, later, when treatment
has been initiated to determine the effect of intervention.
Several rating scales, such as the ADHD Symptoms Rating Scale (ADHD-SRS; Holland, Gimpel, &
Merrill, 2003), ADHD Rating Scale-IV (DuPaul, Power, Anastopoulos, & Reid, 1998), and the Conners’
Rating Scales (Conners, 1997), assess symptom severity specific to AD/HD. General purpose rating
scales, such as the Behavioral Assessment System for Children-Second Edition (BASC-2; Reynolds &
Kamphaus, 2004) and the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001), are also
available. These measures are helpful because they provide information about symptoms related to AD/
Diagnosis of AD/HD in Childhood
81
HD and other frequently occurring disorders or comorbid conditions (Anastopoulos & Shelton, 2001;
Landau & Bircham, 1996). As such, they assist in the consideration of alternative hypotheses for AD/HDlike symptoms. Both types of scales (i.e., symptom-specific and general purpose) are recommended.
Despite their utility, as with all other diagnostic procedures, a rating scale should never be used
in isolation to diagnose AD/HD (Nahlik, 2004). According to Rucklidge and Tannock (2002), this is
particularly important, since some rating scales, one example being the Brown ADD Scale, may be
most useful for screening out AD/HD rather than diagnosing it due to the measure’s low sensitivity (i.e.,
the probability that a child with AD/HD is accurately identified as having AD/HD is low; Rucklidge &
Tannock, 2002).
Different raters, responding to the same rating scale, often provide different results. For example,
agreement between parent and teacher ratings on diagnosis and actual symptoms has been shown to be
low, possibly due to observations occurring in distinct settings (Wolraich et al., 2004). Yet comparison
between raters may show discrepancy related to subtypes, they often correspond at an overall diagnostic
level (Mitsis, McKay, Schulz, Newcorn, & Halperin, 2000).
Similar findings have been obtained in relation to self-report and teacher report measures of behavior
in the diagnosis of AD/HD. Self-report behavioral ratings have been found to underestimate activity
levels as well as attentional problems compared to parent and teacher report (Danckaerts, Heptinstall,
Chadwick, & Taylor, 1999; Smith, Pelham, Gnagy, Molina, & Evans, 2000). Finally, use of teacher rating
scales for adolescents appears questionable. Possibly due to the large size of many high school classes
and the relative brevity of each class meetings, teacher agreement is lower for adolescent behavior
(Molina, Pelham, Blumenthal, & Galiszewski, 1998).
Interviews
Another frequently recommended AD/HD diagnostic procedure is the clinical interview, with 98%
of the sources reviewed for this paper endorsing their use. Clinical interviews are typically conducted
by a mental health professional, such as a psychologist or social worker, or by medical professionals,
including pediatricians or family practitioners. Interviews may be conducted with parents, teachers,
and/or the student who was referred for assessment. According to Root and Resnick (2003) they are “the
most important part of the evaluation process” (p. 36). According to Hinshaw (1994) and Nahlik (2004),
clinical interviews address diagnostic issues including: (a) Whether AD/HD symptoms are present and
under what conditions are they observed, (b) The onset of the AD/HD symptoms?, (c) The duration of
symptom presentation, (d) The presence of environmental factors, (e) Family history of AD/HD, (f) The
developmental history relative to AD/HD, (g) The presence of learning difficulties, and (h) The presence
of emotional difficulties. Although relatively costly and time consuming, an interview can expand upon
the results of behavioral rating scales since it “provides the opportunity to query the informant and make
a better judgment about the symptom in question” (Wender, 2004, p. 48).
Formats for clinical interviews include structured, semi-structured, and unstructured procedures. A
structured interview is conducted using an interview schedule, or a set of questions that are designed to
probe areas of specific concern. Interview questions are read in the order they are written and responses
are recorded categorically (Anastopoulos & Shelton, 2001). Structured diagnostic interviews have been
defined as the “preferential method of diagnosis because of their exhaustive and direct correspondence
to the DSM criteria” (McGrath, Handwerk, Armstrong, Lucas, & Freman, 2004, p. 350). Semi-structured
interviews incorporate a specific set of questions while allowing the clinician to engage in open-ended
inquiries to obtain further information about relevant points. An unstructured interview is conducted
82
The California School Psychologist, 2007, Vol. 12
through the use of open-ended questioning.
Parent interviews. The parent interview is considered an indispensable part of the AD/HD evaluation (Barkley, 1998; Landau & Burcham, 1996). In fact, it has been asserted that the “diagnosis of
ADHD and/or other psychiatric comorbidities should be based largely on the information gained in the
interviews” (Nahlik, 2004, p. 2). This is because parents are typically able to provide the most detailed
and ecologically relevant information to the assessment process (Barkley, 1998). Interview data can
effectively guide development of a holistic picture of the child and his or her experiences, filling in the
gaps that may remain subsequent to use of other assessment techniques.
In addition to offering information about AD/HD symptoms, semi-structured parent interviews
can provide relevant information about a child’s developmental, school, family, and psychiatric history
(Barkley, 1998; Hinshaw, 1994). According to the DSM-IV-TR diagnostic criteria, specific data regarding
the onset of AD/HD symptoms and family history of AD/HD should be obtained (APA, 2000) and,
accordingly, this information can be provided by the child’s parents or caregivers. While the parent
interview yields critically important information, it can be subject to bias and therefore not provide a true
picture of the student’s functioning (Guevremmont, DuPaul, & Barkley, 1990). Thus, triangulation of
data sources is required. In addition, it is important to note that Barkley (1998) asserts that the reliability
and validity of parent interviews depends largely on the clinician’s ability to conduct the interview and
to ask relevant and specific questions.
Teacher interviews. Of equal importance to the AD/HD assessment and diagnostic process, is the
teacher interview. Both accurate diagnosis and evaluation of treatment effects have been reported to
be dependent on teacher observation of student behavior. Thus, the teacher interview is critical given
the significant amount of time spent at school and the importance of attention in relation to academic
success (Molina et al., 1998). As is the case with the parent interview, the teacher interview is an important complement to teacher behavior rating scales (Molina, Smith, & Pelham, 2001), and for the school
psychologist is a readily accessible data source.
Student interviews. The student interview may also be incorporated into the AD/HD diagnostic
process. Information obtained directly from children can be helpful, particularly if other psychopathologies, such as depression or psychosis, are suspected. However, regarding externalizing behavior disorders
(e.g., AD/HD) it is important to acknowledge that children commonly report fewer symptoms than do
adults (Hart, Lahey, Loeber, & Hanson, 1994; Volpe, DuPaul, Loney, & Salisbury, 1999). Nevertheless,
at least in the case of adolescents, interviews may be particularly important in obtaining their collaboration in acceptance of the AD/HD diagnosis, as well as ensuring treatment compliance (Nahlik, 2004).
Laboratory/Psychological Testing
The use of laboratory and/or psychological tests is also common in the diagnosis of AD/HD, with
90% of the sources reviewed for this paper endorsing their use. According to Anastopoulos and Shelton
(2001), standardized, norm-referenced measures are widely used due to “concerns that interviews and
rating scales are not objective, are not pure measures of attention, and do not permit a component analysis of the construct of attention” (p. 106). However, Barkley (1998) argues that “the fact that a series of
tests is characterized as neuropsychological does not guarantee it actually taps into relevant neuropsychological processes” (p. 299). In other words, just because a test is purported to measure attention, does
not mean it is a valid and reliable measure of a child’s ability to focus, inhibit and/or sustain responses
in natural or clinic settings.
Cognitive assessment. A significant amount of research has been conducted regarding the relevance
Diagnosis of AD/HD in Childhood
83
of IQ testing to the AD/HD diagnosis. Findings have been mixed, with some research indicating that children with AD/HD are likely to perform several points lower than peers on IQ tests (Doyle, Biederman,
Seidman, Weber, & Faraone, 2000; Fischer, Barkley, Fletcher, & Smallish, 1990). Others have found
that AD/HD and IQ function independent of one another (Shuck & Crinella, 2005). While controversy
about the relationship between intelligence test results and AD/HD persists, determining a child’s IQ
remains important given the DSM-IV-TR (APA, 2000) diagnostic criterion that rules out the diagnosis if
a student’s level of hyperactivity, impulsivity and/or inattention is commensurate with his or her developmental level.
Psycho-educational assessments. Psycho-educational assessment can provide unique information to
the diagnostic process as it provides information related to problems that can be either associated with
AD/HD or that might serve as alternative explanations for the symptoms. For example, in some cases
comorbid learning (Barkley, 1998; Barry, Lyman, & Klinger, 2002), language (Barkley, 1998; Cohen et
al., 2000; Riccio & Jemison, 1998) and processing (Bedard, Martinussen, Ickowicz, & Tannock, 2004)
disorders may best account for the behaviors often associated with AD/HD. Consequently, some have
recommended tests of academic achievement and other measures that assess these abilities be a part of
the diagnostic process (Selikowitz, 2004).
Neuropsychological assessments. Some sources reviewed suggested neuropsychological tests to
be sensitive to aberrant cognitive processes associated with AD/HD (e.g., inattention and impulsive
response patterns; Dige & Wik, 2005; Kaplan & Stevens, 2002). The neuropsychological evaluation
may also contribute to understanding children with attentional difficulties who do not meet DSM-IV-TR
criterion for a diagnosis of AD/HD (Baron, 2004).
Research has offered some support for the sensitivity of certain neuropsychological tests as measures
of attention skills (Roth & Saykin, 2004). For example, measures such as the Wide Range Assessment
of Memory and Learning, Second Edition (WRAML 2; Sheslow & Adams, 2003), the California Verbal
Learning Test, Children’s Version (CVLT-C; Delis, Kramer, Kaplan, Ober, & Fridlund, 1998), and the
Wisconsin Card Sorting Test (WCST; Heaton, Chelune, Talley, Kay, & Curtis, 1993) have been suggested
to be sensitive to neuropsychological functions including attention span, sustained attention, both singleand repeated-trial learning, response inhibition, and working memory. In addition, individually administered assessment batteries designed specifically to measure executive functions related to AD/HD have
recently been developed. These include the Delis Kaplan Executive Functioning System (DKEFS; Delis,
Kaplan, & Kramer, 2001) and the Test of Everyday Attention for Children (TEA-Ch; Manly, Robertson,
Anderson, & Nimmo-Smith, 1999). These tools are designed to measure functions associated with AD/
HD, such as sustained attention and vigilance and response inhibition.
Computerized neuropsychological assessments designed to evaluate specific aspects of AD/HD were
also recommended by some of the sources reviewed. Among these are the Conners’ Continuous Performance Test (Conners, 2000), the Test of Variables of Attention (TOVA; Greenberg, Corman, & Kindschi,
2001), and the Gordon Diagnostic System (Gordon, McClure, & Aylward, 1996). These tools purport to
evaluate a child’s vigilance and sustained attention using computerized formats and are commonly used
in evaluation of suspected AD/HD. These measures of sustained attention typically require a child to
listen to or look at a series of numbers or letters and respond (often by pressing a button whenever certain
stimuli or pairs of stimuli are presented). Scores are usually calculated according to correct responses,
errors of omission (correct answers that were overlooked by the child), and errors of commission (incorrect answers that were selected by the child).
While neuropsychological testing has been advocated by some to be an important part of the AD/
84
The California School Psychologist, 2007, Vol. 12
HD diagnostic process, it is important to acknowledge that the use of these tests as part of the assessment
and diagnostic process is controversial. Because the sensitivity and specificity of neuropsychological
measures has been brought into question, they should not be utilized in isolation (Barkley, 1998; Doyle
et al., 2000; McGee, Clark, & Symons, 2000). To date researchers suggest that neuropsychological
assessment tools cannot accurately differentiate between subtypes of AD/HD (Geurts, Verté, Oosterlaan,
Roeyers, & Sergeant, 2005), although they have been reported to effectively distinguish AD/HD children
from those with other diagnoses (Dunn & Kronenberger, 2003).
Direct Behavioral Observation
As part of the assessment and diagnosis of AD/HD, direct observations are frequently recommended, with 68% of the sources reviewed for this paper endorsing their use. Direct observation is typically conducted by either mental health or educational professionals and frequently occurs in the school
setting. They are designed to compliment rating scale and clinical interview data (Parker, 1992), as well
as to assess interpersonal and social skills (Hinshaw, 1994).
Despite its relatively high cost (when compared to rating scales and interviews), direct observation
of behavior is often recommended because of its importance to the differential diagnosis of AD/HD
(Anastopoulos & Shelton, 2001). In fact, Barkley (1990) asserted that observations of student behavior
is “…likely to prove as useful as (or more useful than) any other sources of information in the evaluation, because they directly assess the actual AD/HD symptoms of concern to the child’s teacher” (p.
339). However, in the 2006 edition of his book, Attention Deficit Hyperactivity Disorder: A Handbook
for Diagnosis and Treatment, Barkley argues that formal behavioral coding is not practical. He states,
“Although a number of studies support the benefit of incorporating structured classroom observations
into the diagnostic process, they are not enough to justify the considerable cost and effort they involve”
(p. 383). However, it is important to acknowledge that Barkley is a clinician and not a school-based
professional. Unlike most clinical psychologists, the school psychologist has easy access to this important data source. Thus, his criticism of the use of direct behavioral observation may not be applicable to
the school psychologist.
If a behavioral observation is included in the AD/HD diagnostic process, it may be informal and
unsystematic, or formal and systematic. When conducting a behavioral observation in a natural setting,
such as the child’s classroom, formal procedures include defining behaviors of concern, observing them
at regular intervals (e.g., internal time sampling procedures), calculating the rate of a behavior, and
comparing these rates to those of non-referred peers. Formal structured coding systems have been developed. For example, the BASC offers a paper-and-pencil and a Portable Observation Program (Reynolds
& Kamphaus, 2004).
Tempering the possible utility of direct behavioral observations is the finding that direct behavioral
observation has been shown to be highly correlated with teacher behavior rating scale data. Given this
finding, it has been argued by some clinicians that this assessment technique often does not provide
unique assessment data (Lett & Kamphaus, 1997), and given its relatively high cost may not be necessary in all cases. However, it should be acknowledged that behavioral observations are an important
diagnostic strategy for confirming the AD/HD diagnosis and monitoring AD/HD symptoms, and given
that they are readily accessible to the school psychologist, they should be employed whenever possible.
Diagnosis of AD/HD in Childhood
85
Medical Evaluations
A medical evaluation was recommended as a component of the AD/HD diagnostic process in 34%
of the sources reviewed for this paper. In fact, some authors have argued that the “best person to make
a diagnosis is a specialist pediatrician with an interest and expertise” in the area (Selikowitz, 2004, p.
123). Others, however, have noted that a medical evaluation is by itself inadequate to diagnose AD/HD
(Barkley, 1990) and that: “Routine physical examinations of children with ADHD frequently indicate
no physical problems and are of little help in diagnosing the condition or suggesting its management”
(Barkley, 2006, p. 360). Perhaps most importantly, the findings of a medical evaluation can support the
diagnostic process, particularly by providing information important to differential diagnosis by ruling
out those relatively rare medical conditions that may be the cause of the AD/HD-like symptoms such as
pinworms and absence seizures (Anastopoulos & Shelton, 2003; Robin, 1998).
When employed in the diagnostic process, the medical evaluation commonly includes a medical
interview and a physical examination. The physician typically assesses for genetic syndromes, neurological abnormalities, gross sensory motor, hearing, vision, and physical impairments (Robin, 1998;
Selikowitz, 2004). It has been suggested that the medical examination is especially critical for children
with histories of a seizure disorder. Approximately 30% of children with a seizure disorder develop
AD/HD, or have its symptoms worsened with anticonvulsants, such as Dilantin or Phenobarbital (Wolf
& Forsythe, 1978). Furthermore, research investigations indicate that children with AD/HD demonstrate
differential brain wave patterns as compared to control subjects (Clarke et al., 2003; Lazzaro, Gordon,
Whitmont, Meares, & Clarke, 2001), and this difference can be detected using electrophysiological
assessment procedures. Given these findings and the significant number of children with seizure disorders who also have AD/HD, it is not surprising that electrophysiological measures (e.g., EEG) are among
the most commonly utilized medical testing procedures in the diagnostic assessment of AD/HD (Loo &
Barkley, 2005).
Although a higher rate of AD/HD is not observed among children with asthma (Daly, Biederman,
& Bostic, 1996), commonly prescribed asthma medications are reported to affect attention span and
may exacerbate a preexisting case of AD/HD (Barkley, 1990; Parker, 1992). Furthermore, Albuterol,
a commonly used inhalant medication for asthma, can cause side effects such as increased heart rate,
tremor, and nervousness, which may be confused with AD/HD symptoms (Robinson & Geddes, 1996)
Diagnostic imaging. The use of diagnostic imaging (e.g., PET [positron emission tomography] scans,
CAT [computed axial tomography] scans, MRIs [magnetic resonance images]) has increased in recent
years, although it remains controversial. None of the sources meeting literature review search criteria
recommended the use of these techniques as part of the diagnostic evaluation. However, another paper
that addressed “executive dysfunction” was located that did identify neuroimaging techniques as a part
of their recommended AD/HD diagnostic process (Roth & Saykin, 2004). Some experts, such as Robin
(1998), have argued that “there is no evidence for the utility” for PET scans, CAT scans, MRIs, regular
or enhanced EEGs in routine clinical assessments (p. 86). Many of the studies using neuroimaging techniques in the assessment and diagnosis of AD/HD have utilized very small samples and, therefore, have
limited generalizability. However, some researchers have suggested these measures to have the ability to
differentiate between AD/HD and non-AD/HD children (Kim, Lee, Shin, Cho, & Lee, D.S., 2002; Kim,
Lee, Cho, & Lee, 2001; Rubia et al., 1999).
From the findings mentioned above, it is not surprising that diagnostic imaging techniques, such
as MRI, CAT scans, and PET scans, are rarely used in clinical diagnoses. However, they have provided
important insights into the brain of the child with AD/HD. For example, MRI studies demonstrate differ-
86
The California School Psychologist, 2007, Vol. 12
ential activation of the left and right prefrontal cortex, left anterior cingulate, basal ganglia, and cerebellum on tasks that require selective attention (Roth, & Saykin, 2004; Schulz et al., 2005; Willis &
Weiler, 2005). Similarly, PET scans have indicated differential neurotransmitter and receptor binding in
the brains of AD/HD children (Jucaite, Fernell, Halldin, Forssberg, & Farde, 2005).
Review of School Records
Examination of school records was also suggested by this literature review to be helpful in the
diagnosis of AD/HD, with 24% of the sources reviewed for this paper endorsing their use. Given that
the DSM-IV-TR (APA, 2000) diagnostic criteria require symptom onset to occur prior to the age of 7,
it is likely the impulse control, hyperactivity, and attention difficulties of children will be documented
throughout their school careers. Review of school records, including report cards and disciplinary histories, may yield information regarding when symptoms were first observed and their severity across time
(Brock, 1999). Additionally, they can provide information related to a child’s work habits, task completion, and academic functioning.
Peer Assessments
Peer nominations and peer ratings are another set of procedures suggested by three sources (7%
of the sources reviewed) to be useful in the diagnosis of AD/HD. The utility of these procedures is
based upon the well-documented social difficulties experience by AD/HD children (Whalen & Henker,
1985) and that the severity of these difficulties is an indicator of later adolescent and adult adjustment
(Hinshaw, 1994; Weiss & Hechtman, 1986). Peer nominations typically require children to nominate
those classmates whom they like the most, and those whom they like the least. Atkins and Pelham (1991)
report that AD/HD children are usually rated as less popular and more disliked then children without
this disorder. Peer ratings, on the other hand, obtain from classmates information regarding specific
behaviors that lead to rejections, neglect, and popularity. Regarding these procedures, Schaughency and
Rothlind (1991) suggest that “. . . peers are able to identify attention problems among their classmates
who are referred for adjustment difficulties and to differentiate among their classmates who are referred
for adjustment difficulties and to differentiate among the externalizing behavior problems of their classmates” (p. 196).
Concluding Comments:
Implications for the School Psychologist
No one diagnostic procedure, or set of procedures, has been identified that will diagnose AD/HD
with perfect reliability. The diagnosis of this disorder is complicated due to the nature of its symptoms, many of which can be attributed to other psychiatric disorders observed in childhood; and among
younger children, to be similar to normative developmental behavior. Given the challenges of making an
accurate differential diagnosis of AD/HD, the importance of working with a multidisciplinary team of
medical and educational professionals, along with the family, the teacher, and the referred child should
be clear.
As discussed at the beginning of this paper there are important legal reasons for the school psychologist being involved in AD/HD diagnoses (i.e., the combination of child find regulations and requirements
that children be assessed in all areas of suspected disability). However, from this literature review it
should be clear that in addition to these legal motivations, there are also practical reasons for the school
Diagnosis of AD/HD in Childhood
87
psychologist being an important part of the AD/HD diagnosis. Specifically, there is no one mental health
professional that has more access to the multiple information sources and diagnostic procedures (i.e.,
rating scales; parent, teacher, and student interviews, psychological testing, and behavioral observations)
considered to be required for an AD/HD diagnosis. For example, the classroom observational data that
some working in clinical practice (e.g., Barkley, 2006) have come to argue are too costly to include in
the standard diagnostic process, are readily accessible to the school psychologist.
While the diagnosis of AD/HD is complicated and perfect diagnostic reliability has yet to be
obtained, from this review of the literature it is clear that there is significant consensus among authorities in the field regarding what the comprehensive evaluation of the child suspected to have AD/HD
should involve. Specifically, it would appear that rating scales, interviews, psychological testing, and
behavioral observations are the most commonly recommended procedures. Awareness of this consensus
can not only help to guide practice, but can also be used to support the ability of the school psychologist
to make this diagnosis. Specifically, it is clear that not only do school psychologists have ready access
to the most frequently recommended diagnostic elements, but they also have as a part of their standard
pre-service preparation training in the use of these techniques (i.e., how to use rating scales, conduct
interviews, administer and interpret psychological tests, and conduct behavioral observations). Thus,
given the appropriate supervised practice, it is argued that the school psychologist is well positioned to
assist in the AD/HD diagnosis. At the very least, knowledge of the consensus regarding elements of the
AD/HD diagnosis will be indispensable when evaluating the adequacy of the many students who present
to their school psychologists with AD/HD diagnoses made by other (typically non-school based) health
and mental health professionals. It is hoped that this paper has provided information that will assist these
school psychologist in critically evaluating these independent AD/HD diagnostic assessments.
References
Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms and profiles. Burlington, VT,
University of Vermont, Research Center for Children, Youth, & Families.
American Academy of Pediatrics. (2000). Clinical practice guideline: Diagnosis and evaluation of the child with
attention-deficit/hyperactivity disorder. Pediatrics, 105, 1158-1170.
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., Text Rev.).
Washington, DC: Author.
Anastopoulos, A., & Shelton, T. (2001). Assessing attention-deficit/hyperactivity disorder. New York: Kluwer
Academic/Plenum Publishers.
Atkins, M. S., & Pelham, W. E. (1991). School-based assessment of attention deficit-hyperactivity disorder. Journal
of Learning Disabilities, 24, 197-204.
Barkley, R. A. (1990). Attention deficit hyperactivity disorder: A handbook for diagnosis and treatment. New York:
Guilford Press.
Barkley, R. A. (1991). Diagnosis and assessment of attention deficit-hyperactivity disorder. Comprehensive Mental
Health Care, 1, 27-43.
Barkley, R. A. (1997a). ADHD and the nature of self-control. New York: Guilford Press.
Barkley, R. A. (1997b). Attention-deficit/hyperactivity disorder. In E. J. Marsh & L. G. Terdal (Eds.), Assessment of
childhood disorders (3rd ed., pp. 71-129). New York: Guilford Press.
Barkley, R. A. (1998). Attention-deficit/hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed.).
New York: Guilford Press.
Barkley, R. A. (2006). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (3rd ed.).
New York: Guilford.
Baron, I. S. (2004). Neuropsychological evaluation of the child. New York: Oxford University Press.
Barry, T., Lyman, R., & Klinger, L. (2002). Academic underachievement and attention deficit/hyperactivity disorder:
The negative impact of symptom severity on school performance. Journal of School Psychology, 40, 259-283.
88
The California School Psychologist, 2007, Vol. 12
Bedard, A. C., Martinussen, R., Ickowicz, A., & Tannock, R. (2004). Methylphenidate improves visual-spatial
memory in children with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and
Adolescent Psychiatry, 43, 260-268.
Brock, S. E. (1999) Diagnosis of attention-deficit/hyperactivity disorder (ADHD) in childhood. The California
School Psychologist, 4, 18-29.
Brown, M. E. (2000). Diagnosis and treatment of children and adolescents with attention-deficit/hyperactivity
disorder. Journal of Counseling & Development, 78, 195-203.
Burcham, B., & DeMers, S. T. (1995). Comprehensive assessment of children and youth with ADHD. Intervention
in School and Clinic, 30, 211-220.
Burke, J. D., Loeber, R., & Lahey, B. B. (2001). Which aspects of ADHD are associated with tobacco use in early
adolescence? Journal of Child Psychology & Psychiatry, 42, 493-502.
Campbell, S. B. (1990). Behavior problems in preschool children. New York: Guilford Press.
Casat, C. D., Pearons, D. A., & Casat, J. P. (2001). Attention-deficit/hyperactivity disorder. In V. H. Booney & A.
Pumariega (Eds.), Clinical assessment of child and adolescent behavior (pp. 263-306). Hoboken, NJ: Wiley.
Cipkala-Gaffin, J. A. (1998). Diagnosis and treatment of attention-deficit hyperactivity disorder. Perspectives in
Psychiatric Care, 34, 18-25.
Clarke, A. R., Barry, R. J., McCarthy, R., Selikowitz, M., Clarke, D. C., & Croft, R. J. (2003). EEG activity in girls
with attention-deficit/hyperactivity disorder. Clinical Neurophysiology, 114, 319-328.
Cohen, N. J., Vallance, D. D., Barwick, M., Im, N., Menna, R., Horodezky, N. B., & Isaacson, L. (2000). The
interface between ADHD and language impairment: An examination of language, achievement, and cognitive
processing. Journal of Child Psychology & Psychiatry, 41, 353-362.
Conners, K. (1997). Conners’ Rating Scales –Revised. North Tonawanda, NY: Multi Health Systems.
Conners, K. (2000). Conners’ Continuous Performance Test II. North Tonawanda, NY: Multi Health Systems.
Daly, J. M., Biederman, J., & Bostic, J. Q. (1996). The relationship between childhood asthma and attention deficit
hyperactivity disorder: A review of the literature. Journal of Attention Disorder, 1, 31-40.
Danckaerts, M., Heptinstall, E., Chadwick, O., & Taylor, E. (1999). Self report in attention-deficit and hyperactivity
disorder in adolescents. Psychopathology, 32, 81-92.
Davila, R. R., Williams, M. L., & MacDonald, J. T. (1991, September 16). Memorandum to chief state school
officers re: Clarification of policy to address the needs of children with attention deficit disorders with general
and/or special education. Washington, DC: U.S. Department of Education.
Delis, D., Kaplan, E., & Kramer, J. (2001). The Delis-Kaplan Executive Function System. San Antonio, TX: The
Psychological Corporation.
Delis, D., Kramer, J., Kaplan, E., Ober, B., & Fridlund, A. (1998). California Verbal Learning Test, Children’s
Version. San Antonio, TX: PsychCorp.
Detweiler, R., Hicks, A., & Hicks, M. (1999). A multi-modal approach to the assessment and management of ADHD.
In P. Cooper & K. Bilton (Eds.), ADHD research, practice and opinion. London: Whurr.
Dige, N., & Wick, G. (2005). Adult attention deficit hyperactivity disorder identified by neuropsychological testing.
International Journal of Neuroscience, 11, 169-183.
Doyle, A. E., Biederman, J., Seidman, L. J., Weber, W., & Faraone, S. V. (2000). Diagnostic efficiency of neuropsychological test scores for discriminating boys with and without ADHD. Journal of Consulting Clinical
Psychology, 68, 477-488.
Dunn, D., & Kronenberger, W. (2003). Attention-deficit/hyperactivity disorder in children and adolescents. Neurologic Clinics, 21, 933-940.
DuPaul, G. J., Guevremont, D. C., & Barkley, R. A. (1991). Attention deficit-hyperactivity disorder in adolescence:
Critical assessment parameters. Clinical Psychology Review, 11, 231-245.
DuPaul, G. J., & Stoner, G. (1994). ADHD in the schools: Assessment and intervention strategies. New York: Guilford Press.
DuPaul, G. J., Power, T. J., Anastopoulos, A. D., & Reid, R. (1998). ADHD rating scale-IV: Checklists, norms, and
clinical interpretations. New York: Guilford Press.
Fischer, M., Barkley, R. A., Fletcher, K., & Smallish, L. (2002). The adolescent outcome of hyperactive children
diagnosed by research criteria, II: Academic, attentional, and neuropsychological status. Journal of Consulting
and Clinical Psychology, 58, 580-588.
Geurts, H. M., Verté, S., Oosterlaan, J., Roeyers, H., & Sergeant, J.A. (2005). ADHD subtypes: Do they differ in
their executive functioning profile? Archives of Clinical Neuropsychology, 20, 457-477.
Gimpel, G. A., & Kuhn, B. R. (1998).Maternal report of attention deficit hyperactivity disorder symptoms in
preschool children. Child: Care, Health, and Development, 26, 163-175.
Diagnosis of AD/HD in Childhood
89
Gordon, M., McClure, F. D., & Aylward, G. P. (1996). The Gordon Diagnostic System. DeWitt, NY: GSI Publications.
Greenberg, L. M., Corman, C. L., Kindschi, C. L. (2001). Test of Variables of Attention. Los Alamitos, CA: Universal
Attention Disorders.
Guevremont, D. C., & Barkley, R. A. (1992). Attention deficit-hyperactivity disorder in children. In S. R. Hooper,
G. W. Hynd, & R. E. Mattison (Eds.), Child psychopathology: Diagnostic criteria and clinical assessment (pp.
137-177). Hillsdale, NJ: Lawrence Erlbaum.
Guevremont, D. C., DuPaul, G. J., Barkley, R. A. (1990). Diagnosis and assessment of attention-deficit/hyperactivity disorder in children. Journal of School Psychology, 28, 51-78.
Hardy, E., Warmbrodt, L., & DeBasio, N. O. (2004). Diagnosis and treatment of children with attention deficit
disorder. Clinical Excellence for Nurse Practioners, 8, 126-134.
Hart, E. L., Lahey, B. B., Loeber, R., & Hanson, K. S. (1994). Criterion validity of informants in the diagnosis of
disruptive behavior disorders in children: A preliminary study. Journal of Consulting and Clinical Psychology,
62,410-414.
Heaton, R. K., Chelune, G. J., Talley, J. L., Kay, G.C., & Curtis, G. (1993). Wisconsin Card Sorting Test. Odessa,
FL: Psychological Assessment Resources.
Hechtman, L. (2000). Assessment and diagnosis of attention-deficit/hyperactivity disorder. Child and Adolescent
Psychiatric Clinics of North America, 9, 481-498.
Hinshaw, S. P. (1994). Attention deficits and hyperactivity in children. Thousand Oaks, CA: Sage.
Hoff, K. E., Doepke, K., & Landau, S. (2002). Best practices in the assessment of children with attention deficit/
hyperactivity disorder: Linking assessment to intervention. In A. Thomas & J. Grimes (Eds.), Best practices in
school psychology IV (pp. 1129-1150). Bethesda, MD: National Association of School Psychologists.
Holland, M. L., Gimpel, G. A., & Merrill, K. W. (2003). ADHD Symptoms Rating Scale. Lutz, FL: Psychological
Assessment Resources.
Jucaite, A., Fernell, E., Halldin, C., Forssberg, H., & Farde, L. (2005). Reduced midbrain dopamine transporter
binding in male adolescents with attention deficit/hyperactivity disorder: Association between striatal dopamine
markers and motor hyperactivity. Biological Psychiatry, 57, 229-238.
Kaplan, R. F., & Stevens, M. C. (2002). A review of adult ADHD: A neuropsychological and neuroimaging perspective, CSN Spectrum, 7, 355-362.
Kim, B. N., Lee, J. S., Cho, S. C., & Lee, D. S. (2001). Methylphenidate increased regional cerebral blood flow in
subjects with attention-deficit/hyperactivity disorder. Yonsei Medical Journal, 42, 19-29.
Kim, B. N., Lee, J. S., Shin, M. S., Cho, S. C., & Lee, D. S. (2002). Regional cerebral perfusion abnormalities in
attention deficit/hyperactivity disorder: Statistical parametric mapping analysis. European Archives of Psychiatry and Clinical Neuroscience, 252, 219-225.
Landau, S., & Burcham, B. G. (1995). Best practices in the assessment of children with attention disorders. In A.
Thomas & J. Grimes (Eds.), Best practices in school psychology – III (pp. 817-829). Washington, DC: National
Association of School Psychologists.
Lazzaro, I., Gordon, E., Whitmont S., Meares, R., & Clarke, S. (2001). The modulation of late component event
related potentials by pre-stimulus EEG theta activity in ADHD. International Journal of Neuroscience, 107,
247-264.
Leach, D. J., & Brewer, D. F. (2005). The assessment and diagnosis of attention deficit hyperactivity disorder
(ADHD) in children. In R. F. Waugh (Ed.), Frontiers in educational psychology (pp. 281-307). Hauppauge, NY:
Nova Science.
Learner, J. W., Lowenthal, B., & Learner, S. R. (1995). Attention deficit disorders: Assessment and teaching.
Belmont, CA: Brooks/Cole.
Lett, N. J., & Kamphaus, R. W. (1997). Differential validity of the BASC Student Observation system and the BASC
Teacher Rating Scale. Canadian Journal of School Psychology, 13, 1-14.
Levy, F., Hay, D., Bennett, K., & McStephen, M. (2004). Gender differences in ADHD subtype comorbidity. Journal
of the American Academy of Child and Adolescent Psychiatry, 44, 369-376.
Loo, S., & Barkley, R. (2005). Clinical utility of EEG in attention deficit hyperactivity disorder. Applied Neuropsychology, 12, 64-76.
Loughran, S. B. (2003). Agreement and stability of teacher rating scales for assessing ADHD preschoolers. Early
Childhood Education Journal, 30, 247-253.
Manly, T., Robertson, I. H., Anderson, V., & Nimmo-Smith, I. (1999). The Test of Everyday Attention for Children.
Bury St. Edmunds, UK: Thames Valley Test Company.
Martin, G. L. (2003). Current developments in the assessment of attention deficit hyperactivity disorder. Marriage
& Family: A Christian Journal, 6, 171-194.
90
The California School Psychologist, 2007, Vol. 12
McGee, R. A., Clark, S. E., & Symons, D. K. (2000). Does the Conners’ Continuous Performance Test aid in ADHD
diagnosis? Journal of Abnormal Child Psychology, 28, 415-424.
McGrath, A., Handwerk, M., Armstrong, K., Lucas, P., & Freman, C. (2004). The validity of the ADHD section of
the Diagnostic Interview Schedule for Children. Behavior Modification, 28, 349-374.
Meyer, A. (1999). Attention-deficit/hyperactivity disorder: Diagnosis and treatment. In S. N. Madu, P. K. Baguma,
& A. Pitz (Eds.), Cross-cultural dialogue on psychotherapy in Africa (pp. 86-98). Sovenga, South Africa: UNIN
Press.
Miranda, A., Presentación, M. J., & Soriano, M. (2002). Effectiveness of a school-based multicomponent program
for the treatment of children with ADHD. Journal of Learning Disabilities, 35, 546-562.
Mitsis, E., McKay, K., Schulz, K., Newcorn, J., & Halperin, J. (2000). Parent-teacher concordance for DSM-IV
attention-deficit/hyperactivity disorder in a clinic-referred sample. Journal of the American Academy of Child
and Adolescent Psychiatry, 39, 308-313.
Molina, B. S. G., Pelham, W. E., Blumenthal, J., & Galiszewski, E. (1998). Agreement among teachers’ behavior
ratings of adolescents with a history of attention deficit hyperactivity disorder. Journal of Clinical Child
Psychology, 27, 330-339.
Molina, B. S., Smith, B. H., & Pelma, W. E. (2001). Factor structure and criterion validity of secondary school
teacher ratings of ADHD and ODD. Journal of Abnormal Child Psychology, 29, 71-82.
Nahlik, J. (2004). Issues in the diagnosis of attention-deficit/hyperactivity disorder in adolescents. Clinical Pediatrics, 43, 1-10.
Oesterheld, J. R., Shader, R. I., & Wender, P. H. (2003). Diagnosis and treatment of attention deficit hyperactivity
disorder in youth and adults. In R. I. Shader (Ed.), Manual of psychiatric therapeutics (3rd ed., pp. 346-364).
Philadelphia, PA: Lippincott Williams & Wilkins.
Parker, H. C. (1992). The ADD hyperactivity handbook for schools. Plantation, FL: Impact Publications.
Pelham, W. E., Fabiano, G. A., & Massetti, G. M. (2005). Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 449-476.
Power, T., Costigan, T., Eiraldi, R., & Leff, S. (2004). Variations in anxiety and depression as a function of ADHD
subtypes defined by DSM-IV: Do subtype differences exist or not? Journal of Abnormal Child Psychology, 32,
27-37.
Quinlan, D. M. (2000). Assessment of attention-deficit/hyperactivity disorder and comorbidities. In T. E. Brown
(Ed.). Attention-deficit disorders and comorbidities in children, adolescents, and adults (pp. 455-507). Washington, DC: American Psychiatric Publishing.
Quinn, P. O. (1997). Attention deficit disorder: Diagnosis and treatment from infancy to adulthood. New York:
Brunner/Mazel.
Rapport, M. D. (1993). Attention deficit hyperactivity disorder. In T. H. Ollendick & M. Hersen (Eds.), Handbook
of child and adolescent assessment (pp. 269-291). Needham Heights, MA: Allyn & Bacon.
Reynolds, C., & Kamphaus, R. (2004). Behavior Assessment System for Children (2nd ed.). Circle Pines, MN: American Guidance Service.
Robin, A. L. (1998). ADHD in adolescents: Diagnosis and treatment. New York: Guilford Press.
Robinson, D. S., & Geddes, D.M. (1996). Inhaled corticosteroids: Benefits and risks. Journal of Asthma, 33, 5-16
Root, R., & Resnick, R. (2003). An update on the diagnosis and treatment of attention deficit/hyperactivity disorder
in children. Professional Psychology: Research & Practice, 34, 34-41.
Roth, R., & Saykin, A. (2004). Executive dysfunction in attention-deficit/hyperactivity disorder: Cognitive and
neuroimaging findings. Psychiatric Clinics of North America, 27, 83-96.
Riccio, C. A., & Jemison, S. J. (1998). ADHD and emergent literacy: Influence of language factors. Reading &
Writing Quarterly, 14, 43-59.
Rubia, K., Overmeyer, S., Taylor, E., Brammer, M., Williams, S.C., Simmons, A., et al. (1999). Hypofrontality in
attention-deficit/hyperactivity disorder during higher-order motor control: A study with functional MRI. American Journal of Psychiatry, 156, 891-896.
Rucklidge, J. J., & Tannock, R. (2002). Validity of the Brown ADD Scales: An investigation in a predominantly
inattentive ADHD adolescent sample with and without reading disabilities. Journal of Attention Disorders, 5,
155-164.
Schaughency, E. A., & Rothlind, J. (1991). Assessment and classification of attention deficit hyperactivity disorders.
School Psychology Review, 20, 187-202.
Searight, H. R., Nahlik, J. E., & Campbell, D. C. (1995). Attention-deficit/hyperactivity disorder: Assessment, diagnosis, and management. Journal of Family Practice, 40, 270-279.
Selikowitz, M. (2004). ADHD: The facts. Oxford: Oxford University Press.
Shaffer, D., Fisher, P., Dulcan, M., Davies, M., Piacentini, J., Schwab-Stone, M., et al. (1996). The NIMH Diag-
Diagnosis of AD/HD in Childhood
91
nostic Interview Schedule for Children Version 2.3 (DISC-2.3): Description, acceptability, prevalence rates,
and performance in the MECA Study. Journal of the American Academy of Child Adolescent Psychiatry, 35,
865-877.
Shelton, T. L., & Barkely, R. A. (1994). Critical issues in the assessment of attention deficit disorders in children.
Topics in Language Disorder. Special Issue: ADD and its Relationship to Spoken and Written Language, 14,
26-41.
Shelton, T. L., & Barkley, R. A. (1995). The assessment and treatment of attention-deficit/hyperactivity disorder in
children. In M. C. Roberts (Ed.), Handbook of pediatric psychology (2nd ed., pp. 633-654). New York: Guilford
Press.
Sheslow, D. & Adams, W. (2003). Wide Range Assessment of Memory and Learning (2nd Ed.). Lutz, FL: Psychological Assessment Resources.
Schulz, K., Tang, C., Fan, J., Marks, D., Newcorn, D., Cheung, A., & Halperin, M. (2005). Differential prefrontal
cortex activation during inhibitory control in adolescents with and without childhood attention-deficit/hyperactivity disorder. Neuropsychology, 19, 390-402.
Shuck, S., & Crinella, F. (2005). Why children with ADHD do not have low IQs. Journal of Learning Disabilities,
38, 262-280.
Sliver, L. B. (1999). Attention-deficit/hyperactivity disorder: A clinical guide to diagnosis and treatment for health
and mental health professionals (2nd ed.). Washington, DC: American Psychiatric Publishing.
Sliver, L. B. (2004). Attention-deficit/hyperactivity disorder: A clinical guide to diagnosis and treatment for health
and mental health professionals (3rd ed.). Washington, DC: American Psychiatric Publishing.
Slomka, G. (1998). Attention deficit hyperactivity disorder. In P. J. Snyder & P. D. Nussbaum (Eds.), Clinical
Neuropsychology: A pocket handbook for assessment (pp. 124-140). Washington, DC: American Psychological
Association.
Smith, B. H., Pelham, W. E., Gnagy, E., Molina, B., & Evans, S. (2000). The reliability, validity, and unique contributions of self-report by adolescents receiving treatment for attention-deficit hyperactivity disorder. Journal of
Consulting and Clinical Psychology, 68, 489-499.
Swanson, M. E., & Smith, T. E. (1996). Attention deficit hyperactivity disorder: Diagnosis and treatment. In D. K.
Reid, W. P. Hresko, & H. L. Swanson (Eds.), Cognitive approaches to learning disabilities (3rd ed., pp. 113-151).
Austin, TX: PRO-ED.
Volpe, R. J., DuPaul, G. J., Loney, J., & Salisbury, H. (1999). Alternative selection criteria for identifying children
with ADHD: Observed behavior and self-reported internalizing symptoms. Journal of Behavioral Disorders, 7,
103-109.
Weiss, G., & Hechtman, L. T. (1986). Hyperactive children grown up. New York: Guilford.
Wender, E. (2004). ADHD symptoms and parent-teacher agreement. Developmental and Behavioral Pediatrics, 25,
48-49.
Whalen, C. K., & Henker, B. (1985). The social worlds of hyperactive (ADDH) children. Clinical Psychology
Review, 5, 447-478.
Willis, W., & Weiler, M. (2005). Neural substrates of childhood attention deficit/hyperactivity disorder: Electroencephalographic and magnetic resonance imaging evidence. Developmental Neuropsychology, 27, 135-182.
Wolf, S. M., & Forsythe, A. (1978). Behavior disturbance, Phobarbital, and febrile seizures. Pediatrics, 61, 728731.
Wolraich, M., Lambert, E. W., Bickman, L., Simmons, T., Doffing, M., & Worley, K., (2004). Assessing the impact
of parent and teacher agreement on diagnosing attention-deficit hyperactivity disorder. Developmental and
Behavioral Pediatrics, 25, 41-47.
Wolraich, M. L., Wibbelsman, C. J., Brown, T. E., Evans, S. W., Gotlieb, E. M., Knight, J. R., Ross, E. C., Shubiner,
H. H., Wender, E. H., & Wilens, T. (2005). Attention-deficit/hyperactivity disorder among adolescents: A review
of the diagnosis, treatment, and clinical implications. Pediatrics, 115, 1734-1746.
The California School Psychologist, Vol. 12, pp. 93 – 105, 2007
Copyright 2007 California Association of School Psychologists
93
Current Educational Practices in Classifying and Serving
Students with Obsessive-Compulsive Disorder
Gail B. Adams &
Thomas J. Smith
Northern Illinois University
Sara E. Bolt
Michigan State University
Patrick Nolten
Illinois Indian Prairie School District 204
Current educational practices for classifying and serving students with mental health disorders such as
obsessive-compulsive disorder (OCD) have been associated with specific problems. These include the
stigma of labeling, misalignment of school-based categories (e.g., E/BD, OHI) with clinical diagnoses,
and concerns regarding the provision of appropriate services to these students. In the present study, Illinois school psychologists completed a survey on current practices for classifying and serving students
with a primary diagnosis of OCD. The results indicated that 0.7% of the students served by school
psychologists had a primary diagnosis of OCD. The majority of these students (74.5%) were served
under IDEA. Of the students receiving services under IDEA, 51.4% were classified under E/BD and
31.8% under OHI. Approximately two-thirds of the students with OCD (67.1%) were educated in less
restrictive settings (e.g., regular classroom with or without resource/part-time special class). School
psychologists’ comments suggested a pattern of ambiguity and uncertainty surrounding the appropriateness of IDEA categories for OCD, concerns regarding the stigma of labeling, and problems related
to providing appropriate services to these students. Response-to-Intervention (RtI) as an alternative to
current evaluation practices is proposed and recommendations for improving traditional categorical
service delivery models when RtI is not implemented are provided.
Keywords: obsessive-compulsive disorder, mental disorders, emotional disturbance, behavioral disorders, other health impaired, labeling (of persons), school psychologists
Based on recent estimates, approximately one-fifth of all students have diagnosable mental disorders (Hoagwood & Johnson, 2003). One mental disorder that recently has appeared on the educational
landscape is obsessive-compulsive disorder (OCD), an anxiety disorder characterized by the presence
of obsessions and/or compulsions. Obsessions are recurring thoughts, ideas, images, or impulses that
are inappropriate, intrusive, and produce considerable distress or anxiety. Compulsions, or rituals, are
purposeful, repetitive, behaviors that individuals perform, either overtly or covertly (i.e., mentally), to
relieve, prevent, or undo the anxiety or discomfort created by the obsessions, or to avert a feared situation. The obsessions or compulsions are time consuming, cause substantial distress, or disrupt the individual’s academic, occupational, or social functioning (American Psychiatric Association [APA], 2000).
Hence, a person with obsessions related to contamination may engage in washing and cleaning rituals for
hours to reduce the anxiety precipitated by contamination fears. (The reader is referred to the Diagnostic
and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision [DSM-IV-TR; APA, 2000] for
more detailed information on OCD.)
National and international studies have yielded heterogeneous results with regard to prevalence
Please send correspondence regarding this manuscript to; Gail Adams, Ed.D.; Department of Teaching and Learning;
Northern Illinois University; DeKalb, IL 60115 or e-mail [email protected]
94
The California School Psychologist, 2007, Vol. 12
estimates of OCD in children and adolescents. Fontenelle, Mendlowicz, and Versiani (2006) reviewed
numerous studies of OCD prevalence rates, including 16 studies of childhood OCD conducted between
1987 and 2003. The reviewers reported a wide range of OCD prevalence rates in children and adolescents (0.0%-4.0%). Potential explanations for this diversity include such variables as the population
studied (e.g., mean age of participants) and methodological decisions (e.g., instrument selected to diagnose OCD, parent vs. child report) (Fontenelle, et al.; Rapoport, Weissman, Narrow, Jensen, Lahey, &
Canino, 2000). In half of these studies, however, reported prevalence rates ranged from 1.9% to 4.0%.
Once thought to be rare, OCD now is considered “a common disorder in both adults and the young”
(Riddle, 1998, p. 92). The prevalence of OCD and other clinical diagnoses among today’s children
demands that school psychologists put into place procedures that promote access to effective mental
health services for these students. Moreover, the link between mental health and optimal student learning
outcomes is strong (Becker & Luthar, 2002), further highlighting the need for school psychologists to
implement measures that enhance mental health among students.
Categorizing Students with Mental Health Disorders
Under the current public education system, students with clinical diagnoses must undergo a separate
educational evaluation process to determine whether they qualify for special education services due to
their disabilities. This process can result in an educational disability category diagnosis that is much
broader and less specific than the clinical diagnosis. For example, the DSM-IV-TR clearly articulates
OCD as a disability category (APA, 2000); however, a student with OCD may qualify for educational
services according to several broader categories such as emotional disorder, behavioral disorder, or other
health impairment. Meeting the traditional legal requirements for proper educational identification and
categorization of students tends to be a time-intensive process (Hosp & Reschly, 2002) and does not
necessarily lead to improved intervention and outcomes among students (Carlburg & Kavale, 1980;
Thurlow & Ysseldyke, 1982). Therefore, it remains questionable whether conventional school-based
evaluation procedures enhance treatment selection and related outcomes among students with mental
health problems. Because maladaptive behavior associated with OCD frequently occurs across a variety
of settings (e.g., home, school, community) and involves clinical treatment (Piacentini, Bergman, Keller,
& McCracken, 2003), collaboration between school psychologists and clinical psychologists may be
particularly helpful in developing effective evaluation and treatment plans.
Another concern associated with categorical diagnoses is labeling, which may lead to a variety
of negative outcomes. First, labels can create the perception that students within a given diagnostic
category are more alike than different, causing those who work with them to neglect their individual
needs and strengths (Bianco, 2005; Smith, 2003). Because students with clinical diagnoses exhibit an
array of symptoms, as evidenced by the frequency with which comorbidity is present among these individuals (APA, 2000), they may require a wide variety of treatment options. Second, stigma often is
associated with disability labels, which not only may have a negative impact on an individual’s selfconcept, but also may alter the responses of individuals who come into contact with a person who
has a mental disability (Susman, 1994). Third, labels have been shown to reduce learning expectations
for students with disabilities, which may have damaging effects on their actual achievement (Foster &
Ysseldyke, 1976). Therefore, it is uncertain whether the perceived benefits of labels (e.g., facilitating
efficient communication among individuals, providing services similar to those shown to be effective for
students displaying comparable behavior; Gallagher, 1976) outweigh the potential drawbacks.
Classifying And Serving Students With OCD
95
Categorizing Students with OCD
OCD can be an insidious disorder, interfering with all aspects of a young person’s functioning
– social, behavioral, and academic (Adams, 2004; Adams, Waas, March, & Smith, 1994; Clarizio, 1991).
In a seminal study that made use of parent and self-report measures to examine the psychosocial functioning of children and adolescents with OCD, Piacentini et al. (2003) found that among 151 children
and adolescents with a primary diagnosis of OCD, almost 90% reported a significant problem in at
least one area of functioning (school/academic, home/family, or social); close to half reported at least
one significant problem in each of these three areas. Furthermore, 47% of parents and 44% of children
reported that OCD caused significant problems in school/academic functioning. Overall, the two most
significant OCD problems reported by parents and children alike were difficulties with concentrating
on schoolwork and doing homework. Similarly, Sukhodolsky, do Rosario-Campus, Scahill, Katsovich, & Pauls (2005) found that children and adolescents with OCD fared significantly more poorly
than children in a control group on several measures of adaptive, family, and emotional functioning. In
addition, youth with OCD performed significantly worse on the school competence scale of the Child
Behavior Checklist than did comparison peers. Children and adolescents with OCD also are at risk for
comorbid psychiatric disorders, including tic disorders, major depression, anxiety disorders (e.g., panic
disorder), disruptive behavior disorders (attention-deficit/hyperactivity disorder [AD/HD] and oppositional defiant disorder), specific developmental disorders, and enuresis (Geller, 2006; Sukhodolsky et
al., 2005; Zohar, 1999), further complicating the behavioral and academic functioning of these students.
Because of the difficulties they experience in school, many children and adolescents with OCD
require school-based accommodations and modifications and/or special education and related services
to facilitate their educational functioning. In the past, some students have received accommodations and
modifications via Section 504 of the Rehabilitation Act of 1973; others have gone through the traditional
educational evaluation process and obtained special education and related services through the Individuals with Disabilities Education Act (IDEA) (Adams, 2004). Students who have received services
under IDEA frequently have been classified under the federal category of “Emotional Disturbance” (ED)
(Adams, 2004). However, parents of children with OCD and other concerned individuals, including
mental health professionals and educational experts in the field of OCD, increasingly have expressed
serious reservations about the ED classification and concomitant label. Indeed, a number of researchers
have advocated that children and adolescents with OCD be identified under the “Other Health Impaired”
(OHI) category (Adams, 2004; Chansky, 2000; Dornbush & Pruitt, 1996).
Several arguments might be offered for such a reclassification. First, many are troubled by the
stigma sometimes associated with the ED label (such terms as “Emotional/Behavioral Disorders,” or
E/BD, may be used within various state systems; Muller & Markowitz, 2004) and the connotations it
may invoke (Tourette Syndrome Association, 2006b). Particularly in cases where the media describe
perpetrators of criminal and/or violent behavior as “emotionally disturbed,” parents or guardians of
children with OCD may have serious and legitimate concerns when the same label is applied to their
child. Second, OCD has a documented neurobiological basis, including a link to the neurotransmitter
serotonin (APA, 2000; Blier, Habib, & Flament, 2006), suggesting that OCD has a physiological, rather
than a behavioral or emotional, basis. Moreover, AD/HD and Tourette Syndrome (TS), two other neurobiologically based disorders, are conditions listed under the IDEA “Other Health Impaired” category (34
C.F.R.§ 300.8[c][9][i]). During the reauthorization of IDEA in 1997, many individuals and organizations (e.g., Children and Adults with Attention-Deficit/Hyperactivity Disorder, or CHADD) advocated
vigorously that AD/HD be included under the Other Health Impaired category (CHADD of Alachua
96
The California School Psychologist, 2007, Vol. 12
County, n.d.). In the final regulations enforcing Part B of IDEA 2004, Tourette Syndrome was listed as a
disability under OHI rather than ED. Immediately after the release of the final regulations, the Tourette
Syndrome Association (TSA) issued a public announcement entitled Major Victory for Children with
Tourette Syndrome: Individuals with Disabilities Education Act to Classify Tourette Syndrome as Other
Health Impaired (2006a). In its announcement, the TSA stated that “many educators…erroneously see
TS as a behavioral or conduct disorder because of the nature of its symptoms and therefore classify these
children under the Emotionally Disturbed (ED) category” (para. 9).
Finally, some individuals may be apprehensive about the ED label for students with OCD because
of its potential ramifications for educational placement. As indicated earlier, labeling may create the
perception that a particular treatment will be effective for most, if not all, students with a given label,
and may lead to inappropriate decisions about how to address their individual needs (Bianco, 2005;
Smith, 2003). One psychiatrist known for his work in childhood OCD expressed concern regarding
students with OCD who are classified under ED and placed in self-contained classrooms for students
with behavioral disorders. He noted that in some cases, the students with behavior disorders who exhibit
externalizing, aggressive, acting out behaviors “…have a feeding frenzy at the expense of the kids with
OCD. It can result in either withdrawal – sometimes out of school – or lashing back (often futility)” (A.
J. Allen, personal communication, September 30, 2002).
Similarly, in its public announcement regarding the inclusion of TS under OHI in IDEA 2004,
the Tourette Syndrome Association stated that classifying students with TS under the ED category
“frequently results in students being placed in programs that are designed for students with emotional
disorders where bullying and teasing generally increase, as does the punishment for their symptoms”
(2006a, para. 9). Although a review of the literature yielded no information regarding the extent to which
this phenomenon occurs for students with OCD, the potential for its existence raises serious concerns.
In sum, many of the previously cited problems related to educational practices for classifying and
labeling students with mental health disorders, in general, have been associated with students who have
OCD. These include the stigma linked to attaching the ED label to students with OCD, difficulties
aligning a school-based category with a clinical diagnosis, and concerns regarding the provision of
services to students with OCD. An understanding of current practices in identifying and classifying
students with obsessive-compulsive disorder is essential to inform and advance best practices for serving
these students. School psychologists, who play a crucial role in the identification and classification
process, represent a key source of information regarding these practices. Yet none of the research that
has appeared in the school psychology literature on childhood OCD since 1991 (Adams et al., 1994;
Clarizio, 1991; McGough, Speier, & Cantwell, 1993; Sabuncuoglu & Berkem, 2006) has examined
school psychologists’ experiences and perceptions related to school-based practices for identifying and
categorizing students with a clinical diagnosis of OCD.
The purpose of the present investigation was to conduct descriptive research with school psychologists in the state of Illinois first to determine how commonly childhood OCD is seen in educational
settings and current patterns related to serving these students in school (e.g., 504, IDEA). Second, this
study explored how students with a primary diagnosis of OCD served under IDEA are identified and
categorized as well as school psychologists’ opinions of these practices. Finally, the types of educational placements in which these students receive instruction were examined. To that end, the following
research questions were posed:
1.What percentage of students evaluated or served by school psychologists have a primary diagnosis of OCD?
Classifying And Serving Students With OCD
97
2.Among students with a primary diagnosis of OCD, how many are served via 504 plans, and how
many are served under IDEA?
3.Among students with a primary diagnosis of OCD who are served under IDEA, what is the
disability category (e.g., E/BD, OHI) under which they are most frequently classified, and under
which disability category do school psychologists believe these students should be classified?
4.Among students with a primary diagnosis of OCD who are served under IDEA, in which educational settings are they placed (e.g., general education class, resource room)?
Method
Participants
To facilitate the selection of a suitable sample, a listing of the names and addresses of all school
psychologists in the state of Illinois was acquired from the Illinois State Board of Education. From this
list, a random sample of 400 names was chosen, using a computer-generated list of random numbers.
Of the 400 individuals randomly selected, 123 school psychologists responded, for a response rate of
31%. Participants were recruited in accordance with the Institutional Review Board (IRB) from the first
two authors’ institution, which approved the study as exempt from the Code of Federal Regulations for
the protection of human subjects. All respondents were entered into a drawing for three U.S. Savings
Bonds.
Instrumentation
To gather data relevant to the stated research questions, a survey was constructed that asked respondents to provide (1) numerical responses, e.g., “How many students have you evaluated and/or provided
services for (including such services as consulting) this school year?”; (2) categorical responses, e.g.,
“In your opinion, students with a primary diagnosis of OCD should be served under…” (respondents
marked one of several categories provided); and (3) short verbal responses, e.g., “If you marked ‘Strongly
Disagree’ or ‘Disagree’ above, please explain.” For several items, respondents were asked to indicate the
number of students with OCD who fit into each of three different severity categories. The severity levels
were defined as follows:
Mild OCD: Slight interference with performance, but overall performance is not impaired
Moderate OCD: Definite interference with performance, but still manageable
Severe OCD: Substantial impairment in performance
These definitions were adapted from the Yale-Brown Obsessive-Compulsive Scale (Goodman,
Price, Rasmussen, Mazure, Fleischman, Hill, et al., 1989), a scale that measures the severity of classic
OCD symptoms.
Procedures
In April, 2003, a copy of the survey, two copies of the cover letter, and a self-addressed, stamped
envelope were sent to the 400 randomly chosen school psychologists. Participants were asked to sign
and return one copy of the cover letter along with the completed survey to indicate consent to participate
in the study. Follow-up surveys were sent to those psychologists who did not respond after the initial
mailing, and follow-up calls were made to individuals who did not respond to either mailing. Data were
compiled and then analyzed using descriptive statistics.
98
The California School Psychologist, 2007, Vol. 12
Results and Discussion
Research Question 1
The first research question addressed the percentage of students evaluated or served by school
psychologists who had a primary diagnosis of OCD. Of the 12,685 students reported as being evaluated
or served, 94 (0.7%) had a primary diagnosis of OCD. While this figure does not necessarily represent
the incidence of primary OCD in the population of students at large, it is interesting to note that this
statistic is identical to the one-year incidence rate suggested by the APA (0.7%) and relatively close to
the 1%, one-year incidence rate established by Flament, Whitaker, Rapoport, Davies, Berg, Kalikow, et
al. (1988). The results appear to confirm the assertion by Riddle (1998) that OCD is not a rare disorder
among young people.
Research Question 2
Research question two addressed the percentage of students with a primary diagnosis of OCD
served via 504 plans and the percentage served under IDEA. School psychologists reported that of the 94
students with a primary diagnosis of OCD, 12 (12.8%) received services under Section 504, 70 (74.5%)
were served under IDEA, and the remaining 12 students (12.8%) were receiving no services. Acknowledging the role of OCD severity level, some respondents reported that when OCD symptoms were mild
or well controlled by medication, no services or modifications were considered necessary.
As a point of interest, the relationship between OCD severity level and the type of services received
(i.e., 504 or IDEA) also was examined using a chi-square test of independence. No significant relation2
ship between severity level and the type of service received was found (x (2, N = 82) = 4.38, p = .11),
and the effect size for this relationship was small (Cramer’s V = .23, with a small observed tendency
for children with more severe OCD to be served under IDEA rather than 504). It is important to note,
however, that because the large majority of students with a primary diagnosis of OCD in the current
study were reported as being served under IDEA, it appears that the typical impact of OCD on educational performance was deemed serious enough to warrant special education and related services.
Research Question 3
The first part of research question three addressed the disability category under which students with
a primary diagnosis of OCD served under IDEA were classified. The category school psychologists most
frequently reported was “E/BD” (51.4%), followed by “Other Health Impaired” (31.8%). The combination category “E/BD or OHI” was less likely to be reported (10.3%), with “Other” and various combination categories comprising only 6.5% of the responses. Thus, E/BD was the IDEA category under which
the majority of students with a primary diagnosis of OCD were reported as being classified in the present
study (see Table 1).
The second part of research question three was addressed by the survey item, “In your opinion,
students with a primary diagnosis of OCD should be identified under the following IDEA category for the
OCD…” (five categories were provided: Emotional/Behavioral Disorders, Learning Disabilities, Other
Health Impaired, Other, or Need to create a new category under IDEA for these students). The obtained
frequencies indicate that there was strong consistency between the category under which students with
OCD were reported to be classified (Table 1) and school psychologists’ opinions of the most appropriate
IDEA category for these students (Table 2). A small percentage of psychologists believed that fewer
students should be classified under E/BD than actually were. Nonetheless, 47.7% of school psycholo-
Classifying And Serving Students With OCD
99
TABLE 1. Frequency Distribution of Responses: Reported IDEA Disability Categories for Students
with a Primary Diagnosis of OCD
Category
Frequency
Percent
E/BD
LD
OHI
Other
E/BD or OHI
LD and OHI
E/BD and LD
E/BD, OHI, and Other
55
0
34
2
11
1
3
1
51.4%
0.0
31.8
1.9
10.3
0.9
2.8
0.9
Total
107
100.0
Note. E/BD=Emotional/Behavioral Disorder, LD = Learning Disability, OHI = Other Health Impaired
TABLE 2. Frequency Distribution of Responses: Psychologists’ Opinions of Appropriate IDEA Category for Students with a Primary Diagnosis of OCD
Category
Frequency
Percent
E/BD
LD
OHI
Other
New category needed
E/BD or OHI
E/BD, OHI, and Other
Other/new category needed
52
0
35
2
6
12
1
1
47.7%
0.0
32.1
1.8
5.5
11.0
0.9
0.9
Total
109
100.0
Note. E/BD=Emotional/Behavioral Disorder, LD = Learning Disability, OHI = Other Health Impaired
gists selected E/BD as the most appropriate category for students with a primary diagnosis of OCD.
Approximately 5.5% of the respondents reported that a new category is needed to classify students with
OCD.
As a follow up to this question, respondents were asked to explain why they chose a particular IDEA
category. Overall, two patterns emerged from an analysis of the school psychologists’ comments. First,
psychologists expressed considerable disagreement as to which IDEA category was the most appropriate
for these students. Those who viewed OCD as a social/emotional or behavior disorder chose the E/BD
category for OCD. Others, who perceived OCD as medical/clinical in nature, believed OHI was the best
fit. In certain cases, psychologists referred to the IDEA category definition to classify OCD. For example,
some chose OHI on the basis of the “limited attention” component of the definition, while others chose
E/BD because of the “inappropriate types of feelings or behavior under normal circumstances” clause of
the ED definition. Still others referred to the relationship between OCD and the DSM-IV-TR. Ambiguity
100
The California School Psychologist, 2007, Vol. 12
and uncertainty surrounding the appropriate IDEA classification for OCD also appeared in the remarks
of several respondents who chose “E/BD or OHI.” One of these psychologists stated, “I do not define
it as an emotionally based disorder, and although it may be physiologically based, I have a hard time
saying it is a health impairment...”
Overall, the results of the current study indicate that there is a clear lack of agreement among
school psychologists with regard to the most appropriate school-based disability category for OCD. It
may be that the students with a primary diagnosis of OCD in the present study represented a wide range
of difficulties and comorbid disorders. Nonetheless, the psychologists’ varied responses to questions
concerning the categorization of students with OCD seem to suggest that current training and guidance
for evaluating and classifying OCD are inadequate.
A second pattern inherent in psychologists’ remarks regarding the most suitable educational category for OCD was concern regarding the stigma that may be associated with the ED label. A number of
psychologists specifically alluded to the potentially negative impact of the ED label and parent preference for the OHI label. These results thus suggest that many of the problems that have been cited in the
literature regarding educational practices for classifying and labeling students with mental health disorders also apply to students with the specific diagnosis of OCD.
Research Question 4
The final research question addressed the types of educational settings in which students with a
primary diagnosis of OCD served under IDEA are placed. In one of the survey items, respondents were
presented with a variety of educational placements and asked to indicate the number of students served
within each of those settings. As is evident in Table 3, placements for students with OCD represent
almost the full continuum of IDEA services. Of the 70 students served under IDEA, 47 (67.1%) were
reported as receiving their instruction in less restrictive environments. More specifically, 10.0% were
placed in full-time regular classrooms only, 11.4% received instruction in the regular classroom and
part-time special classes, and a sizeable percentage (45.7%) received instruction in the regular classroom
with part-time resource assistance.
Approximately 29% of the students with a primary diagnosis of OCD served under IDEA were
educated in more restrictive settings. Of these students, 4.3%, 7.2%, and 5.7% were placed in LD, ED,
and E/BD self-contained classrooms, respectively. An additional 11.4% received instruction in special
day schools, and no students were placed in residential settings. Because there was only one residential
school in the state of Illinois known to treat students with OCD at the time the survey was administered,
this finding was not unexpected. As indicated in Table 3, 4.3% of students were reported as being served
in placements designated as “Other.”
The results of this study indicate that the large majority of students with a primary diagnosis of OCD
served under IDEA received instruction in regular classrooms, alone or in combination with resource
services or part-time special classes. Hence, it appears that the IDEA provision of the least restrictive
environment was a guiding principle in decisions related to educational placements for students with
OCD.
Study Limitations and Future Directions
A limitation of this investigation was that the results were based upon school psychologists’ reports
of students on their caseloads who had a primary diagnosis of OCD. Formal clinical evaluations of the
presence or absence of primary OCD among students using such standardized instruments as diagnostic
Classifying And Serving Students With OCD
101
TABLE 3. Frequency Distribution of Responses: Reported Educational Placements for Students with
a Primary Diagnosis of OCD Served under IDEA
Placement
Frequency
Percent
7
32
8
10.0%
45.7
11.4
(47)
(67.1)
3
5
0
4
0
8
0
4.3
7.2
0.0
5.7
0.0
11.4
0.0
(Total)
(20)
(28.6)
Other
3
4.3
Grand Total
70
100.0
FT regular classroom
Reg. class & PT resource rm
Reg. class & PT spec.class
(Total)
FT self-contained LD
FT self-contained ED
FT self-contained BD
FT self-contained E/BD
FT self-contained OHI Special day school
Residential setting
Note. FT = Full time, PT = Part time, E/BD=Emotional/Behavioral Disorder, LD = Learning Disability, OHI = Other
Health Impaired
interviews may have yielded different results. Also, due to the self-selection nature of the sample, it is
possible that school psychologists with a greater number of students with OCD on their caseloads were
more likely to respond (although the majority of respondents reported having no students with OCD on
their current caseloads). Future research could expand upon the results of the present study by obtaining
data from a national sample of school psychologists. Future studies also might collect data on students
who have OCD as either a primary or secondary diagnosis (only students with a primary diagnosis of
OCD were included in the present investigation). It is important to note that, as the first of its kind to be
conducted, this study is exploratory in nature. Additional research is necessary not only to confirm or
refute the findings of this investigation, but also to gather information in such areas as actual services
provided to students with OCD.
Conclusions/Implications for School Psychologists
The results of this study strongly suggest that the difficulties encountered in current educational practices for classifying and serving students with OCD mirror those associated with classifying and serving
students with mental disorders, in general. The traditional educational evaluation process requires the
identification of a particular categorical diagnosis for students with OCD – a process that is characterized
by ambiguity and lack of agreement. Additionally, concerns exist regarding the stigma of the ED label
for children and adolescents with OCD. Furthermore, because comorbidity among students with OCD is
very common, these students require many different levels of support.
Although evidence from the present study as well as the existing OCD literature provide support
for identifying students with OCD under the IDEA category of OHI (e.g., Adams, 2004), an alternative
approach might better ensure that these students receive appropriate services. More specifically, the time
102
The California School Psychologist, 2007, Vol. 12
currently spent evaluating students with OCD to meet traditional legal requirements for proper educational identification and categorization might be used more effectively to implement evaluation procedures to identify successful services and interventions for these students. Once such interventions are
implemented, a heavy emphasis should be placed upon careful monitoring of service effectiveness over
time to promote the most positive outcomes possible. Such procedures are beginning to be put into practice for students with learning disabilities as a result of the Response-to-Intervention (RtI) focus within
IDEA 2004. A related non-categorical model for service delivery has been developed and implemented
in Iowa over the past decade, in which educational services for students with social-emotional problems
are provided through a Response-to-Intervention framework within the school setting (see Grimes &
Kurns, 2003). Similarly, Gresham (2005) supports RtI as an alternative method for identifying students
as having an emotional disturbance, stating that the RtI method is in direct contrast to current practice,
which is based on a “refer-test-place model in which students are not exposed to systematic, evidencebased interventions to ameliorate behavior problems” (Gresham, p. 341).
Application of a three-tier RtI model might encourage school systems to conduct universal screening
for emotional and behavioral difficulties and implement intervention strategies that show promise for
alleviating learning and behavior problems students with OCD frequently experience. Screening tools for
detecting emotional and behavioral disorders are available but very infrequently used in school settings
(Forness, Serna, Nielsen, Lambros, Hale, & Kavale, 2000). Prevention techniques such as teaching all
students strategies for coping with anxiety (Merrell, 2001) and creating a positive and calm classroom
environment (Lehr & Christiansen, 2002) might be incorporated as part of a first tier level of service
provided to all students.
At the second tier, students with OCD whose symptoms interfere with learning and/or behavior could
be brought to the attention of a school psychologist. The psychologist might conduct further assessment
and collaborate with families and professionals outside the school to implement such empirically based
interventions as teaching more advanced coping strategies, cognitive-behavioral therapy (Freeman et
al., 2007), and/or pharmacotherapy treatment (Franklin, March, & Garcia, 2007). A psychologist in a
clinical rather than educational setting may play the major role in implementing certain interventions
for OCD (e.g., pharmacotherapy). Educational professionals, however, are instrumental in supporting
and monitoring the effects of clinical interventions. If these strategies prove ineffective or are highly
resource intensive (e.g., additional funding is needed to ensure that the student receives ongoing services
to facilitate appropriate progress), an evaluation might be conducted to determine whether the student
qualifies for special education services. The evaluation process might be part of a third tier of support
provided to students with substantial needs. This evaluation may involve the implementation of intervention strategies that are very resource intensive (e.g., frequent therapy sessions, comprehensive behavior
intervention plan) to identify the conditions under which the student is successful. If it is determined that
the student requires such intervention strategies in order to make sufficient progress, he or she eventually
may be provided those strategies on an ongoing basis using funding available through special education.
Monitoring of progress over time would be essential to verify the efficacy of these interventions.
Within the three-tiered RtI model described above, interventions that are effective for a given individual are identified via the evaluation process, placing the focus on instructional needs rather than
disability categorization (Grimes and Kurns, 2003). Instead of having to fit a particular disability category, a student may receive special education services based on the conditions under which he or she
was found to be successful.
Although the RtI school psychology service delivery model for students with or at-risk for OCD
Classifying And Serving Students With OCD
103
may be ideal for some school systems, others may not be capable of facilitating such change in the near
future. Therefore, traditional categorical service delivery models may continue in many locations. The
clear lack of agreement among school psychologists regarding an appropriate category for students with
OCD observed in the present study suggests that there is a need for training and guidance with regard
to evaluating and classifying OCD. Such training would help ensure that services provided through
categorical service delivery models are tailored to meet individual student needs. By acquiring a basic
understanding of conditions that may affect the learning and behavior of students with OCD, school
psychologists would be in a position to inform the IEP team about services and settings that would lead
to student success.
To ensure that students with OCD receive appropriate services, the potential consequences of categorical diagnoses must be considered carefully. For example, a student with OCD may need substantial
structure to avoid being overwhelmed by his or her thoughts and corresponding rituals. Such structure
initially might be considered as being most easily facilitated in a self-contained setting. However, other
students in that type of setting may engage in behaviors (e.g., teasing, joking) that have the potential
to exacerbate the student’s obsessions and compulsions. Additionally, care should be taken to place
students with OCD in a setting that optimizes access to services that address their needs. Within the
evaluation process, consideration should be given to the students’ specific educational needs and how
those can best be met within the existing categorical service delivery system.
In conclusion, the best treatment options and services for students with OCD might result when
school psychologists, clinical psychologists, teachers, parents, and the students collaborate in the
development of intervention and monitoring plans that can be carried out in clinical, home, and school
settings. Additionally, these plans should be informed by research on effective services for students with
OCD. To that end, the following recommendations for best practice with regard to OCD interventions
are proposed:
(1) Focus on implementing preventative interventions prior to considering 504 or IDEA eligibility, which should include a heavy emphasis on educating teachers, psychologists, and other
school personnel about OCD. Such strategies as teaching all students techniques for coping
with anxiety, holding clear classroom structure and expectations, and creating a generally calm
classroom climate may prevent OCD from inhibiting a student’s learning in school (Paige,
2004);
(2) Collaborate with clinical psychologists, parents, teachers and students to design and implement
interventions that are aligned across the settings in which a child is engaged;
(3) Collaborate with clinical psychologists to find and implement interventions with empirical
support such as cognitive behavior therapy (Freeman et al., 2007) and pharmacotherapy,
including selective serotonin reuptake inhibitors (Franklin et al., 2007), to address behaviors
that interfere with academic achievement and/or social functioning;
(4) Emphasize program content and relevant goals, i.e., focus on identifying measurable, relevant
goals; and
(5) Emphasize progress monitoring and evaluation of student outcomes, which may include assistance with monitoring a student’s response to medication (Volpe, Heick, & Guerasko-Moore,
2005).
School psychologists can play a vital role in creating, implementing, and monitoring school-based
interventions for children and adolescents with OCD that ultimately will promote mental health and
learning outcomes for these students.
104
The California School Psychologist, 2007, Vol. 12
References
Adams, G. B. (2004). Identifying, assessing, and treating obsessive-compulsive disorder in school-aged children:
The role of school personnel. Teaching Exceptional Children, 37(2), 46-53.
Adams, G. B., Waas, G. A., March, J. S., & Smith, M C. (1994). Obsessive-compulsive disorder in children and
adolescents: The role of the school psychologist in identification, assessment, and treatment. School Psychology
Quarterly, 9(4), 274-294.
American Psychiatric Association (2000). Diagnostic and statistical manual of mental disorders (4th ed., Text Revision). Washington, DC: Author.
Becker, B., & Luthar, S. (2002). Social-emotional factors affecting achievement outcomes among disadvantaged
students: Closing the achievement gap. Educational Psychologist, 37, 197-214.
Bianco, M. (2005). The effects of disability label and special education and general education teachers’ referrals for
gifted programs. Learning Disability Quarterly, 28, 285-293.
Blier, P., Habib, R., & Flament, M. F. (2006). Pharmacotherapies in the management of obsessive-compulsive
disorder. Canadian Journal of Psychiatry, 51(7), 417-430.
Carlburg, C., & Kavale, K. (1980). The efficacy of special versus regular class placement for exceptional children:
A meta-analysis. Journal of Special Education, 14, 295-309.
CHADD of Alachua County (n.d.). Welcome to CHADD—Information. Retrieved March 22, 2007, from http://
ourworld.cs.com/alachuachadd/
Chansky, T. E. (2000). Freeing your child from Obsessive-Compulsive Disorder. New York: Crown Publishers
Crown Publishers.
Clarizio, H. F. (1991). Obsessive-compulsive disorder: The secretive syndrome. Psychology in the Schools, 28(2),
106-115.
Dornbush, M. P., & Pruitt, S. K. (1996). Teaching the tiger. Duarte, CA: Hope Press.
Flament, M. F., Whitaker, A., Rapoport, J. L., Davies, M., Berg, C. Z., Kalikow, K. et al. (1988). Obsessive compulsive disorder in adolescence: An epidemiological study. Journal of the American Academy of Child and Adolescent Psychiatry, 27(6), 764-771.
Fontenelle, L. F., Mendlowicz, M. V., & Versiani, M. (2006). The descriptive epidemiology of obsessive-compulsive
disorder. Progress in Neuro-Psychopharmacology & Biological Psychiatry 30, 327-337.
Forness, S., Serna, L., Nielsen, E., Lambros, K., Hale, M., & Kavale, K. (2000). A model for early detection and
primary prevention of emotional and behavioral disorders. Education and Treatment of Children, 23, 325-345.
Foster, G., & Ysseldyke, J. (1976). Expectancy and halo effects as a result of artificially induced teacher bias.
Contemporary Educational Psychology, 1, 37-45.
Franklin, M. E., March, J. S., & Garcia, A. (2007). Treating obsessive-compulsive disorder in children and adolescents. In M. M. Antony, C. Purdon, & L. Summerfeldt (Eds.) Psychological treatment of obsessive-compulsive
disorders: Fundamentals and beyond. (pp. 253-266). Washington, DC: American Psychological Association.
Freeman, J. B., Choate-Summer, M. L., Moore, P. S., Garcia, A. M., Sapyta, J. J., Leonard, H.L., et al. (2007).
Cognitive-behavioral treatment for young children with obsessive compulsive disorder. Biological Psychiatry,
61, 337-343.
Gallagher, J. J. (1976). The sacred and profane uses of labeling. Mental Retardation, 14, 3-7.
Geller, D. A. (2006). Obsessive-compulsive and spectrum disorders in children and adolescents. Psychiatric Clinics
of North America, 29, 353-370
Goodman, W. K., Price, L. H., Rasmussen, S. A., Mazure, C., Fleischman, R. L., Hill, C. L., et al. (1989). The YaleBrown obsessive-compulsive scale: I. Development, use and reliability. Archives of General Psychiatry, 46(11),
1006-1011.
Gresham, F. M. (2005). Response to intervention (RTI): An alternative means of identifying students as emotionally
disturbed. Education and Treatment of Children, 28, 328-344.
Grimes J., & Kurns, S. (2003, December). An intervention-based system for addressing NCLB and IDEA expectations: A multiple tiered model to ensure every child learns. Paper presented at the National Research Center on
Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.
Hoagwood, K., & Johnson, J. (2003). School psychology: A public health framework: I. From evidence-based policies. Journal of School Psychology, 41, 3-21.
Hosp, J., & Reschly, D. (2002). Regional differences in school psychology practice. School Psychology Review, 31,
11-29.
Lehr, C., & Christensen, S. (2002). Best practices in promoting a positive school climate. In A. Thomas & J. Grimes
(Eds.) Best practices in school psychology IV (pp. 929-948). Bethesda, MD: National Association of School
Psychologists.
Classifying And Serving Students With OCD
105
McGough, J. J., Speier, P. L., & Cantwell, D. P. (1993). Obsessive-compulsive disorder in children and adolescents.
School Psychology Review, 22, 243-251.
Merrell, K. (2001). Helping students overcome depression and anxiety: A practical guide. New York: Guilford
Publications.
Muller, E., & Markowitz, J. (2004). Disability categories: State terminology, definitions, and eligibility criteria.
Alexandria, VA: National Association of State Directors of Special Education. Retrieved on March 17, 2007
from http://www.ecs.org/html/offsite.asp?document=http%3A%2F%2Fwww%2Enasdse%2Eorg%2Fpublicatio
ns%2Fdisability%5Fcategories%2Epdf +.
Paige, L. Z. (2004). Obsessive-compulsive disorder: Information for parents and educators. In A. Canter, L. Paige,
M. Roth, I. Romero, and S. Carroll (Eds.), Helping children at home and school II: Handouts for families and
educators. Bethesda, MD: National Association of School Psychologists.
Piacentini, J., Bergman, R. L., Keller, M., & McCracken, J. (2003). Functional impairment in children and adolescents with obsessive-compulsive disorder. Journal of Child and Adolescent Psychopharmacology, 13(Suppl. 1),
S61-S69.
Rapoport, J. L., Weissman, M. M., Narrow, W. E., Jensen, P. S., Lahey, B. B., & Canino, G. (2000). Childhood
obsessive-compulsive disorder in the NIMH MECA study: Parent versus child identification of cases. Journal
of Anxiety Disorders, 14(6), 535-548.
Riddle, M. (1998). Obsessive-compulsive disorder in children and adolescents. British Journal of Psychiatry,
173(Suppl. 35), 91-96.
Sabuncuoglu, O., & Berkem, M. (2006). The presentation of childhood obsessive-compulsive disorder across home
and school settings: A preliminary report. School Psychology International, 27, 248-256.
Smith, V. (2003). You have to learn who comes with the disability: Students’ reflections on service learning experiences with peers labeled with disabilities. Research and Practice for Persons with Severe Disabilities, 28(2),
79-90.
Sukhodolsky, D. G., do Rosario-Campus, M. C., Scahill, L., Katsovich, L., Pauls, D. L., Peterson, B. S., et al. (2005).
Adaptive, emotional, and family functioning of children with obsessive-compulsive disorder and comorbid
attention deficit hyperactivity disorder. American Journal of Psychiatry, 162(6), 1125-1132.
Susman, J. (1994). Disability, stigma and deviance. Social Science and Medicine, 38, 15-22.
Thurlow, M. L., & Ysseldyke, J. E. (1982). Instructional planning: Information collected by school psychologists vs.
information considered useful by teachers. Journal of School Psychology, 20, 3-10.
Tourette Syndrome Association (2006a, August 3). Major victory for children with Tourette Syndrome: Individuals
with Disabilities Education Act to classify Tourette Syndrome as Other Health Impaired. Retrieved August 31,
2006, from http://www.tsa-usa.org/news/TOURETTEOHI0806.htm.
Tourette Syndrome Association (2006b, Fall). What’s in a name? A lot if the name is a label used by a school.
Tourette Syndrome Association, Inc. Newsletter, 34(3), Insert, p. 2.
Volpe, R. J., Heick, P. F., & Guerasko-Moore, D. (2005). An agile behavioral model for monitoring the effects of
stimulant medication in school settings. Psychology in the Schools, 42, 509-523.
Zohar, A. H. (1999). The epidemiology of obsessive-compulsive disorder in children and adolescents. Child and
Adolescent Psychiatric Clinics of North America, 8(3), 445-460.
The California School Psychologist, Vol. 12, pp. 107 – 120, 2007
Copyright 2007 California Association of School Psychologists
107
Developing Emotional Competence in Preschoolers:
A Review of Regulation Research
and Recommendations for Practice
Renée M. Tobin
Illinois State University
Frank J. Sansosti
Kent State University
Laura Lee McIntyre
Syracuse University
Regulation has been implicated in the development of emotional and behavioral disorders in childhood. Indeed, emotion dysregulation is one of the most common reasons families seek psychological
services and behavioral supports. Interventions to support children with regulatory difficulties may
be enhanced if they are informed by basic psychological research on the topic. This paper includes
a review of basic regulation research conducted over the last 20 years. This research base about the
positive development of regulatory skills is then related to the treatment of emotion regulation deficits,
emphasizing the role that school psychologists and school-based interventions may play in supporting
appropriate regulatory strategies for young children.
Keywords: emotion regulation, early childhood development, emotional competence, emotional
development, emotional control, emotion
One of the most common reasons for seeking childhood psychological services is difficulty with
emotion regulation (Linscott & DiGiuseppe, 1994). Katz and Gottman (1995) define emotion regulation
as “children’s ability to deal with having a strong negative or positive emotion and organize themselves
in the service of an externally imposed goal” (p. 84). That is, the practice of recognizing emotions and
developing the skills needed to manage reactions to them. Although emotion regulation deficits are not
considered a disorder in their own right, they often are associated with a host of disorders across early
ages. When children receive treatment for emotion regulation deficits, it typically is provided within the
context of treatment for behavioral disorders (e.g., Oppositional Defiant Disorder, Conduct Disorder),
mood disorders (e.g., Major Depressive Disorder), anxiety disorders (e.g., Separation Anxiety Disorder,
Generalized Anxiety Disorder), or other disorders of childhood.
Over the last 20 years, basic research on the topic of emotion regulation has increased significantly
(Eisenberg, 2002). Despite an overall increase in the amount of research available from various fields
of psychology (e.g., clinical, social), there currently is a paucity of applied literature that draws on this
basic research base to inform treatment and intervention development for children with emotion regulation issues. Thus, a review of basic research may help provide useful information for school psychologists responsible for developing appropriate interventions to support children with disorders related to
emotion dysregulation (Eisenberg, Fabes, Guthrie et al., 1996).
According to Thompson (1994), “Emotion regulation consists of the extrinsic and intrinsic processes
Please address all correspondence to Renée M. Tobin, Ph.D., Campus Box 4620, Psychology Department, Illinois State
University, Normal, IL 61790-4620.
108
The California School Psychologist, 2007, Vol. 12
responsible for monitoring, evaluating, and modifying emotional reactions, especially their intensive and
temporal features, to accomplish their goals” (pp. 27-28). A host of psychological systems are thought to
be regulated, and a subset of these are assumed to become self-regulating as part of normal development.
These systems include control of emotions, reactions to failure and disappointment, and most forms
of moral and achievement activities. Indeed, when certain systems fail to demonstrate self-regulation,
psychologists often assume that normal development has been disrupted. As such, studying function as
well as dysfunction in emotion regulation informs the clinical practice of school psychology. That is,
interventionists may benefit from a better understanding of how systems work when they are functioning
well, by examining the basic research literature, and when they are disrupted, by examining the applied
literature. Here, the developmental literature for both regulation and dysregulation of emotion is reviewed
and the implications for both assessment and intervention with preschool children are discussed.
The Development of Emotion Regulation
A review of the empirical studies of regulation from infancy through preschool produces considerable material on which to base assessment and intervention practices for preschool children. Basic
research on regulation has identified a number of factors that influence the development of these
processes at different ages, particularly biobehavioral processes, individual differences in temperament,
and interpersonal processes (i.e., modulation of emotion in one person through the activity of another;
see Tobin & Graziano, 2006, for a detailed review). Each of these areas are reviewed briefly to provide
school psychologists with foundational knowledge related to the development of emotion regulation in
children, as well as to provide opportunities to address problems in emotion regulation as it relates to
preschool education.
During infancy, regulation focuses on biobehavioral processes such as biological rhythmicity, security, and synchronicity with a caregiver. In particular, vagal tone, a physiological measure of the inhibitory influence of the parasympathetic branch of the autonomic nervous system on the heart (responsible
for slowing the heart rate and decreasing blood pressure), has been consistently linked to regulatory
processes through its role in soothing and restoring calmness in the body (Katz & Gottman, 1995). For
example, Calkins, Smith, Gill, and Johnson (1998) reported that negative control behavior by mothers
(e.g., negative control: scolding, anger expressions, derogatory remarks, threats, no’s; physical control:
restricting child’s movement, pulling, pushing, picking child up, hand slapping; and verbal control:
directing the child’s activity, telling the child what to do) was related to poor child physiological regulation (as measured by vagal tone), less adaptive emotional regulation, and noncompliant behavior. It
appears that these early infant-mother dyad influences not only shape children’s developing characteristic physiological response style (Bornstein & Suess, 2000), but also may result in the child’s difficulty to respond appropriately to a variety of emotion-eliciting stimuli. Lower baseline vagal tone in
infants and toddlers is predictive of later difficulties in social interactions that require reciprocal engagement (Porges, Doussard-Roosevelt, Portales, & Greenspan, 1996) and the development of anxious and
depressed symptoms in preschool and early elementary years (Cole, Zahn-Waxler, Fox, Usher, & Welsh,
1996). Consistent with these findings, Calkins (1997) found that toddlers who had ongoing suppressions
in vagal tone during emotionally evocative situations (i.e., the ability to regulate physiological function
to meet external demands, referred to as the vagal brake) engaged in more behavioral regulation during
emotion-eliciting tasks.
Regulatory differences can also be measured at the hormonal level. The stress response typically
involves a cascade of hormones produced by the adrenal cortex. The presence of elevated levels of these
Emotional Competence in Preschoolers
109
hormones in the blood (including and especially, cortisol) is usually interpreted as the result of stress.
In a study linking basic biological processes to emotion regulation, Gunnar, Mangelsdorf, Larson, and
Hertsgaard (1990) found that infants who displayed more distress than their peers during laboratory tests
showed greater adrenocortical activity. Similarly, Spangler, Schieche, Ilg, Maier, and Ackermann (1994)
found that elevated blood cortisol was more frequently observed among children of highly insensitive
mothers, suggesting a role for maternal behavior in infant behavioral regulation. Evidence from human
and animal studies clearly shows that in stressful situations, the presence of familiar others, especially if
there is close physical contact, reduces the levels of stress hormones in infants.
Related to basic biological factors, the development of self-regulatory processes is largely influenced by individual differences in temperament. Stifter and Braungart (1995) suggested a connection to
both individual differences in temperament (self-soothing during decreasing negative arousal) and interaction with caregivers (communicative behavior during increasing distress). Additionally, Goldsmith,
Buss, and Lemery (1997) found that toddler and preschool displays of positive emotion share significant
environmental influence, whereas emotion regulation was related to both environmental and genetic
influence. Blackford and Walden (1998) found that temperament is more closely related to regulation
and responsiveness to differences in communication from a parent. “Effortful control,” the ability to
ignore a dominant response in favor of a subdominant response (Rothbart & Bates, 1998), has also been
linked to young children’s emotion. Researchers found that effortful control in preschoolers is related to
regulation of negative affect (Kochanska & Knaack, 2003; Kochanska, Murray, & Harlan, 2000), as well
as positive affect (Kieras, Tobin, Graziano, & Rothbart, 2005).
The importance of the relationship with a caregiver in developing emotion regulation skills is also
evident as early as infancy. Walden and Baxter (1989) found that older infants demonstrated regulation
through social referencing of a parent. Yet another study found that securely attached infants demonstrated greater parent-oriented regulation relative to infants who were less securely attached to one
or both parents (Diener, Mengelsdorf, McHale, & Frosch, 2002). Based on a sample of 223 children,
Vondra, Shaw, Swearingen, Cohen, and Owens (2001) concluded that attachment classification, though
not always stable between infancy and preschool, provides the most valuable information about child
functioning.
The influence of both temperament and relationship with caregivers continues to be important as
predictors of functioning throughout early childhood. Feldman, Greenbaum, Yirmiya, and Mayes (1996)
found that patterns of synchrony between mothers and infants predicted verbal and general IQ at age two.
In addition, research demonstrated that preschoolers high in the temperamental dimension of effortful
control were unlikely to experience high levels of negative emotional arousal in response to peer interactions (Fabes et al., 1999). Moreover, when interactions were of elevated intensity, highly regulated
preschoolers were likely to show socially competent responses. Rubin, Coplan, Fox, and Calkins (1995)
reported similar links between temperament and peer relations.
Recent research suggests that biobehavioral processes associated with cardiovascular function may
relate to basic regulatory processes and contribute to the development of subsequent regulatory processes
in other domains. It is unclear whether poor vagal suppression is simply a marker of regulatory dysfunction or dysfunctions in the vagal response system cause emotional dysregulation. Basic biological results
also suggest that parent-child relationships play an important role in the development of young children’s
physiological systems and may provide an additional target of assessment and intervention modalities.
Adding more complexity, the presence of strong biological precursors of regulation does not exclude
social and interpersonal influences. Even in infancy, individual differences in temperament and interper-
110
The California School Psychologist, 2007, Vol. 12
sonal relations, especially with caregivers, can influence developing regulatory systems. Thus, it is vital
that school psychologists are aware of the bidirectional effects among interpersonal processes, temperament, and basic biological functioning within a developing system. As an interventionist, it is helpful to
highlight the important role that adult-child relationships play in the healthy development of biological
and temperamental systems.
Taken together, these results suggest that regulation is not a single process or set of processes.
Further, basic researchers provide knowledge to inform clinical practice that is within the bounds of
current treatment methods. That is, school psychologists have the assessment tools and interventions
available to address both within-child and interpersonal processes of emotion regulation. These findings
suggest a need for greater communication between the scientific community and practitioners so that the
specific findings of basic research may be used to full advantage in treatment.
Measuring Regulation
Basic psychological research has provided multiple methods of measuring regulation in preschool
children. These methods vary considerably in their feasibility for practicing school psychologists. For
example, physiological measures are frequently used as indicators of regulation in early childhood;
however, most school psychologists do not have the time, financial resources, or training to assess children’s vagal tone, blood chemistry, or electroencephalography (EEG) activity. Fortunately, regulation
researchers have developed several other methods to assess regulation processes in early childhood. Of
course, no single assessment measure or method is all-encompassing, and it is generally better practice to
orient assessment to referral concerns, but these methods are likely to provide valuable information for
practicing school psychologists. Table 1 provides examples of regulation measures for preschool aged
children. Although this list is not comprehensive, it provides several research-based options for use by
school psychologists.
One common assessment method that is used frequently by school psychologists is rating scales
completed by knowledgeable adult informants such as parents and teachers. When determining which
ratings scales to use, however, most school psychologists tend to rely heavily on established clinical
measures such as the Achenbach System of Empirically Based Assessment (ASEBA; Achenbach &
Rescorla, 2004) and the Behavior Assessment System for Children (BASC-2; Reynolds & Kamphaus,
2004) without much consideration of the valuable measures available from basic developmental research.
Several such measures exist to assess regulation within a preschool population. For example, Rothbart
and colleagues (Putnam & Rothbart, 2007; Rothbart, Ahadi, Hershey, & Fisher, 2001) have generated
several temperamental measures of regulation in preschoolers including an Effortful Control factor on
the Children’s Behavior Questionnaire. Their extensive research has generated instruments that parents
use to rate a child’s behavior over the last 6 months using 7-point Likert-type scales.
Building on Rothbart’s work, Eisenberg and Fabes (1995) adapted one of the Rothbart measures
to study social competence, regulation, and emotionality. These researchers also developed vignettes
to assess children’s emotion regulation based on parents’ reported expectations for their children’s
behavior. Similarly, Block and Block (1980) generated yet another parent-report measure of regulation
by assessing Ego Control and Ego Resiliency using a Q-sort method.
Emotion regulation in preschoolers has also been measured using observational or behavioral
methods. One of the earliest and best known measures of regulation is delay of gratification tasks
(Mischel & Baker, 1995; Mischel & Ebbesen, 1970). During these tasks, children’s ability to wait for a
Emotional Competence in Preschoolers
111
TABLE 1. Selected List of Research-Based Regulation Measures
Measure
Approx. Administration Time
Relevant References
Questionnaire Methods
Emotion Regulation subscale
10-20 minutes
of the CBCL 1 1/2 – 5
Achenbach &
Rescorla (2004)
Emotional Self-Control
10-20 minutes
Content Scale of the BASC-2
Reynolds &
Kamphaus (2004)
Children’s Behavior
Questionnaire
Standard form (195 items)
60 minutes
Short form (94 items)
30-40 minutes
Very short form (36 items)
10-15 minutes
Rothbart et al. (2001)
Putnam & Rothbart (2007)
Observation Methods
Delay of Gratification
15 minutes
Mischel & Baker (1995);
Mischel & Ebbesen (1970)
Effortful Control
1-3 minutes each;
Behavioral Tasks
30-40 minutes for total battery
Kochanska &
Knaack (2003);
Kochanska et al. (1997);
Kochanska et al. (2000)
Lack of Control Scale
Caspi et al. (1995);
Henry et al. (1999)
desirable object (e.g., candy) and their regulation methods (e.g., looking away, distraction) are recorded
and coded. According to Metcalfe and Mischel (1999), the ability to shift attention from the tempting
object and delay gratification is a measure of emotion regulation. Furthermore, children’s ability to delay
gratification in the preschool years may be related to adaptation to kindergarten (McIntyre, Blacher, &
Baker, 2006). In a study examining children’s social and behavioral adaptation to kindergarten, McIntyre
et al. found that children’s latency to touch a desired toy during a delay of gratification laboratory task
at age 36 months predicted teacher reports of behavior problems and quality of student-teacher relationships at school entry when children were 5 or 6 years old.
Building on Mischel’s work, Kochanska and colleagues (Kochanska, Coy, & Murray, 2001;
Kochanska & Knaack, 2003; Kochanska et al., 2000; Kochanska, Murray, & Coy, 1997) developed a
series of activities to assess effortful control in preschoolers. These activities involve a child performing
tasks such as moving an animal at various speeds across a game board, whispering, walking a straight
line slowly, and resisting the temptation of candy. Children’s behaviors are recorded and coded as indicators of effortful control.
Another early observational measure of regulation was developed in 1975. Reported by Caspi,
Henry, McGee, Moffitt, and Silva (1995), the Lack of Control observational measure is used to assess
112
The California School Psychologist, 2007, Vol. 12
regulation in preschool children. Henry, Caspi, Moffitt, Harrington, and Silva (1999) found that Lack of
Control scores in preschool interacted with school attendance in predicting criminal behavior in males
during adolescence. That is, school attendance served as a protective factor from later criminal activity
for boys with high Lack of Control scores.
Measures generated by developmentalists are readily available and may be easily integrated into a
multi-method, multi-setting, multi-informant assessment, particularly in evaluations of preschool children. Considering emotion regulation as both a target of assessment and intervention may be particularly
important for school psychologists who support young children with or at-risk for behavior disorders.
Evidence suggests that self-regulation, specifically effortful control, delay of gratification, and negative emotional responses, may predict externalizing and internalizing disorders (e.g., Eisenberg, Fabes,
Guthrie et al., 1996). Furthermore, children who use effective strategies to modulate negative emotional
and behavioral responses are more likely to develop social competence (Eisenberg, Fabes, Karbon et
al., 1996). Although children’s development is undoubtedly affected by genetic and environmental
influences (including parenting), implementing school-based interventions that target the development
of prosocial behavior, self-control, and problem solving may foster children’s emotion regulation and
decrease the likelihood of developing emotional or behavioral disorders.
Implications for Practice
From the available research examining the relation of emotion regulation and dysregulation to children’s behavioral and social developmental trajectories, it is evident that young children who do not
develop regulatory control are likely to exhibit emotional and behavioral problems in both school and
community settings. These difficulties may emerge as students negotiate the preschool – kindergarten
transition. In a recent study, a nationally representative sample of kindergarten teachers reported that over
half of their students had a difficult time adjusting to school (Rimm-Kaufman, Pianta, & Cox, 2000). Top
concerns included difficulties following directions and working independently (Rimm-Kaufman et al.),
both of which may be related to children’s ability to self-regulate.
In light of school entry difficulties, early childhood education programs that equip young children
with school readiness skills have been increasingly emphasized. As an example, the National Education
Goals Panel Document Ready Schools states that all children should have access to high quality and
developmentally appropriate preschool programs in preparation for their transition to formal schooling.
The foremost goal is that “all children in America will start school ready to learn” (National Education
Goals Panel, 1998, p.1).
The early childhood education and school readiness national trends include frameworks for
addressing the social and emotional development of young children in home and school settings (BrooksGunn, Berlin, & Fuligini, 2000). Furthermore, there is increasing advocacy and empirical support for
implementing preventive and early intervention services for young children with behavioral concerns
and poor social-emotional competence using a Positive Behavior Support (PBS) approach (Joseph &
Strain, 2003; Lynch, Geller, & Schmidt, 2004; Webster-Stratton & Reid, 2003).
Embedded within national trends and the empirical literature are important contextual factors that
must be considered when developing early social-emotional interventions for young children. Critical
among those factors are the development of programs based on a resiliency framework that incorporate risk reduction of antisocial behaviors (e.g., noncompliance, defiance, aggression), enhancement of
protective factors, and direct teaching of affective and behavioral skills necessary for the development
of positive social-emotional well-being (Joseph & Strain, 2003; Miller, Brehm, & Whitehouse, 1998).
Emotional Competence in Preschoolers
113
For example, research has demonstrated that the establishment of problem-solving skills, healthy coping
mechanisms, and self-discipline (to name a few) during the preschool years, along with the involvement
of a caring, competent adult predicts more favorable outcomes for children (e.g., Born, Chevalier, &
Humblet, 1997; Cowen, Wyman, Work, & Iker, 1995; Rutter, 1987; Werner, 1993). These findings are
consistent with the basic research literature and highlight the importance of both within-child and interpersonal processes in the development of regulatory skills and biological systems.
Although parent-child relationships are often identified in the developmental literature as crucial
for children’s socioemotional development, the importance of early student-teacher relationships cannot
be underestimated. Evidence suggests that positive student-teacher relationships in kindergarten predict
academic and sociobehavioral adjustment across the elementary school years (Hamre & Pianta, 2001).
In addition, a large body of research supports the use of PBS techniques for teaching and maintaining
appropriate behavior in children (e.g., Eber, Sugai, Smith, & Scott, 2002; Sugai, Horner, & Gresham,
2002) that can be implemented effectively within preschool settings (e.g., Fox & Little, 2001). Closer
examination of such research not only suggests that a host of environmental factors may help buffer
young children from negative influences, but also emphasizes the positive role that parents and early
educational environments have in the development of socially and emotionally competent children.
Therefore, it is essential for social-emotional programming during preschool years to incorporate parent
support/training that coincides with the establishment of positive early educational environments that
utilize PBS strategies.
Parent Support/Training
According to a number of parent surveys, corporal punishment such as spanking, slapping, grabbing, and shoving is used with more than 90% of children in the United States and is most commonly
used with young children (Giles-Sims, Straus, & Sugarman, 1995). Such findings are staggering when
considering that early hostile or negative parent interactions are associated with increased defiance and
disobedience during later development (Keenan & Shaw, 1995). Consideration of parent support, therefore, will need to incorporate instruction that targets increased caregiver sensitivity and appropriate
behavior management skills.
The basic research reviewed clearly identifies caregiver-child interactions as vital to the development of children’s emotion regulation. An important aspect of these interactions is parental sensitivity.
It includes timely and appropriate responding to a child’s efforts to interact, participating in activities
and games that involve turn-taking, and interacting in ways that are equally rewarding for both parent
and child. Helping parents learn how to interact may include providing opportunities to engage in play
routines (playing games) or other transactions that involve turn-taking in which each partner plays a role
that encourages the other in ways that are mutually reinforcing (e.g., peek-a-boo, pat-a-cake). As children grow older, the context of these interactions, and subsequent training opportunities for parents, may
build upon socially mediated conflicts that children encounter. In this regard, the parent may need encouragement in helping, guiding, and reassuring the child as to the most appropriate method for resolving a
particular conflict. Within these situations, it may be effective to offer parenting groups that discuss children’s ongoing social-emotional problems where information is provided and problems are discussed.
Essentially, parents are trained on how to react to their child. From this perspective, parent training
involves a contextualized approach (i.e., parent-child interactions). Such parent training programs can
be readily incorporated into early childhood education programs (e.g., McIntyre & Phaneuf, (in press)
2007) and include aspects supported by findings from the basic and applied literature.
114
The California School Psychologist, 2007, Vol. 12
In addition to promoting positive parent-child interactions, efforts must be taken to support parents
in using discipline effectively (Webster-Stratton, 1990). A complete review of specific strategies is not
within the scope of this discussion, rather a collection of themes is provided so readers can develop
trainings that coincide with the particular needs of families. First, any discipline trainings should begin
with a review of the developmental needs of young children at various age levels. Along with a discussion of the developmental needs of children, parents should be oriented to the differences in parenting
styles. Second, trainings should incorporate discussions/activities that examine the functions (motives)
behind negative behavior(s) observed in young children. Within such discussions, it may be helpful
to signal how “punishing” children may serve to maintain problematic behaviors. Such programmatic
approaches will begin to elucidate the difference between discipline and punishment. Third, training
components should incorporate information on “how to” practice positive discipline in the home by
arranging consistent consequences to increase appropriate behavior or decrease inappropriate behavior
(e.g., positive reinforcement, differential reinforcement). Finally, trainings should include opportunities
for troubleshooting common or specific problems that families encounter.
One example of an evidence-based parent training program that incorporates positive, preventive strategies as well as dealing with childhood challenging behavior is the Incredible Years program
(Webster-Stratton, 1990; 2001). The Division 12 (clinical psychology) task force of the American
Psychological Association deemed Webster-Stratton’s Incredible Years series as one of two well-established psychosocial treatments for childhood conduct problems (Brestan & Eyberg, 1998) based on
effect sizes, sampling, methodology, treatment integrity, and a host of other criteria (Lonigan, Elbert, &
Johnson, 1998). Webster-Stratton’s program (www.incredibleyears.com) offers a number of programs
targeted at parents, teachers, and children (see Webster-Stratton, 2000, for a review).
Overall, programs that provide families with support/training with regard to sensitivity and discipline procedures have demonstrated promising outcomes (e.g., Brooks-Gunn et al., 2000; Yoshikawa,
1995) and are in line with data provided by basic emotion regulation researchers. Moreover, parents who
receive support are more emotionally accommodating, less detached, and have more positive interactions with their children than control group families (Love et al., 2002). Despite evidence that parent
training as a treatment approach is often effective within home settings, similar improvements at school
and with peers have been less encouraging (e.g., Webster-Stratton, 1990). More recent efforts at developing social-emotional competence in young children have turned to early educational environments
that utilize prevention and early intervention strategies.
Early Education Environments
Although early education environments cannot adjust the basic biological foundations of poor
emotion regulation, such environments can provide more opportunities for students to learn the necessary skills for developing positive social-emotional health. For example, early education environments
can be used as resources to teach young children with regulation difficulties directly how to understand
their emotions, as well as adjust their reactions to various environmental stimuli. In order to support
early intervention and prevention efforts, many professionals are working to establish systems of PBS
within preschool settings. For example, Stormont, Lewis, and Buckner (2005) described how features of
PBS could be adapted and implemented with early childhood educational environments (e.g., programs
serving children age 3 years to kindergarten). In addition, Fox, Dunlap, and Cushing (2002) outlined the
logical extension of PBS systems to a preschool level. Essentially, such extensions have advocated for a
three-tiered model of service delivery that incorporates increasing the intensity of instruction to the level
Emotional Competence in Preschoolers
115
of student need (similar to recent suggestions for the use of Response to Intervention, RtI). Embedded
within each tier is a set of instructional practices or evidence-based approaches designed to improve
student outcomes, as well as frequent, ongoing assessments of student skills that are collected to monitor
systematic efforts.
At the universal level (Tier I), all young children should receive sufficient concentration of positive
feedback from teachers and caregivers (Shores, Gunter, & Jack, 1993; Sugai et al., 2002). Therefore, the
foundation of an effective early educational program is the time spent building a strong, positive relationship between educators and children, as well as with families. Investing time and attention in getting
to know children parallels information presented regarding early parent-child interactions. That is, children notice responsive, caring adults and are more likely to pay attention to what a teacher says and does.
Activities may include organized time to engage in frequent, positive adult-child interactions, as well
as opportunities to review basic skills in a fun, interactive manner. Of critical importance at this level
is classroom environment. Specifically, early educators will need to maintain a predictable schedule,
minimize transitions, provide visual reminders of rules, give time and attention for appropriate behavior,
use positive reinforcement to promote appropriate behavior, provide choices where appropriate, and
maximize children’s engagement to minimize problem behaviors (see Lawry, Danko, & Strain, 1999,
for a complete review). Consistent with findings from both basic and applied research literature, the
combination of positive relationships and classroom preventive practices decreases the likelihood of
inappropriate behavior.
At the secondary level (Tier II), social-emotional curricula should be adopted and implemented for
those children who do not respond positively to universal strategies. In fact, some children may need
explicit instruction to ensure they develop competence in emotional literacy, interpersonal problemsolving, and friendship skills (Webster-Stratton & Reid, 2003). Teaching children skills such as how to
play with other children, recognizing and expressing feelings, exercising self-control, and negotiating
conflict situations may result in fewer displays of inappropriate behavior. Specifically, early educators
may want to incorporate lessons that: (a) teach feelings directly through pairing pictures of emotional
expressions with a feeling word, (b) provide practice in recognizing emotions through games (e.g.,
Feeling Face Bingo), and (c) engage in instruction that allows children to observe a model and then role
play specific skills related to social problem-solving (e.g., making coping statements, exploring solutions
to problems) and friendship making skills (e.g., sharing, turn taking, giving compliments). As is the case
with all instruction, effective teaching at the second tier requires careful planning and the adoption of
specified curricula. For a complete review of evidenced-based, social-emotional curricula, see Joseph
and Strain (2003). Most importantly, early educators should be attentive and offer praise/reinforcement
to children when they are engaged in socially competent behavior such as following directions, helping
friends, and sharing.
Even when educators are responsive, implement preventive strategies, and explicitly teach skills, a
few children may continue to display poor social-emotional competence and/or challenging behavior(s).
At this point (tertiary level, Tier III), assessment-based interventions will need to be planned and implemented by a team of individuals in home, early education, and community settings. Most important to
this level is the completion of a functional behavior assessment (FBA) that identifies factors related
to the child’s challenging behavior. When performed by a team of individuals, a complete FBA leads
to the development of a behavior support plan that will include: (a) strategies to prevent problematic
behavior(s) from occurring, (b) techniques for teaching new skills (e.g., social skills groups), and (c)
changes in how family members and/or teachers respond to problematic behavior(s) (Horner, 2000). The
116
The California School Psychologist, 2007, Vol. 12
team then implements this comprehensive plan at home and in preschool settings and monitors changes
in the problem behavior, as well as the development of important social skills or other child outcomes.
Role of School Psychologists
Given the critical contexts in the early childhood field (e.g., national focus on school readiness
and early intervention services), it may be necessary for many school psychologists to extend their
expertise to preschool populations. At the most simplistic level, school psychologists can offer their
expertise and knowledge regarding early childhood development by consulting/collaborating with local
early childcare agencies/centers. Such support to local agencies/centers could assist with planning of
interventions in collaboration with pre-kindergarten and kindergarten teachers, Head Start personnel,
child care providers, administrators, parents, and community members. This level of involvement would
allow for greater ease in transition for preschool programs to kindergarten classrooms. At a more direct
level, school psychologists can organize, develop, and implement parent training programs that focus on
positive parent-child interactions, behavior management skills, and child guidance procedures, and offer
such programs in conjunction with systems that are already established. Furthermore, school psychologists could be part of a school-based team that helps preschool agencies within or outside of their local
school district with the incorporation of PBS principles. Because school psychologists understand the
importance of designing interventions with contextual fit in mind, using prevention and early intervention approaches, and collecting data for monitoring progress, they stand at the forefront of assisting
families and educators with the design, implementation, and evaluation of early care curricula and interventions.
In conclusion, basic research exploring children’s regulation, including emotion displays, delay of
gratification, effortful control, and adult-child interactions, may be particularly useful in developing early
interventions to promote preschool students’ social competence and to decrease emotional and behavioral disorders. Although early childhood populations are not often the emphasis of school psychologists’
efforts, school psychologists may be in a unique position to provide both direct and indirect support to
families and early childhood educators. In so doing, school psychologists may assist with the national
priority of school readiness for all children.
References
Achenbach, T. M., & Rescorla, L. (2004). Manual for the ASEBA preschool forms and profiles. Burlington, VT:
University of Vermont Department of Psychiatry.
Blackford, J. U., & Walden, T. A. (1998). Individual differences in social referencing. Infant Behavior & Development, 21(1), 89-102.
Block, J. H., & Block, J. (1980). The role of ego-control and ego-resiliency in the organization of behavior. In W.
A. Collins (Eds.), Development of cognition, affect, and social relations. The Minnesota symposium on child
psychology (Vol. 13, pp. 39-101). Hillsdale, NJ: Erlbaum.
Born, M., Chevalier, V., & Humblet, I. (1997). Resilience, desistance and delinquent career of adolescent offenders.
Journal of Adolescence, 20, 679–694.
Bornstein, M. H., & Seuss, P. E. (2000). Child and mother cardiac vagal tone: Continuity, stability, and concordance
across the first 5 years. Developmental Psychology, 36(1), 54-65.
Brestan, E. V., & Eyberg, S. M. (1998). Effective psychosocial treatments of conduct-disordered children and
adolescents: 29 years, 82 studies, and 5,272 kids. Journal of Clinical Child Psychology, 27(2), 180-189.
Brooks-Gunn, J., Berlin, L., & Fuligni, A. (2000). Early childhood intervention programs: What about the family? In
J. Schonkoff & S. Meisels (Eds.), Handbook of early childhood intervention (2nd Ed., pp. 549-589). New York:
Cambridge University Press.
Calkins, S. D. (1997). Cardiac vagal tone indices of temperamental reactivity and behavioral regulation in young
children. Developmental Psychobiology, 31(2), 125-135.
Emotional Competence in Preschoolers
117
Calkins, S. D., Smith, C. L., Gill, K. L., & Johnson, M. C. (1998). Maternal interactive style across contexts: Relations to emotional, behavioral, and physiological regulation during toddlerhood. Social Development, 7(3), 350369.
Caspi, A., Henry, B., McGee, R., Moffitt, T. E., & Silva, P. A. (1995). Temperamental origins of child and adolescent
behavior problems: From age three to age fifteen. Child Development, 66, 55-68.
Cole, P. M., Zahn-Waxler, C., Fox, N. A., Usher, B. A., & Welsh, J. D. (1996). Individual differences in emotion
regulation and behavior problems in preschool children. Journal of Abnormal Psychology, 105(4), 518-529.
Cowen, E. L., Wyman, P. A., Work, W. C., & Iker, M. R. (1995). A preventive intervention for enhancing resilience
among highly stressed urban children. Journal of Primary Prevention, 15, 247–260.
Diener, M. L., Mengelsdorf, S. C., McHale, J. L., & Frosch, C. A. (2002). Infants’ behavioral strategies for emotion
regulation with fathers and mothers: Associations with emotional expressions and attachment quality. Infancy,
3(2), 153-174.
Eber, L., Sugai, G., Smith, C. R., & Scott, T. M. (2002). Wraparound and positive behavioral interventions and
supports in the schools. Journal of Emotional and Behavioral Disorders, 10, 171-180.
Eisenberg, N. (2002). Emotion-related regulation and its relation to quality of social functioning. In W. Hartup & R.
A. Weinberg (Eds.), Child psychology in retrospect and prospect: In celebration of the 75th anniversary of the
Institute of Child Development (pp. 133-171). Mahwah, NJ: Erlbaum.
Eisenberg, N., & Fabes, R. A. (1995). The relation of young children’s vicarious emotional responding to social
competence, regulation, and emotionality. Cognition & Emotion, 9(2-3), 203-228.
Eisenberg, N., Fabes, R. A., Guthrie, I. K., Murphy, B., Poulin, R., & Shepard, S. (1996). The relations of regulation and emotionality to problem behavior in elementary school children. Development & Psychopathology, 8,
141-162.
Eisenberg, N., Fabes, R. A., Karbon, M., Murphy, B. C., Wosinski, M., Polazzi, L., Carlo, G., & Juhnke, C. (1996).
The relations of children’s dispositional prosocial behavior to emotionality, regulation, and social functioning.
Child Development, 67, 974-992.
Fabes, R. A., Eisenberg, N., Jones, S., Smith, M., Guthrie, I., Poulin, R., Shepard, S., & Friedman, J. (1999). Regulation, emotionality, and preschoolers’ socially competent peer interactions. Child Development, 70(2), 432-442.
Feldman, R., Greenbaum, C. W., Yirmiya, N., & Mayes, L. C. (1996). Relations between cyclicity and regulation
in mother-infant interaction at 3 and 9 months and cognition at 2 years. Journal of Applied Developmental
Psychology, 17(3), 347-365.
Fox, L., Dunlap, G., & Cushing, L. (2002). Early intervention, positive behavior support, and transition to school.
Journal of Emotional and Behavioral Disorders, 10, 149-157.
Fox, L., & Little, N. (2001). Starting early: Developing school-wide behavior support in a community preschool.
Journal of Positive Behavior Interventions, 3, 251-254.
Giles-Sims, J., Straus, M.A., & Sugarman, D.B. (1995). Child, maternal and family characteristics associated with
spanking. Journal of Applied Family and Child Studies, 44, 170-176.
Goldsmith, H. H., Buss, K. A., & Lemery, K. S. (1997). Toddler and childhood temperament: Expanded content,
stronger genetic evidence, new evidence for the importance of environment. Developmental Psychology, 33(6),
891-905.
Gunnar, M. R., Mangelsdorf, S., Larson, M., & Hertsgaard, L. (1990). Attachment, temperament, and adrenocortical
activity in infancy: A study of psychoendocrine regulation. Developmental Psychology, 25, 355-363.
Henry, B., Caspi, A., Moffitt, T. E., Harrington, H., & Silva, P. A. (1999). Staying in school protects boys with poor
self-regulation in childhood from later crime: A longitudinal study. International Journal of Behavioral Development, 23(4), 1049-1073.
Hamre, B. K., & Pianta, R. C. (2001). Early teacher-child relationships and the trajectory of children’s school
outcomes through eighth grade. Child Development, 72, 625-638.
Horner, R. H. (2000). Positive behavior supports. Focus on Autism and Other Developmental Disabilities, 15, 97105.
Joseph, G. E., & Strain, P. S. (2003). Comprehensive evidence-based social-emotional curricula for young children:
An analysis of efficacious adoption potential. Topics in Early Childhood Special Education, 23, 65-76.
Katz, L. F., & Gottman, J. M. (1995). Vagal tone protects children from marital conflict. Development and Psychopathology, 7, 83-92.
Keenan, K., & Shaw, D. S. (1995). The development of coercive family processes: The interaction between aversive
toddler behavior and parenting factors. In J. McCord (Ed.), Coercion and punishment in long-term perspectives
(pp. 165-180). New York: Cambridge University Press.
Kieras, J. E., Tobin, R. M., Graziano, W. G., & Rothbart, M. K. (2005). You can’t always get what you want:
Effortful control and young children’s responses to undesirable gifts. Psychological Science, 16, 391-396.
118
The California School Psychologist, 2007, Vol. 12
Kochanska, G., Coy, K. C., & Murray, K. T. (2001). The development of self-regulation in the first four years of
life. Child Development, 72(4), 1091-1111.
Kochanska, G., & Knaack, A. (2003). Effortful control as a personality characteristic of young children: Antecedents, correlates, and consequences. Journal of Personality, 71(6), 1087-1112.
Kochanska, G., Murray, K. T., & Coy, K. C. (1997). Inhibitory control as a contributor to conscience in childhood:
From toddler to early school age. Child Development, 68, 263-277.
Kochanska, G., Murray, K. T., & Harlan, E. T. (2000). Effortful control in early childhood: Continuity and change,
antecedents, and implications for social development. Developmental Psychology, 36(2), 220-232.
Lawry, J., Danko, C., & Strain, P. (1999). Examining the role of the classroom environment in the prevention
of problem behaviors. In S. Sandall & M. Ostrosky, (Eds.), Young exceptional children: Practical ideas for
addressing challenging behaviors (pp. 49-62). Longmont, CO: Sopris West.
Linscott, J., & DiGiuseppe, R. (1994). Rational emotive therapy with children. In C. W. LeCroy (Ed.), Handbook of
child and adolescent treatment manuals (pp. 5-40). New York: Lexington.
Lonigan, C. J., Elbert, J. C., & Johnson, S. B. (1998). Empirically supported psychosocial interventions for children:
An overview. Journal of Clinical Child Psychology, 27(2), 138-145.
Love, J. M., Kisker, E. E., Ross, C. M., Schochet, P. Z., Brooks-Gunn, J., Paulsell, D., Boller, K., Constantine, J.,
Vogel, C., Fuligni, A. S., & Brady-Smith, C. (2002). Making a difference in the lives of infants and toddlers and
their families: The impact of early Head Start: Volume 1. Final technical report. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families.
Lynch, K. B., Geller, S. R., & Schmidt, M. G. (2004). Multi-year evaluation of the effectiveness of a resilience-based
prevention program for young children. The Journal of Primary Prevention, 24, 335-346.
McIntyre, L. L., Blacher, J., & Baker, B. L. (2006). Transition to school: Adaptation in young children with and
without developmental delays. Journal of Intellectual Disability Research, 50, 349-361.
McIntyre, L. L., & Phaneuf, L. (In Press) (2007). A three-tier model of parent education in early childhood: Applying
a problem-solving model. Topics in Early Childhood Special Education.
Metcalfe, J., & Mischel, W. (1999). A hot-cool system analysis of delay of gratification: Dynamics of willpower.
Psychological Review, 106(1), 3-19.
Miller, G. E., Brehm, K., & Whitehouse, S. (1998). Reconceptualizing school-based prevention for antisocial
behavior within a resiliency framework. School Psychology Review, 27, 364–379.
Mischel, W., & Baker, N. (1995). Cognitive appraisals and transformations in delay behavior. Journal of Personality
and Social Psychology, 31, 254-261.
Mischel, W., & Ebbesen, E. B. (1970). Attention in delay of gratification. Journal of Personality and Social
Psychology, 16, 329-337.
National Education Goals Panel (1998). Ready schools. Washington, DC: Author.
Porges, S. W., Doussard-Roosevelt, J. A., Portales, A. L., & Greenspan, S. I. (1996). Infant regulation of the vagal
“brake” predicts child behavior problems: A psychobiological model of social behavior. Developmental Psychobiology, 29(8), 697-712.
Putnam, S. P., & Rothbart, M. K. (2007). Development of Short and Very Short Forms of the Children’s Behavior
Questionnaire. Journal of Personality Assessment, 87, 103-113.
Reynolds, C. R. & Kamphaus, R. W. (2004). BASC-2: Behavior assessment system for children, second edition.
Circle Pines: AGS.
Rimm-Kaufman, S., Pianta, R., & Cox, M. (2000). Teachers’ judgments of problems in the transition to kindergarten. Early Childhood Research Quarterly, 15(2), 147-166.
Rothbart, M. K., Ahadi, S. A., Hershey, K. L., & Fisher, P. (2001). Investigations of temperament at 3-7 years: The
Children’s Behavior Questionnaire. Child Development, 72 , 1394-1408.
Rothbart, M. K. & Bates, J.E. (1998). Temperament. In W. Damon (Series Ed.) & N. Eisenberg (Vol. Ed.), Handbook
of child psychology: Vol. 3. Social, emotional, personality development (pp. 105-176). New York: Wiley.
Rubin, K. H., Coplan, R. J., Fox, N. A., & Calkins, S. D. (1995). Emotionality, emotion regulation, and preschoolers’
social adaptation. Development & Psychopathology, 7(1), 49-62.
Rutter, M. (1987). Psychosocial resilience and protective mechanisms. American Journal of Orthopsychiatry, 57,
316–331.
Shores, R. E., Gunter, P. L., & Jack, S. L. (1993). Classroom management strategies: Are they setting events for
coercion? Behavioral Disorders, 18, 92-102.
Spangler, G., Schieche, M., Ilg, U., Maier, U., & Ackermann, C. (1994). Maternal sensitivity as an external organizer
for biobehavioral regulation in infancy. Developmental Psychobiology, 27(7), 425-437.
Stifter, C. A., & Braungart, J. M. (1995). The regulation of negative reactivity in infancy: Function and development.
Developmental Psychology, 31(3), 448-455.
Emotional Competence in Preschoolers
119
Stormont, M., Lewis, T. J., & Beckner, R. (2005). Positive behavior support systems: Applying key features in
preschool settings. Teaching Exceptional Children, 37, 42-49.
Sugai, G. Horner, R. H., & Gresham, F. M. (2002). Behaviorally effective school environments. In M. R. Shinn, H.
M. Walker, & G. Stoner (Eds.), Interventions for academic and behavior problems II: Preventative and remedial
approaches (pp. 315-350). Bethesda, MD: National Association of School Psychologists.
Thompson, R. A. (1994). Emotion regulation: A theme in search of definition. In N. A. Fox (Ed.), Monographs of
the Society for Research in Child Development, 59 (2-3, Serial No. 240).
Tobin, R. M., & Graziano, W. G. (2006). Development of regulatory processes through adolescence: A review of
recent empirical studies. In D. Mroczek & T. Little (Eds.), Handbook of Personality Development (pp. 263-283).
Mahwah, NJ: Erlbaum.
Vondra, J. I., Shaw, D. S., Swearingen, L., Cohen, M., & Owens, E. B. (2001). Attachment stability and emotional
and behavioral regulation from infancy to preschool age. Development & Psychopathology, 13(1), 13-33.
Walden, T. A., & Baxter, A. (1989). The effect of context and age on social referencing. Child Development, 60(6),
1511-1518.
Webster-Stratton, C. (1990). Long-term follow-up of families with young conduct problem children: From preschool
to grade school. Journal of Clinical Child Psychology, 19, 144–149.
Webster-Stratton, C. (2000). The Incredible Years Training series. Office of Juvenile Justice and Delinquency
Prevention Bulletin Review, June, 1-24.
Webster-Stratton, C. (2001). The Incredible Years: Parents, teachers, and children training series. Leader’s guide.
Seattle, WA: author.
Webster-Stratton, C., & Reid, M. J. (2003). Treating conduct problems and strengthening social and emotional
competence in young children: The Dina Dinosaur treatment program. Journal of Emotional and Behavioral
Disorders, 11(3), 130-143.
Werner, E. E. (1993). Risk, resilience, and recovery: Perspectives from the Kauai Longitudinal Study. Development
& Psychopathology, 5, 503–515.
Yoshikawa, H. (1995). Long-term effects of early childhood programs on social outcomes and delinquency. The
Future of Children, 5, 51-75.
120
The California School Psychologist, 2007, Vol. 12
Author Note
This paper was developed, in part, through the authors’ participation in the 2007 School Psychology Research
Collaboration Conference (SPRCC) co-sponsored by The Society for the Study of School Psychology (SSSP), The
National Association of School Psychologists (NASP), Pearson Assessments, Division 16 of the American Psychological Association (APA), Council of Directors of School Psychology Programs (CDSPP), AIMSweb, Trainers of School
Psychologists (TSP), Riverside Publishing, APA Board of Educational Affairs, and Elsevier.
The California School Psychologist, Vol. 12, pp. 121 – 132, 2007
Copyright 2007 California Association of School Psychologists
121
A Comparison of Classification Methods
for Use in Predicting School-Based Outcomes
Erin Dowdy
University of California, Santa Barbara
Randy W. Kamphaus
Georgia State University
There is growing evidence that current classification methods are not consistent with current empirical
knowledge of childhood psychopathology and the optimal way to classify school-age children remains
controversial. The current study investigated three classification methods (categorical, dimensional,
person-oriented) for use in predicting school-based outcomes. Children (grades 1-5; N=558) were
administered the Behavior Assessment System for Children – Teacher Rating Scale and results were
used to form three classification systems. Educational outcome variables were collected seven months
later and the predictive validity of the three classification systems was compared using regression
analyses. Findings indicated that all three methods for predicting educational outcomes were modest
and were best able to predict later grade point averages. Results indicate the relative superiority of
person-oriented and dimensional methods of classification; however these classification methods
warrant further investigation.
Keywords: classification, diagnosis, person-oriented, psychopathology
The fields of psychiatry and psychology have been grappling with the issue of classification for
decades (Achenbach, 1998; 2001). Practitioners, researchers, and educators agree about the importance
of classification for a variety of reasons including enhanced communication among professionals, ease
of description, and the ability to differentiate individuals (Scotti & Morris, 2000; Blashfield, 1998;
Cantwell, 1996). Accurate classification for school-age children is particularly critical considering the
fact that the developmental courses or pathways of children are likely to influence subsequent outcomes
(Jimerson, Coffino, & Sroufe, 2007; Sroufe, Egeland, Carlson, & Collins, 2005). Insight into children’s
adjustment and risk status (Kagan, 1997), tracking developmental pathways (Richters, 1997), differentiating individuals by etiology (Cantwell, 1996), and predicting effective treatment approaches (Scotti
& Morris, 2000) are among the most salient reasons that accurate classification in school-age children is
important. However, children are often classified into groups that receive services only after they exhibit
significant impairment. This “wait-to-fail” treatment approach could result from current classification
systems that fail to identify subsyndromal psychopathology or current risk status. Classification systems
that more accurately identify children for services are thus needed as these systems could effectively aid
daily decisions regarding prevention, early intervention, and treatment for children.
There is growing consensus that current diagnostic systems have lagged behind the increase in
knowledge about psychopathology and classification (Beutler & Malik, 2002; Houts, 2002). Currently,
most school-age children are primarily classified and diagnosed using categorical methods. This approach
uses variables to form “all-or-nothing” categories based on the assumption that disorders form discrete
categories (Millon, 1991). Specifically, students are placed into categories specified by the DSM (DSMIV; American Psychiatric Association) or the Individuals with Disabilities Education Act (IDEA). There
are several limitations to these methods of classification including that only qualitative differences are
Please send correspondence to Erin Dowdy, at University of California, Santa Barbara; Gevirtz Graduate School of
Education; Counseling, Clinical, and School Psychology; Phelps Hall; Santa Barbara,CA, USA; 93106-9490 or e-mail
[email protected]
122
The California School Psychologist, 2007, Vol. 12
noted. However, throughout the scientific literature evidence exists suggesting that symptoms of hyperactivity/impulsivity, inattention, conduct problems, depression, and anxiety occur along a continuum, or
show evidence of quantitative differences (Deater-Deckard, et al., 1997; Fergusson & Horwood, 1995;
Hudziak, Wadsworth, Heath, & Achenbach, 1999; Nease, Volk, & Cass, 1999). Other limitations of
categorical classification methods include the failure to account for comorbidity (van Lier, Verhulst, van
der Ende, & Crijnen, 2003), normally or marginally functional behavioral systems (Jensen, et al., 1996),
or subsyndromal psychopathology (Cantwell, 1996). A study by Scahill, et al. (1999) found that children
beneath the diagnostic threshold for ADHD still possessed evidence of functional impairment in school,
which was nearly identical to the impairment experienced by children above the diagnostic threshold.
This study suggests that under a purely categorical model, such as the DSM-IV or IDEA, students who
experience functional impairment might not be classified, and thus fail to receive services. Furthermore,
there is no differentiation among individuals with lower levels of risk, yielding no useful information for
planning prevention or early intervention services.
Considering limitations of categorical methods, dimensional and person-oriented methods
have been proposed as alternative approaches to classification. Dimensional approaches to classification
assume that behavior does not occur dichotomously, but rather along a continuum. Descriptive variables
are collected and combined with other correlated variables to form a dimension, which summarizes
information about the descriptive variables into an abstract, higher-order variable (Blashfield, 1998).
Dimensional methods of classification improve on categorical methods by accounting for quantitative
differences in symptomatology. Namely, this method includes a wider variety of information and has
the ability to identify and classify all children, not just the ones with the most severe psychopathology.
However, dimensional methods often focus on variables of interest and produce a system that is arguably
less parsimonious than a categorical system (Helzer & Hudziak, 2000).
Person-oriented, or multivariate, methods of classification attempt to blend categorical and dimensional methods by producing a categorical classification system through the use of dimensional scales.
The resulting typology is a different type of categorical classification system that encompasses a full
range of dimensionally scaled variables. Person-oriented approaches have been proposed due to their
strength in emphasizing the individual as a whole, not just a linear combination of variables (Bergman
& Magnusson, 1997), being conducive to a fuller understanding of the complexity and range of child
behaviors (Meehl, 1995; Speece & Cooper, 1991), and providing consistency with psychological theoretical models of psychological systems development (Gottlieb, 2000; Waddington, 1971). Multivariate
behavior typologies, derived through cluster analytic techniques, are also gaining wider acceptance as
a model of classification due to the evidence supporting the relative superiority of multivariate methods
in explaining the complex interactions, correlates, and comorbidities in children (van Lier, et al., 2003;
Greenberg, Speltz, DeKlyen, & Jones, 2001).
However, before behavioral typologies are proposed as an alternative classification method, a direct
comparison of methods is needed. Few systematic comparisons of classification methods have been
conducted. Fergusson and Horwood (1995) examined the relationship between categorical, dimensional, and a series of outcome measures and found dimensional methods to result in stronger predictions of outcomes. However, findings by Jensen et al. (1996) suggest that categorical and dimensional
approaches to classification might produce similar results when similar methods are used, even though
highly specific diagnostic categories show fewer relationships with external validators. Furthermore,
Mattison and Spitznagel (1999) found prior studies comparing DSM categories to Child Behavior
Checklist dimensional scales that suggest that neither system is superior when compared to external
validators.
Comparison Of Classification Methods
123
Theoretically, person-oriented methods of classification are superior to categorical and dimensional methods due to their ability to account for the interactional and additive nature among variables
(Kamphaus, DiStefano, & Lease, 2003; Dowdy, Hendry, & Kamphaus, 2006). However, it is not known
whether person-oriented clusters, derived from such diagnostic tools as teacher rating scales, demonstrate an increased ability to predict future outcomes. The ability of a classification system to predict
future outcomes should guide thinking about its utility (Bergman & Magnusson, 1997). Before clusteranalytically derived typologies can be introduced as alternatives, research must examine their ability
to predict and generalize based on the attributes of the individual (Lessing, 1982). For example, it is
unknown if the additional dimensional scales used to create a person-oriented classification system are
more predictive than the single dimensional scale used in a dimensional system.
Initial research by Flanagan, Bierman, and Kam (2003) found cluster membership to be predictive of
later outcomes for first grade children, and Toshiaki, et al. (1995) found cluster membership to be predictive of outcomes in adults. Additionally, Fergusson and Horwood (1995) found dimensionally scored
measures to show better evidence of predictive validity than categorical methods. A study by Greenberg,
Speltz, DeKlyen, and Jones (2001) found person-oriented methods to be superior to individual variable
approaches in significantly predicting risk factors of conduct problems. However, Haapasalo, Tremblay,
Boulerice, and Vitaro (2000) found prediction of problem behavior in kindergartners to be equally accurate using either cluster or variable approaches. Blanchard, Morgenstern, Morgan, Labouvie, and Bux
(2003) concluded that the utility of clusters to inform clinicians about the future behavior of individuals
is unknown.
These discrepant findings suggest that additional research should be conducted on classification
methods in an attempt to determine the optimal way to classify school-age children. A direct comparison
of methods and information regarding their ability to predict later outcomes is needed. The current study
sought to: (1) classify children into categories according to categorical, dimensional, and person-oriented
methods and (2) provide a comparison of classification methods for predicting school-based outcomes.
Method
Subjects
Data for this study were collected as part of Project ACT Early, funded by Field-Initiated Studies
grants (R306F60158, R305T990330) from the Institute for At-Risk Children of the Office of Educational
Research and Improvement, United States Department of Education. (Grant principal investigators: Jean
A. Baker, Randy W. Kamphaus, and Arthur M. Horne). Project ACT Early was a research grant designed
to study the ecological context of risk in elementary schools and was aimed at teacher professional
development designed to improve classroom management. The sample consisted of 558 children (grades
1-5; N=558) and is approximately one half female (N= 298; 53.4%). Approximately 52% of the children
were African American (N=295), 30% Caucasian (N=169), 7% Hispanic (N=38), 2% Asian American
(N=10), and 2% multiracial (N=13).
Instruments
Children’s behavior problems and adaptive competencies were assessed with the Behavior Assessment System for Children - Teacher Rating Scale - Child (BASC-TRS-C; Reynolds & Kamphaus, 1992),
designed for students ages 6-11. The BASC-TRS-C is a 148-item, nationally standardized measure that
yields ten problem behavior scales and four adaptive behavior scales (Reynolds & Kamphaus, 1992).
124
The California School Psychologist, 2007, Vol. 12
The BASC manual provides reliability and validity psychometric information and descriptions of the
TRS-C scales. The 148 behavioral items are rated on a 4-point Likert scale (1=never, 2=sometimes,
3=often, 4=almost always).
Procedure
BASC-Teacher Rating Scales were collected in the fall of the academic school year for each participating child. Results from the BASC-TRS were used to form three classification models: a categorical
classification model examining symptoms based on DSM-IV criteria (categorical), a dimensional system
based on dimensional scales (dimensional), and a categorical system formed by examining the multiple
dimensions of symptoms exhibited by individuals (person-oriented, cluster). Each child was concurrently placed into these three separate classification systems.
Approximately seven months later, educational outcome variables were collected for each child.
The predictive validity of the three classification systems was compared using regression techniques.
Categorical Classification Model
To construct a model consistent with DSM-IV criteria (American Psychiatric Association,
1994), the BASC-TRS-C was inspected for items with content similar to diagnostic criteria following the
procedure of van Lier, et al., (2003). Based on this analysis, it was determined that a sufficient amount of
items existed to account for symptoms of inattention, hyperactivity, impulsivity, oppositional defiance,
conduct, and anxiety. However, due to sample sizes needed for regression techniques, diagnostic groups
were formed only if 25 individuals from the sample met diagnostic criteria.
The following diagnostic groups were formed based on items consistent with a DSM-IV diagnosis: (1) Attention-Deficit/Hyperactivity Disorder, Predominantly Inattentive Type; (ADHDI; DSM-IV
314.00) and (2) Oppositional Defiant Disorder (ODD; DSM-IV 313.81). To account for the considerable
comorbidity that empirical research has found to occur between behavior disorders (Barkley, 1996), a
3rd diagnostic group was formed that consisted of children with ADHD plus another behavior disorder,
specifically (3) ADHD + CD or ODD.
A 4th diagnostic group was formed to account for the comorbidity between Generalized Anxiety
Disorder (GAD; DSM-IV 300.02) and ADHDI: (4) GAD and GAD + ADHD, Predominantly Inattentive Type. A 5th group, (5) Other, was also formed to capture individuals that met diagnostic criteria for
a disorder with symptoms of inattention, hyperactivity, impulsivity, oppositional defiance, conduct, and
anxiety but could not be analyzed separately due to small sample sizes. In summary, five psychiatric
diagnostic groups were formed: (1) ADHDI, (2) ODD, (3) ADHD + CD or ODD, (4) GAD and GAD +
ADHDI, and (5) Other.
To form these diagnostic groups, items that were consistent with diagnostic criteria were dichotomized where 0 = never or sometimes, and 1= often or almost always true. Individuals who scored above
the diagnostic threshold for one disorder, determined by receiving ratings of often or almost always true
on a sufficient number of items consistent with a particular diagnosis, were placed in that particular
diagnostic category.
Dimensional Classification Model
Scales from the BASC-TRS were combined to form a dimensional classification model. The BASCTRS yields 10 problem behavior scales: Aggression, Hyperactivity, Conduct Problems, Anxiety, Depres-
Comparison Of Classification Methods
125
sion, Somatization, Attention Problems, Learning Problems, Atypicality, and Withdrawal. Two overarching clinical composite dimensions, which are supported by factorial validity evidence, are formed
using these scales: Externalizing and Internalizing Problems. The Externalizing Problems dimension
is formed by combining the Hyperactivity, Aggression, and Conduct Problems scales. The Internalizing Problems dimension consists of the Anxiety, Depression, and Somatization scales (Reynolds
& Kamphaus, 1992). Individuals were assigned T scores on both the Externalizing and Internalizing
dimensions. These dimensional scores were used as the basis for comparison to the other two classification systems.
Person-oriented Classification Model
Teacher ratings of children, using the BASC-TRS, have been utilized in multivariate, or personoriented, methods to develop a classification system for child behavior in school. Kamphaus, et al.,
(1997) used a two-step cluster analytic technique involving a Ward hierarchical analysis followed by an
iterative cluster partitioning via a K-means analysis. A seven-cluster solution was proposed to classify
the behavioral adjustment of children in elementary school. The proposed clusters that were found to
be adequate for classification were (1) Well Adapted, (2) Average, (3) Disruptive Behavior Problems,
(4) Academic Problems, (5) Physical Complaints/Worry, (6) General Problems-Severe, and (7) Mildly
Disruptive. This seven-cluster solution was substantially replicated across: samples in the U.S. population (Kamphaus et al., 1997), a U.S. urban sample (DiStefano, et al., 2003), a U.S. rural sample (DiStefano, et al., 2003), and a sample in Medellin, Colombia (Kamphaus & DiStefano, 2001). For the current
study children were assigned to one of these seven previously constructed behavioral clusters based on
their teachers’ ratings.
Comparison of Classification Models
Once individuals were classified according to categorical, dimensional, and person-oriented methods,
the relationship between the classification models and the ability to predict educational outcome variables was assessed. The following educational outcomes were collected for each child: (1) Grade Point
Average (GPA), (2) Iowa Test of Basic Skills Reading Composite (ITBS Reading; standardized achievement test), (3) Iowa Test of Basic Skills Mathematics Composite (ITBS Math; standardized achievement
test), (4) Number of days absent, (5) Number of days tardy, (6) Number of visits to the Opportunity
Room (OR, indicative of a discipline problem), and (7) Number of Suspensions. These educational
outcomes were collected through examination of school records.
The predictive validity of the classification systems was examined through regression analyses.
Separate regression analyses were computed for each outcome variable. Through regression, unstandardized predicted values of each outcome variable using each classification method were obtained and
used for comparison. Bivariate correlations were computed for each outcome, correlating the outcome
with the unstandardized predicted values obtained using each classification method. Cases were excluded
listwise. Then, T tests were used to compare the differential predictive validity of the three classification
systems to determine if the differences were statistically significant. Specifically, the correlations of classification systems with outcomes were compared with each method (Glass & Stanley, 1970).
126
The California School Psychologist, 2007, Vol. 12
Results
Of the 558 students participating in the cross-sectional study, 166 students met diagnostic criteria
and were placed into one of the following categories: (1) ADHDI, N=30 (2) ODD, N=32; (3) ADHD +
CD or ODD, N=41; (4) GAD and GAD + ADHDI, N=25; and (5) Other, N= 38. T scores on the externalizing and internalizing dimensions were calculated for all 558 students. Scores ranged from 40 to 95 and
39 to 101 respectively. Additionally, each student was placed into one of the person-oriented clusters:
(1) Well Adapted, N= 147 (2) Average, N=87 (3) Disruptive Behavior, N=82 Problems, (4) Academic
Problems, N=66, (5) Physical Complaints/Worry, N=60 (6) General Problems-Severe, N=26 and (7)
Mildly Disruptive, N=90.
Overall Strength of Prediction
Multiple regression techniques were used to predict GPA, ITBS reading and math scores, and
number of days absent, days tardy, opportunity room visits, and suspensions using categorical (DSM),
dimensional (externalizing, internalizing), and person-oriented (cluster) classification methods. Table 1
lists the overall R squared values for this study.
TABLE 1. Classification Methods Predicting Educational Outcomes
R squared values
Outcomes
Categorical
Dimensional
Cluster
GPA
ITBS Read
ITBS Math
# of Days Absent
# of Days Tardy
# of OR visits
# of Suspensions
.197
.047
.074
.030
.006
.320
.107
.200
.082
.086
.048
.013
.416
.138
.366
.110
.100
.060
.017
.294
.079
Note: GPA = Grade Point Average; ITBS Read = Iowa Test of Basic Skills Reading composite; ITBS Math = Iowa Test
of Basic Skills Mathematics composite; OR = Opportunity Room
Predicting Academic Outcomes
GPA. The ability of the three classification methods to predict GPA and standardized achievement
scores was analyzed. Correlations between three academic outcomes (GPA, Iowa Test of Basic Skills
Reading and Mathematics composites) and the unstandardized predicted values using the three classification methods (categorical, dimensional, person-oriented or cluster) were analyzed separately. In order
to make inferences about the equality of the population correlation coefficient values that used the same
sample, T tests were employed. Overall, results suggest that person-oriented methods predicted GPA
significantly better than either dimensional or categorical methods, while there was no significant difference in the prediction of GPA using dimensional or categorical methods. (See Table 2.)
Comparison Of Classification Methods
127
TABLE 2. Correlations between GPA and Predicted Values of GPA
GPA
PGPAC
PGPACL
GPA
PGPAC
PGPACL
PGPAD
1
.443
.605
.447
.650
.735
.694
PGPAD
Note: GPA = Grade Point Average; PGPAC = Predicted GPA using Categorical method; PGPACL = Predicted GPA
using CLuster, person-oriented method; PGPAD = Predicted GPA using Dimensional method
Reading and Math Achievement. The Iowa Test of Basic Skills, Reading (ITBSRead) and Mathematics (ITBSMath) Composites were used as indicators of reading and math achievement. T-tests among
correlations between ITBSMath and the predicted values using the three classification methods did not
yield any significant differences, suggesting that the superiority of any method cannot be established for
use in predicting mathematics achievement scores. However, results from T-tests among correlations
with ITBSRead, suggest that the person-oriented method and the dimensional method predicted reading
scores significantly better than the categorical method. (See Table 3.)
TABLE 3. Correlations between ITBS Reading and Predicted Values
ITBSRead
PITBSReadC
PITBSReadCL
ITBSRead
PITBSReadC
PITBSReadCL
PITBSReadD
1
.217
.232
.287
.583
.653
.747
PITBSReadD
Note: ITBSRead = Iowa Test of Basic Skills Reading composite; PITBSReadC = Predicted ITBSRead using
Categorical method; PITBSReadCL = Predicted ITBSRead using CLuster, person-oriented method; PITBSReadD =
Predicted ITBSRead using Dimensional method
Predicting Attendance and Behavioral Outcomes
The ability of these three classification methods to predict the following outcomes throughout the
school year was examined: number of days absent, number of days tardy, number of times a student
visited the opportunity room (OR, an indicator of discipline problems), and number of suspensions.
Correlations between the outcomes and the predicted values of the outcomes using the three different
classification methods were examined separately.
Days Absent/Tardy. Results from T tests indicated that the person-oriented method predicted the
number of days absent significantly better than the categorical method. However, no other significant
differences were noted for days absent. (See Table 4.) No significant differences were noted between
the three possible methods of predicting the number of days a student was tardy.
Opportunity Room Visits. When examining the ability of the classification methods to predict
Opportunity Room visits, findings suggest that the dimensional method is superior to the categorical and
person-oriented method for predicting the number of OR visits. No significant differences were noted
between person-oriented and categorical methods. (See Table 5.)
128
The California School Psychologist, 2007, Vol. 12
TABLE 4. Correlations between Days Absent (Absent) and Predicted Values
Absent
PAbsentC
PAbsentCL
Absent
PAbsentC
PAbsentCL
PAbsentD
1
.174
.246
.219
.467
.429
.734
PAbsentD
Note: Absent = Number of school days Absent; PAbsentC = Predicted Absent using Categorical method; PAbsentCL =
Predicted Absent using CLuster, person-oriented method; PAbsentD = Predicted Absent using Dimensional method
TABLE 5. Correlations between Number of Visits to OR and Predicted Value
#OR
P#ORC
P#ORCL
#OR
P#ORC
P#ORCL
P#ORD
1
.565
.542
.645
.723
.812
.797
P#ORD
Note: #OR = Number of visits to the Opportunity Room; P#ORP = Predicted #OR using Categorical method; P#ORCL
= Predicted #OR using CLuster, person-oriented method; P#ORD = Predicted #OR using Dimensional method
Suspensions. T tests examining the significant differences using the three classification methods
indicated that the dimensional method of classification was superior to the person-oriented method when
predicting suspensions. No other significant differences were noted. (See Table 6.)
TABLE 6. Correlations between Number of Suspensions and Predicted Values
#Suspend
P#SuspendC
P#SuspendCL
#Suspend
P#SuspendC
P#SuspendCL
P#SuspendD
1
.327
.281
.371
.628
.663
.796
P#SuspendD
Note: Absent = Number of school days Absent; PAbsentC = Predicted Absent using Categorical method; PAbsentCL =
Predicted Absent using CLuster, person-oriented method; PAbsentD = Predicted Absent using Dimensional method
Discussion
The aim of this study was to compare categorical, dimensional, and person-oriented methods of
classification for use in predicting school-based outcomes. Through examination of overall R squared
values, the value of the categorical, dimensional, and person-oriented methods for predicting educational
outcomes was modest. All three classification approaches yielded results suggesting that they were best
able to predict later grade point averages (GPA) and number of visits to the opportunity room when
compared with other outcome variables. However, the overall ability of these classification models for
use in predicting days absent, days tardy, and reading and math achievement is questionable.
Comparison Of Classification Methods
129
Despite somewhat unfavorable results suggesting that these classification methods were not optimal
for predicting educational outcomes, differences among the classification methods were revealed. When
examining GPA, person-oriented methods were clearly superior to both dimensional and categorical
methods. In schools, GPA is often a global indicator of functioning in the classroom suggesting that
person-oriented methods might allow for the prediction of global functioning. Similarly, person-oriented
methods were found to be superior to categorical methods for predicting reading achievement scores and
days absent.
Dimensional methods of classification were found to be superior to categorical methods for predicting
reading achievement scores and number of visits to the opportunity room. Dimensional methods were
also found to be better able to predict number of visits to the opportunity room and number of suspensions than person-oriented methods of classification. This finding suggests that, for behavioral outcomes,
knowledge about a student’s externalizing and internalizing functioning might be sufficient. In other
words, the additional dimensional scales used to create a person-oriented classification system were not
more predictive than the two dimensional scales used in the dimensional system.
In the current study, categorical classification methods using DSM criteria were not found to be
superior for predicting any of the educational outcomes. This knowledge is significant when considering that students in educational systems are currently being classified according to categorical methods
(DSM or IDEA). Person-oriented or dimensional methods of classification were found to better predict
grade point average, standardized reading achievement measures, number of days absent, number of
visits to the opportunity room, and number of suspensions than categorical classification methods.
Similar to results found by Fergusson and Horwood (1995), the current study found the dimensional
classification system to show better evidence of predictive validity than a purely categorical system.
However, this study did not replicate the findings by Greenberg and colleagues (2001) that indicated the
relative superiority of person-oriented methods over individual variable approaches in predicting risk
factors of conduct problems. Specifically, the dimensional method utilized in the current study appeared
to be superior to person-oriented methods when predicting behavioral outcomes. When predicting
many of the educational outcomes, results of the current study were more consistent with findings by
Haapasalo et al. (2000), suggesting few differences between the cluster and dimensional approaches.
While the ability of person-oriented, dimensional, and categorical classification methods to predict
educational outcomes warrants further investigation, the results of this study reveal the relative superiority of person-oriented and dimensional methods of classification over the frequently used categorical
methods. However, it remains unclear if person-oriented methods are superior to dimensional methods
when predicting behavioral outcomes. There are a number of limitations in this study that should be highlighted. This study was limited in
the availability of behavioral outcomes, which would be hypothesized to be more highly correlated with
classification systems utilizing teacher ratings of emotional and behavioral functioning. Additionally,
the validity of some of the outcome measures, particularly number of suspensions, is questionable due
to the fact that they are based on complex teacher and school processes beyond the child’s problems.
Another limitation of this study is that the three classification methods were formed based on results of
one instrument, the BASC-TRS. Information regarding a student’s categorical classification, such as a
current DSM diagnosis, was unavailable. However, it would have been desirable to obtain diagnostic
information through a diagnostic semi-structured interview, such as the Diagnostic Interview Schedule
for Children (DISC-IV), or another comprehensive measure to assist in forming the categorical classification model.
130
The California School Psychologist, 2007, Vol. 12
Despite limitations, these results are consistent with previous research suggesting the inadequacies
of current categorical classification methods. As school psychologists are often called upon to place
students into categories, such as determining their eligibility for special education, they should be aware
of the limitations of current classification methods used in the schools. The ability of current classification models to predict later school outcomes is questionable. Additionally, previous research suggests
that students, while they might not meet criteria for special education placement, could be experiencing
functional impairment or lower levels of risk. This points to the need for prevention and early intervention services for students experiencing significant functional impairment or risk, regardless of classification or placement in special education. Furthermore, school psychologists would benefit from gathering comprehensive information regarding students’ functioning, an approach more consistent with
dimensional and person-oriented methodologies. Relying solely on information regarding placement
eligibility, such as that obtained for categorical classification, might prove insufficient and further the
“wait to fail” treatment approach. It should also be emphasized that the present findings point to the need
for future research into classification methods for use with school-age children. Particularly, methods
utilizing a person-oriented or dimensional approach to classification should be further investigated.
References
Achenbach, T. (1998). Diagnosis, assessment, taxonomy, and case formulations. In T. Ollendick and M. Hersen
(Eds.). Handbook of Child Psychopathology (3rd ed.) (pp. 63-87). New York: Plenum Press.
Achenbach, T. (2001). Challenges and benefits of assessment, diagnosis, and taxonomy for clinical practice and
research. Australian and New Zealand Journal of Psychiatry, 35 (3), 263–271.
American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders (4th ed). Washington, DC: Author.
Barkley, R. A. (1996). Attention-deficit/hyperactivity disorder. In E.J. Mash & R.A.
Barkley (Eds.), Child Psychopathology. New York: Guilford Press.
Bergman, L. R., & Magnusson, D. (1997). A person-oriented approach in research on developmental psychopathology. Development & Psychopathology, 9(2) 291-319.
Beutler, L. E. & Malik, M. L. (2002). Rethinking the DSM: A psychological perspective. Washington, DC: American
Psychological Association.
Blanchard, K. A., Morgenstern, J., Morgan, T. J., Labouvie, E. & Bux, D. A. (2003).
Motivational subtypes and continuous measures of readiness for change: concurrent and predictive validity.
Psychology of Addictive Behaviors, 17, 1, 56-65.
Blashfield, R. K. (1998). Diagnostic models and systems. In A. S. Bellack, M. Hersen, & C. R. Reynolds (Eds.),
Comprehensive clinical psychology: (Vol.4) Assessment. New York: Elsevier Science.
Cantwell, D. P. (1996). Classification of child and adolescent psychopathology. Journal of Child Psychology and
Psychiatry, 37, 3-12.
Deater-Deckard, K., Reiss, D., Hetherington, E. M., & Plomin, R. (1997). Dimensions and disorders of adolescent
adjustment: A quantitative genetic analysis of unselected samples and selected extremes. Journal of Child
Psychology and Psychiatry, 38(5), 515-525.
DiStefano, C., Kamphaus, R. W., Horne, A. M., & Winsor, A. P. (2003). Behavioral Adjustment in the U.S. Elementary School: Cross-validation of a Person-oriented Typology of Risk. Journal of Psychoeducational Assessment,
21, 338-357.
Dowdy, E., Hendry, C., & Kamphaus, R. W. (2006). Clusters of child adjustment. In R.W. Kamphaus & J. M. Campbell, (Eds.), Psychodiagnostic Assessment of Children (pp. 437-464). Hoboken, NJ: John Wiley & Sons, Inc.
Fergusson, D. M., & Horwood, J. (1995). Predictive validity of categorically and dimensionally scored measures
of disruptive childhood behaviors. Journal of the American Academy of Child and Adolescent Psychiatry, 34,
477-487.
Flanagan, K. S., Bierman, K. L., & Kam, C. M. (2003). Identifying at-risk children at school entry: the usefulness of
multibehavioral problem profiles. Journal of Clinical Child and Adolescent Psychology, 32, 396-407).
Glass, G., & Stanley, J. (1970). Statistical methods in education and psychology. Oxford, England: Prentice-Hall.
Comparison Of Classification Methods
131
Gottlieb, G. (2000). Understanding genetic activity within a holistic framework. In L. R. Bergman, R. B. Cairns, L.
G. Nilsson, & L. Nested, (Eds.), Developmental Science and the Holistic Approach (pp. 180-201). Mahwah, NJ:
Lawrence Erlbaum Associates, Inc.
Greenberg, M. T., Speltz, M. L., DeKlyen, M. & Jones, K. (2001). Correlates of clinic referral for early conduct
problems: variable- and person-oriented approaches. Development and Psychopathology, 13, 255-276.
Haapasalo, J., Tremblay, R. E., Boulerice, B., & Vitaro, F. (2000). Relative advantages of person–and variable-based
approaches for predicting problem behaviors from kindergarten assessments. Journal of Quantitative Criminology, 16, 2, 145-168.
Helzer, J. E. & Hudziak, J. J. (2000). Defining psychopathology in the 21st century: DSM-V and beyond. Washington, DC: American Psychiatric Association.
Houts, A. C. (2002). Discovery, invention, and the expansion of the modern diagnostic and statistical manuals of
mental disorders. In Defining psychopathology in the
21st century: DSM-V and beyond. Washington, DC: American Psychiatric Association. Hudziak, J. J., Wadsworth,
B. A., Heath, A. C., & Achenbach, T. M. (1999).
Latent class analysis of child behavior checklist attention problems. Journal of the American Academy of Child and
Adolescent Psychiatry, 38, 985–991.
Jensen, P. S., Watanabe, H. K., Richters, J. E., Roper, M., Hibbs, E., Salzberg, A., & Liu, S. (1996). Scales, diagnosis, and child psychopathology: II. Comparing the CBCL and the DISC against external validators. Journal of
Abnormal Child Psychology, 24(2), 151-168.
Jimerson, S. R., Coffino, B., & Sroufe, L. A. (2007). Building school-based Interventions on attachment theory and
research. Journal of Early Childhood and Infant Psychology, 3, 79-94.
Kagan, J. (1997). Conceptualizing psychopathology: the importance of developmental profiles. Development and
Psychopathology, 9, 321-334.
Kamphaus, R. W. & DiStefano, C. (2001). Evaluación multidimensional de la psicopatología infantíl. Revista de
Neuropsicología, Neuropsyqiatría y Neurociencias. 3(1), 85-98.
Kamphaus, R. W., DiStefano, C., & Lease, A. M. (2003). A self-report typology of behavioral adjustment for young
children. Psychological Assessment, 15(1), 17-28.
Kamphaus, R. W., Huberty, C. J., DiStefano, C., & Petoskey, M. D. (1997). A typology of teacher rated child
behavior for a national U. S. sample. Journal of Abnormal Child Psychology, 25, 453-463.
Lessing, E. E., Williams, V., & Gil, E. (1982). A cluster-analytically derived typology: feasible alternative to clinical
diagnostic classification of children? Journal of Abnormal Child Psychology, 10, 451-482.
Mattison, R. E. & Spitznagel, E. L. (1999). Long-term stability of child behavior checklist profile types in a child
psychiatric clinic population. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 700707.
Meehl, P. E. (1995). Bootstraps taxometrics: Solving the classification problem in psychopathology. American
Psychologist, 50, 266-275.
Millon, T. (1991). Classification in psychopathology: rationale, alternatives, and standards. Journal of Abnormal
Psychology, 100, 245-261.
Nease, D. E., Volk, R. J., & Cass, A. R. (1999). Investigation of a severity-based classification of mood and anxiety
symptoms in primary care patients. Journal of the American Board of Family Practice, 12(1), 21-31.
Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children. Circle Pines, MN: American Guidance Service, Inc.
Richters, J. E. (1997). The Hubble hypothesis and the developmentalists’ dilemma. Development and Psychopathology, 9(2), 193-229.
Scahill, L., Schwab-Stone, M., Merikangas, K. R., Leckman, J. F., Zhang, H., & Kasl, S. (1999). Psychosocial and
clinical correlates of ADHD in a community sample of school-age children. Journal of the American Academy
of Child and Adolescent Psychiatry, 38, 976–984.
Scotti, J. R., & Morris, T. L. (2000). Diagnosis and classification. In M. Hersen & R. T. Ammerman (Eds.) Advanced
abnormal child psychology. Mahwah, NJ: Erlbaum.
Speece, D. L., & Cooper, D. H. (1991). Retreat, regroup, or advance? An agenda for empirical classification research
in learning disabilities. In L.V. Feagans, E.J. Short, & L.J. Meltzer (Eds.), Subtypes of learning disabilities. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Sroufe, L. A., Egeland, B., Carlson, E. A., & Collins, W. A. (2005a). The development of the person: The Minnesota
study of risk and adaptation from birth to adulthood. New York: Guilford Press.
Toshiaki, F., Awaji, R., Nakazato, H., & Sumita, Y. (1995). Predictive validity of subtypes of chronic affective disorders derived by cluster analysis. Acta Psychiatrica Scandinavica, 91, 379-385
132
The California School Psychologist, 2007, Vol. 12
Van Lier, P.A.C., Verhulst, F.C., van der Ende, J., & Crijnen, A.A.M. (2003). Classes of disruptive behaviour in a
sample of young elementary school children. Journal of Child Psychology and Psychiatry and Applied Disciplines, 44, 377-387.
Waddington, C.H. (1971). Concepts of Development. In L.R. Aronson, E. Shaw, & E Tobach, (Eds.), Biopsychology
of Development (pp. 17-23). New York: Academic Press.