Download Essentials of Processing Assessment

Using Selective, Cross-Battery
Assessment to Identify Children’s
Psychological Processing Deficits
Milton J. Dehn, Ed.D., NCSP
Schoolhouse Educational Services
Fall 2012
Notice of Copyright 2012
This PowerPoint presentation and
accompanying materials are copyrighted
by Schoolhouse Educational Services,
LLC and Milton J. Dehn. They are not to
be reprinted, copied, or electronically
disseminated without written permission.
To obtain permission, email
[email protected]
Workshop Information Sources
• References in handouts
• Dehn, M. J. (2006). Essentials of
Processing Assessment (revision in 2013)
• Dehn, M. J. (2012). Children’s
Psychological Processes Scale (CPPS)
• Presenter email
[email protected]
• www.psychprocesses.com
Workshop Topics
1.
2.
3.
4.
5.
6.
7.
Overview of psychological processes
Processes to assess
Processes and academic skills
Cross-battery testing
Analyzing cross-battery results
Using processing rating scales
Case studies
Psy Processing and LD
• Federal Definition: “a disorder in one or
more of the basic psychological
processes”
• Disorders other than LD also have
processing disorders/problems
Author’s Personal Goals
1. Help LD students realize their potential
2. Earlier identification of LD
3. Assessment that benefits the learner by
increasing understanding of “why”
4. Develop measurement tools that solve a
problem & have high technical standards
5. Add to the research on psychological
processing deficits
What are psychological processes?
1. Brain processes, operations, functions
2. Any time mental contents are operated on
3. When information is perceived, transformed,
manipulated, stored, retrieved, expressed
4. Whenever we think, reason, problem-solve
5. Can’t learn and perform without processing
6. Learning depends on these processes
7. Doesn’t include knowledge or academic skills
Human Processing Limitations
1. Human limitations
Problems with Past Processing
Assessment
1.
2.
3.
4.
5.
6.
Not enough known about processing
Not sure how to test processing
Assessment too informal
Did not include higher level processes
Memory seldom assessed
Controversies over the value of
processing assessment
7. Some ineffective interventions
Improvements in Processing
Assessment and Intervention
1. More and better processing tests
2. Neuroscience and neuropsychological
evidence on brain and learning
3. Neuropsychological influence on assessment
(what underlies the learning problems)
4. Research on processing, memory, academics
5. More assessment of memory
6. Updated & new research on interventions
7. New interventions
What Have We Learned
1. More about the processes needed for
academic learning
2. The specific areas of the brain involved in
childhood disorders
3. Childhood disorders, including SLD’s
have a neurobiological basis
4. Interventions should be evidence-based
5. The discrepancy model did not work well
for LD identification
Why Processing Assessment is a
Good Idea
1. Benefits the learner, understanding
2. Processing deficits underlie SLD
3. Identifying a processing deficit
differentiates between SLD & slow
learning better than IQ testing
4. Interventions for processing deficits
5. Identifying processing deficits provides
direction for academic interventions
The Need for More
Processing Assessment
1. Part of a neuropsychological approach
2. Not just for LD; e.g. executive for ADHD
3. In depth, brain-related assessment
needed because of more children with
medical conditions; e.g. head trauma
4. Private practitioners often don’t connect
with school environment needs and
criteria
Processing and RTI
1. Processing model consistent with problemsolving and early intervention
2. RTI purists are not current; too behavioral
3. Evidence-based processing interventions
4. Different processing causes of disabilities
5. Why do “blind” general academic interventions
6. Processing & acad. interventions compatible
7. Which academic interventions succeed?
1. Those that address processing problems
“Resistance” to Intervention
• Successful achievement depends on
adequate psychological processes
• Processing deficits create roadblocks to
learning
• A processing deficit can cause an
academic intervention to fail
– When severe
– When not addressed
Characteristics of Psychological
Processes to Assess
1.
2.
3.
4.
5.
Brain-based
Interrelated
Necessary for academic learning
Broad and narrow processes
Related behaviors observable in
classroom
Brain Lobes and Processes
• Frontal Lobes: Attention, Executive
Functions, Working Memory
• Temporal Lobes: Long-Term Memory,
Auditory Processing, Short-Term Memory
• Parietal Lobes: Fine motor, Working
Memory, Auditory, Phonological,
• Occipital Lobes: Visual-spatial processing
Dehn’s Recommended Processes
for SLD Assessment
1. Attention
2. Auditory Processing
3. Executive Functions
4. Fine Motor
5. Fluid Reasoning
6. Long-Term Recall
7. Oral Language
8. Phonological Processing
9. Processing Speed
10. Visual-Spatial Processing
11. Working Memory
Attention
1. Necessary for learning and memory
2. Attention deficits part of LD; not necessarily
ADHD
3. Types: Selective, focused, divided, sustained
4. The problem is attentional control & lack of
inhibition
5. Related to Executive Functions and Working
Memory
Auditory Processing
1. Ability to perceive, analyze, synthesize,
and discriminate auditory stimuli, mainly
speech
2. Perceiving and comprehending
instruction; being able to understand
words with background noise
Executive Functions
1. Management of cognitive functions and
psychological processes
2. Effectiveness depends on self-monitoring, selfregulation, and metacognition
3. Has a longer course of development
4. More to do with classroom performance than
learning of academic skills
Fine Motor
1. Hits developmental plateau by age 7
2. Has weaker relations with cognitive
processes in general but has strong
relations with academics
3. Often pairs up with visual-spatial
processing
Fluid Reasoning
1. Deductive, inductive reasoning,
especially with novel materials
2. Has a longer course of development
3. More important for applied academics
Long-Term Recall
1. Close connection with other processes
and with academic learning in general
2. Includes encoding, consolidation,
storage, and retrieval
3. Rapid automatic naming is part of
Oral Language
1. Not the content (vocabulary) or receptive
language but the oral expression
processes
Phonological Processing
1. Processing of phonemes, e.g. blending
2. Phonemic awareness is part of
Processing Speed
1. How quickly information flows through the
processing system; a matter of efficiency
2. Too slow: info. lost, process not
completed
Visual-Spatial Processing
1. The ability to perceive, analyze,
synthesize, manipulate and think with
visual patterns
2. A strength in most LD cases
3. Weak relations with all academics; more
of a “threshold” process
Working Memory
1. Processing while retaining information
2. Includes short-term memory
3. Both verbal and visual
Processes and Academic Learning
1. Psychological processes are like
“aptitudes”
2. Relations established through research
1. Flanagan et al., & McGrew’s review
2. Swanson, Geary, and others
3. The influence of processes varies by age
4. Look for academic area and related
psychological processes to both be low
5. See Table
Research: SLD by Processing
Subtypes
1. Visual-Spatial Deficits: Math calculation
and math problem solving
2. Processing Speed Deficits: Reading
comprehension, written expression
3. Working Memory Deficit: Math
calculation, Written expression
4. Attention: Written expression
Source: Hain, Hale, Kendorski
Development of Processes
• Early gradual development followed by
long-term plateau: auditory, visual, longterm memory, fine motor
• Rapid early development followed by
gradual decline after age 25: processing
speed, short-term memory
• Gradual development into teens/20s:
attention, executive, fluid reasoning, WM
• Basic functions develop earlier; higher
level, such as executive later
Highest Influence of Processes on
Academics by Grade
• Early elementary: Phonological, visual,
auditory, rapid automatic naming, STM,
sequencing, fine motor control
• Late elementary and beyond: Executive,
working memory, long-term memory
• All grades: Processing speed, attention,
oral expression
Processing Assessment
Challenges
1.
2.
3.
4.
5.
6.
7.
Connecting to academic concerns
Interrelated processes
Informal methods lack validity
Not found in one convenient battery
Doing it efficiently
Having enough expertise
Linking with interventions
Processing Assessment
1.
2.
3.
4.
5.
6.
7.
8.
Should be multimethod/multisource
Multi-settings
Formal and informal
Should include standardized testing
Should be hypothesis driven (suspected)
Based on referral concerns
Selective, cross-battery testing
Integrate data during interpretation
Selective, Cross-Battery Testing
1. Assess areas based on concerns, not on
what a test has to offer
2. Mix tests/batteries to cover all the areas
1. Limit to 2 or 3 batteries
2. Should be normed about the same time
3. Avoid redundancies
4. Ideally, 2 subtests per process
5. Analyze results together by computing a
cross-battery mean or using IQ
Dehn’s Approach to Cross Battery
1. Not limited to CHC factors in Flanagan
and Ortiz method
2. Includes processing factors that are
important for learning of academic skills
3. Analyze scores at the factor (twosubtest) level whenever possible
Hypothesis Testing Approach
1. Collect preliminary data
1. Records review, observation, interview
2. Identify referral concerns
3. Generate processing hypotheses
(suspected processing problems)
4. Select assessment methods
1. Cover all hypotheses
2. Avoid redundancies
Selective Testing
1. Apply a cross-battery approach
2. See selective testing table for cog. &
ach. Link
3. Use a hand computation analysis sheet
or the Psychological Processing
Analyzer
Completing the Processing
Assessment Planner
1. Enter academic learning referral concerns
2. Look at Table 1: Processes and learning
3. Enter hypothesized, suspected processes
4. Select assessment method
5. Look at selective testing tables
6. Select battery
7. Select specific subtest or factor
Planner
Completed Example
Cross-Battery Analysis of Scores
1. See Processing Analysis Worksheet
2. Get factor scores from test manual when
possible
3. Exclude non-processing factors and subtests
4. Compute clinical factor scores by averaging
1. First convert scaled scores to 100/15 metric
5.
6.
7.
8.
Compute processing mean or use IQ
Calculate difference scores
Determine weaknesses and deficits
Do pairwise comparisons Completed Example
Examples of Logical Processing
Pairs
•
•
•
•
•
•
•
•
•
Should be opposites or closely related
Visual vs auditory
STM vs LTR
Phonological vs STM
Attention vs Executive
Auditory vs verbal expression
Verbal expression vs fine motor control
WM vs STM and WM vs LTR
Not related ex.: auditory and fine motor
Identifying Strengths and
Weaknesses
1. These are intra-individual strengths and
weaknesses
1. At least a 12 point standard score difference from
the mean of processing scores
2. As opposed to normative weaknesses
1. Below average score (below 90)
3. For diagnosis, should be both an intraindividual weakness and a normative weakness
4. When it’s both, Dehn refers to as “deficit”
Support for Strengths and
Weaknesses
1. Informal data supports test scores and
results of score analysis
2. Corroboration especially needed when
scores differences are less than one
standard deviation (12-14 points)
3. Integrate data when writing report
4. They match with specific academic areas
they are highly related to
Evidence for a Processing Disorder
and SLD Diagnosis
1. It’s not specific to one environment
2. A normative weakness (below average score)
3. Intra-individual: score is significantly weaker than
predicted from discrepancy analysis
4. Best if it’s an intra-individual weakness and a
normative weakness (this is a deficit)
5. It’s impairing academic learning
6. The low psychological processes and low
academics have research-based links
7. The linked process and academic skills both have
low scores (consistency approach)
8. Non-LD also have strengths and weaknesses
Consistency Approach
• With processing, use a consistency
approach, not a discrepancy approach
– Low process + low academic skill = SLD
– NOT high process + low academic skill
Case Study: “Jacob”
1.
2.
3.
4.
5.
6.
7.
8.
Age 13; 7th grade
Foster care; special ed placement
3 months premature; failure to thrive
Early elementary IQ of 70; recent IQ of 95
ADHD diagnosis; poor organization
Social skills problems
Difficulty completing homework
Moderately high test anxiety
Case Study Continued
1.
2.
3.
4.
5.
6.
7.
Likes to read
Struggles with Math and Written Lang.
Reading Composite – 106
Math Composite – 88
Wr. Lang. Composite – 73
Oral Lang Composite - 87
List your processing hypotheses to account for
low achievement and background factors
Psychological Processing
Analyzer
1. Available at www.psychprocesses.com
2. To identify pattern of strengths and
weaknesses
3. Conducts cross-battery analysis using
composites and/or subtest scores
4. 11 psychological processes
5. From 22 different scales: cognitive,
achieve., rating, and processing scales
(list)
Psychological Processing
Analyzer
1. Composite and subtests are limited to
those that are fairly direct measures
2. Some are re-classified based on the
primary demands of the task; example
3. Difference formulas based on reliability
coefficients of composites/subtests
4. Regression toward the mean
5. .01 or .05 level of significance
Psychological Processing
Analyzer
1. All scores converted to standard scores
2. Non-unitary process scores are flagged
3. Predicted score for each processes
based on mean of other 10
4. Differences greater than critical values
are intra-individual weaknesses
5. Deficits are both types of weaknesses
6. Pairwise comparisons are optional
PPA Case Study
Struggling 2nd year college student
Significant problems with math
Intensive tutoring in past
Indications of memory problems
Family history of LD
Administered WJ III COG., WMS-IV, and
parts of WAIS-IV
7. See results
1.
2.
3.
4.
5.
6.
Measures of Specific Processes
•
•
•
•
•
•
Rating Scales
Achievement Tests
Language
Memory
Phonological
See List
The BRIEF
• Behavior Rating Inventory of Executive
Function
• Parent, Teacher, and Student Forms
• Ages 5-18
• 86 items
• Has a global executive composite score
• Two Indexes: Behavioral Regulation and
Metacognition
Rating Scales: Children’s Psychological
Processes Scale (CPPS)
1.
2.
3.
4.
5.
6.
7.
Standardized teacher rating scale
Ages 5-0-0 to 12-11-30
121 items across 11 subscales
Internet, web-based test
Administration time of 15 minutes
Online scoring and report
Author: Milton Dehn; published by
Schoolhouse Educational Services, 2012
8. Measurement Consultant: Kevin McGrew
Uses of the CPPS
1. LD Evaluations
1.
2.
3.
4.
Identify psych processing deficits
Pattern of strengths and weaknesses
Planning further assessment
Planning interventions
2. Screening
1. Identifies need for intervention
2. Predicts academic skills development
3. Useful in planning comprehensive assessment
3. Measure progress during interventions
1. Through the use of change-sensitive W-scores
CPPS Items
• Grouped by subscale
• In developmental (ability) order from
lowest item to highest item
• Link
• Example of scoring in developmental
sequence Link
CPPS Standardization
1.
2.
3.
4.
5.
6.
7.
1,121 students rated by 278 teachers
128 communities in 30 states
All data collected online
Demographics match U.S. Census well
Scores were weighted
Included children with disabilities
Demographics details Link
W-Scores
1. Used in item analysis, development of
scoring system, and in reports
2. Mean of 500 at age 10; SD of roughly 20
3. Converts ordinal rankings to equal interv.
4. More precise measurement
1. Gradient smaller steps: 4-5 W = 1-2 T-score
2. More sensitive to changes, progress
5. Use when re-evaluating student
How The Online CPPS Works
1. A psychologist’s side and a teacher’s side
2. Psychologist fills in teacher information
and email sent
3. Teacher completes ratings
4. Psych generates report
5. See demonstration video
CPPS Report
1.
2.
3.
4.
Brief narrative, graph, and a table of scores
Change-sensitive W-scores
T-scores; percentiles; confidence intervals
Intra-individual strengths and weakness
discrepancy table
5. T-score to standard score converter
6. Can be re-run with different options (without
a charge) Example
Discrepancy Analysis
1. Use discrepancy table to determine
pattern of strengths and weaknesses
1.
2.
3.
4.
Predicted score based on mean of other 10
Regression toward the mean included
+/- 1.00 to 2.00 SD of SEE discrep options
Strengths and Weakness labeling is
opposite of discrepancy, e.g. “-” value = a
strength
5. Link
Conversion Table
1. Optional
2. Purpose: To see how consistent CPPS
scores are with achievement and
cognitive scores
3. T-score x 1.5 + 25 and then reverse
distance from mean
4. Example: T-score of 60 x 1.5 = 90 + 25 =
115
5. Then subtract 15 from 100 = 85 Link
Diagnostic Utility for LD
1. 37 LD subjects with broad demographics
2. Compared to matched controls, LD
subjects had significantly higher means
on all subscales Link
3. The CPPS has high classification
accuracy in regards to LD
1. Using CPPS GPA cutoff of 60 has 92%
classification accuracy across 74 subjects
Diagnosing LD with the CPPS
1. Use a T-score of 60 or above on the GPA
or a more conservative cutoff of 65
2. Individual subscales of 65 or higher
3. Look for pattern of strengths and
weaknesses (discrepancy table)
1. Weaknesses should also be above average
scores
2. Weaknesses should link to evidence-based
achievement relations Link
Using CPPS Results to Guide
Standardized Testing
1.
2.
3.
4.
The WJ III COG is a good match
Other scales, such as memory, BRIEF
Use selective testing table
Sample all processing areas but
especially weak and borderline areas
5. CPPS processes with scores above 60
should be sampled with at least 2
subtests
Using the CPPS to Measure
Progress
1.
2.
3.
4.
Use W-Scores
Have a mean of 500 at age 10
Standard deviations vary (about 20)
4-5 W-score points for every 1-2 T-score
points
5. A change of 20 points can be considered
statistically significant
Using the CPPS to Plan an Intervention
1. Select processes with scores above
60 (weaknesses)
2. Prioritize if too many
3. Look for strengths (scores below 40)
to counterbalance weakness
Correlations with Achievement
1. High correlations with WJ III Achievement
Test scores Link
1. The broader the achievement score, the
higher the correlations
2. Correlation of .66 between teacher’s
overall ranking of academic skills and
CPPS GPA
3. Parent education level and CPPS GPA is
.33
Factor and Cluster Analysis
1. A general factor; all subtests load on
1. General Processing Ability (GPA) may
reflect processing efficiency or automaticity
Link
2. Second factor is Attention, EF, sometimes WM:
Self-Regulatory Processes
3. Third factor is Fine Motor and Visual-Spatial:
Visual-Motor processes Link
4. Results fairly consistent across age groups
CPPS General Processing
Ability (GPA)
1. Based on average of all process scores
2. Emerges from factor analysis; similar to
concept of general intelligence
3. Processes function in an inter-related
fashion
4. Most processes contribute to any given
behavior, task
5. On CPPS defined as “the underlying
efficiency of automaticity of processing”
Case I Background
1. 5th grade, 12 year old male
2. History of struggling with Math & Wr Lang
3. Received tutoring and extra help at
school
4. Not placed for LD after eval. 2 years prior
5. Average IQ
6. Doesn’t meet ADHD diagnostic criteria
7. Seems immature
Case I CPPS Scores
Attention
62
Auditory Processing
51
Executive Functions
55
Fine Motor
63
Fluid Reasoning
64
Long-Term Recall
61
Oral Language
61
Phonological Processing 53
Processing Speed
58
Visual-Spatial Processing 48
Working Memory
63
General Processing Ability 60
Case I WJ Cog Scores
Comp-Knowledge
LT Retrieval
Vis-Spatial
Auditory Proc
Fluid Reasoning
Process Speed
ST Memory
Phonemic Aware
Working Memory
Broad Attention
Cognitive Fluency
97
*a close match with CPPS
86*
93*
97*
103
82
90
103*
87*
90
85
Case I WJ III ACH
Broad Reading
Broad Math
Math Calc Skills
Broad Wr Lang
Writ Exp
91
89
90
86
85
See CPPS Discrepancy Table
Case I Conclusions
1. From a processing perspective, could
qualify for LD
2. Does not because IQ-Ach discrepancy
not severe enough
3. Nevertheless, struggles because of
processing problems
4. Received working memory training
Conclusions About Processing
Assessment
• It can be done well but there are
challenges
• Professional judgment is necessary
• Selective, cross-battery testing necessary
• Test the processes associated with the
academic deficiency
• Use the consistency approach
• Explain the link