Download About the Achievement Gap Data Dashboard - Yellow

About the Achievement Gap Data Dashboard
Achievement Gap Dashboard, In Short
The achievement gap dashboard displays gaps in performance between a minority group (target group)
and a majority group (comparison group) at school, district, and state levels. Groups include race,
economic status, disability status and English proficiency. Graphs include Badger Exam and ACT
statewide assessments as well as attendance rates.
The Achievement Gap Dashboard Explained
The structure of the achievement gap graphs is different from that of the other graphs in WISEdash that
you may be familiar with. Rather than describing the performance of one student group, the achievement
gap graphs compare the performance of two groups.
Each bar shown in the graphs characterizes a school, district or state gap in performance between
groups. The larger the bar, the larger the gap between the two groups. The smaller the bar, the smaller
the gap between the two groups. If the bar is positive then the comparison group is performing better than
the target group. If the bar is negative then the target group is performing better than the comparison
group.
For example, one bar will represent the gap between students with disabilities and non-disabled students
in the school, another bar will represent that gap in the district, and a third bar will represent the gap
throughout the state. Provided there are enough students for a comparison there will be a school, district
and state bar for each race, disability status, economic status and English proficiency.
The bars representing the gap are shown in effect size units. See the “Effect Size” section below for an
explanation of effect size.
Only students with a Badger Exam or ACT test result are included in the Badger and ACT graphs.
Students who were not tested or who were administered the DLM alternate assessment are not included.
All students are included in the attendance graph.
FAY status is not considered for the achievement gap dashboard. Both students who are and are not FAY
are included in the results.
Achievement Gap Drill-Down Detail
Unlike other dashboards, when a user clicks one of the bars in the graph the achievement gap dashboard
does not display a student listing nor does it drill down to a student profile dashboard. Rather, it displays
the school, district and state-level averages and counts that are used to calculate gaps. Individual student
results are not available from this dashboard.
Dashboard Security
The achievement gap dashboard displays results in a different manner than
other WISEdash dashboards. The three major areas of difference are:
1. Gaps-based instead of Results-based – unlike other dashboards, the purpose of this
dashboard is not to display the results for a given subgroup but rather to display the gap in
results between two subgroups.
2. School-level detail instead of Student-level Detail – in most dashboards, users can drill down
to student-level detail whereas -because the dashboard shows school result gaps instead of
individual student results- the achievement gap dashboard allows a drilldown to school-level
detail instead.
3. Inclusion of State and District-level Comparison – Although state and district -level results are
displayed in WISEdash public, such aggregated results are typically not displayed
in WISEdash for Districts.
Because of these differences, a description of how existing WISEdash user security mechanisms are
applied to this dashboard is warranted and can be found below:

Summary-level Security - Users who don’t have drilldown access to other dashboards will not
be able to view school-level detail despite the fact that such detail is not student-level detail.

District-level Security – If a user has access to view district results – regardless of whether the
user is restricted at the school-level- they can view district and state-level results in the metric
summary of the dashboard. If a user does not have access to a district, they cannot view
district-level results but such restricted district results are included in the aggregate of statelevel results.

School-level Security – Users who are limited by security to select schools in a district –who
can otherwise view detailed information – will only be able to view detailed information for
schools for which they have access. Further, such users would not be able to see summary
school-level information for the schools for which they do not have access; however, schoollevel results are included in aggregated district and state-level results regardless of security.

Economic Status Security – For those users who are restricted from disaggregating or filtering
by or -otherwise seeing- economic status, will not see economic status rows in the
Achievement Gap detail. Further, gap comparisons for economic status (including school,
district and state-level), will also not display in the summary view for such users regardless of
whether there are gap data to present.
What is an Effect Size?
Effect size is a standardized measure of the difference between groups on a given outcome. Effect size is
useful because, as a standardized measure, it allows users to evaluate the magnitude of the difference in
the context of other differences in the school or district as well as in the context of published research on
relevant interventions.
More specifically, the effect size shown in the achievement gap dashboard is a standardized mean
difference. This means that the effect size is measured in standard deviations. For example, an effect size
of 1.0 means that the gap between groups is one standard deviation. An effect size of 0.5 means that the
gap is half of one standard deviation.
The formula used to calculate achievement gaps dashboard effect sizes is Glass’ delta. To calculate
Glass’ delta, subtract the mean of the comparison group from the mean of the target group. Then divide
that difference by the state-level standard deviation of the comparison group.
where
For example, to calculate the effect size for a school-level gap in mathematics mean scale score between
Hispanic students and white students, first subtract the school-level mean scale score for Hispanic
students from the school-level mean scale score for white students. Then divide this difference by the
state-level standard deviation for white students.
How to Interpret an Effect Size?
Direction of the Effect Size
Effect sizes can be positive or negative. To interpret the direction of the effect size you will have to know
how the effect size was calculated. For example, if the effect size is
then a positive effect size would indicate that the mean value associated with the white students is higher
than the mean value associated with the Hispanic students. A negative value would indicate the opposite,
that the mean value associated with the Hispanic students is higher than the mean value associated with
the white students.
Magnitude of the Effect Size
An effect size of zero means that there is no difference between the two groups. A small effect size
means that the difference between the two groups is small. A large effect size means that the difference
between the two groups is large. However, there are a number of ways to interpret the magnitude of an
effect size.
One way to interpret the magnitude of an effect size is to compare that effect size to benchmarks from
scientific literature. For example, in Statistical Power Analysis for the Behavioral Sciences (1988), Cohen
describes effect sizes of 0.2 and smaller as small, effect sizes of 0.2 to 0.5 as medium, and effect sizes
larger than 0.5 as large. Another benchmark has been set by the federal What Works Clearinghouse
(WWC). When evaluating intervention programs, WWC considers effect sizes of 0.25 and greater to be
“substantively important.” These benchmarks may not be appropriate in every context but are a good
starting point for interpretation.
Another approach is to compare the effect size to observations published in scientific literature. For
example, John Hattie’s Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement
provides the effect sizes associated with many education interventions.
One more approach is to compare school-level effect sizes to district-level or state-level effect sizes.
Similarly, effect sizes can be compared across groups or from one outcome to another, such as from the
Badger Exam to the ACT.
How Does Effect Size Differ From Statistical Significance?
Effect size helps users to evaluate the magnitude of a difference between groups. Statistical significance
determines whether the difference is likely to be due to chance. For example, the What Works
Clearinghouse (WWC) uses statistical significance to determine if the difference in outcomes between
treatment and control groups in a study is likely to be due to chance, and uses effect size to determine if
the difference is large enough to have a substantive impact on the treatment group.
It is possible for a difference between groups to be significant without being substantively important.
Statistical significance is dependent on sample size. If a comparison is poorly designed statistical
significance can simply mean that a large sample was used. Effect size is independent of sample size.
Definitions
Comparison Group and Target Group: The performance of two groups must be compared in order to
calculate a gap. Those two groups are the target group and the comparison group. The target group is the
group that the user is interested in studying (students with disabilities, economically disadvantaged
students, English language learners and students of the race/ethnicity groups not selected with the
Comparison Race/Ethnicity filter). The comparison group is the group that the target group is compared to
(students without disabilities, non-economically disadvantaged students, English proficient students, and
students of the race/ethnicity group selected with the Comparison Race/Ethnicity filter).
Effect Size: Effect size is a standardized measure of the difference between groups on a given outcome.
Cautions
Small Group Size: Be careful not to draw erroneous conclusions based on small numbers of students. By
default, gaps are not calculated when either the target or comparison group has less than 20 students. To
see gaps for all comparisons regardless of the number of students set the Small Groups filter to “Show.”
Gaps cannot be shown when there are no students in either the target or comparison group.