Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
About the Achievement Gap Data Dashboard Achievement Gap Dashboard, In Short The achievement gap dashboard displays gaps in performance between a minority group (target group) and a majority group (comparison group) at school, district, and state levels. Groups include race, economic status, disability status and English proficiency. Graphs include Badger Exam and ACT statewide assessments as well as attendance rates. The Achievement Gap Dashboard Explained The structure of the achievement gap graphs is different from that of the other graphs in WISEdash that you may be familiar with. Rather than describing the performance of one student group, the achievement gap graphs compare the performance of two groups. Each bar shown in the graphs characterizes a school, district or state gap in performance between groups. The larger the bar, the larger the gap between the two groups. The smaller the bar, the smaller the gap between the two groups. If the bar is positive then the comparison group is performing better than the target group. If the bar is negative then the target group is performing better than the comparison group. For example, one bar will represent the gap between students with disabilities and non-disabled students in the school, another bar will represent that gap in the district, and a third bar will represent the gap throughout the state. Provided there are enough students for a comparison there will be a school, district and state bar for each race, disability status, economic status and English proficiency. The bars representing the gap are shown in effect size units. See the “Effect Size” section below for an explanation of effect size. Only students with a Badger Exam or ACT test result are included in the Badger and ACT graphs. Students who were not tested or who were administered the DLM alternate assessment are not included. All students are included in the attendance graph. FAY status is not considered for the achievement gap dashboard. Both students who are and are not FAY are included in the results. Achievement Gap Drill-Down Detail Unlike other dashboards, when a user clicks one of the bars in the graph the achievement gap dashboard does not display a student listing nor does it drill down to a student profile dashboard. Rather, it displays the school, district and state-level averages and counts that are used to calculate gaps. Individual student results are not available from this dashboard. Dashboard Security The achievement gap dashboard displays results in a different manner than other WISEdash dashboards. The three major areas of difference are: 1. Gaps-based instead of Results-based – unlike other dashboards, the purpose of this dashboard is not to display the results for a given subgroup but rather to display the gap in results between two subgroups. 2. School-level detail instead of Student-level Detail – in most dashboards, users can drill down to student-level detail whereas -because the dashboard shows school result gaps instead of individual student results- the achievement gap dashboard allows a drilldown to school-level detail instead. 3. Inclusion of State and District-level Comparison – Although state and district -level results are displayed in WISEdash public, such aggregated results are typically not displayed in WISEdash for Districts. Because of these differences, a description of how existing WISEdash user security mechanisms are applied to this dashboard is warranted and can be found below: Summary-level Security - Users who don’t have drilldown access to other dashboards will not be able to view school-level detail despite the fact that such detail is not student-level detail. District-level Security – If a user has access to view district results – regardless of whether the user is restricted at the school-level- they can view district and state-level results in the metric summary of the dashboard. If a user does not have access to a district, they cannot view district-level results but such restricted district results are included in the aggregate of statelevel results. School-level Security – Users who are limited by security to select schools in a district –who can otherwise view detailed information – will only be able to view detailed information for schools for which they have access. Further, such users would not be able to see summary school-level information for the schools for which they do not have access; however, schoollevel results are included in aggregated district and state-level results regardless of security. Economic Status Security – For those users who are restricted from disaggregating or filtering by or -otherwise seeing- economic status, will not see economic status rows in the Achievement Gap detail. Further, gap comparisons for economic status (including school, district and state-level), will also not display in the summary view for such users regardless of whether there are gap data to present. What is an Effect Size? Effect size is a standardized measure of the difference between groups on a given outcome. Effect size is useful because, as a standardized measure, it allows users to evaluate the magnitude of the difference in the context of other differences in the school or district as well as in the context of published research on relevant interventions. More specifically, the effect size shown in the achievement gap dashboard is a standardized mean difference. This means that the effect size is measured in standard deviations. For example, an effect size of 1.0 means that the gap between groups is one standard deviation. An effect size of 0.5 means that the gap is half of one standard deviation. The formula used to calculate achievement gaps dashboard effect sizes is Glass’ delta. To calculate Glass’ delta, subtract the mean of the comparison group from the mean of the target group. Then divide that difference by the state-level standard deviation of the comparison group. where For example, to calculate the effect size for a school-level gap in mathematics mean scale score between Hispanic students and white students, first subtract the school-level mean scale score for Hispanic students from the school-level mean scale score for white students. Then divide this difference by the state-level standard deviation for white students. How to Interpret an Effect Size? Direction of the Effect Size Effect sizes can be positive or negative. To interpret the direction of the effect size you will have to know how the effect size was calculated. For example, if the effect size is then a positive effect size would indicate that the mean value associated with the white students is higher than the mean value associated with the Hispanic students. A negative value would indicate the opposite, that the mean value associated with the Hispanic students is higher than the mean value associated with the white students. Magnitude of the Effect Size An effect size of zero means that there is no difference between the two groups. A small effect size means that the difference between the two groups is small. A large effect size means that the difference between the two groups is large. However, there are a number of ways to interpret the magnitude of an effect size. One way to interpret the magnitude of an effect size is to compare that effect size to benchmarks from scientific literature. For example, in Statistical Power Analysis for the Behavioral Sciences (1988), Cohen describes effect sizes of 0.2 and smaller as small, effect sizes of 0.2 to 0.5 as medium, and effect sizes larger than 0.5 as large. Another benchmark has been set by the federal What Works Clearinghouse (WWC). When evaluating intervention programs, WWC considers effect sizes of 0.25 and greater to be “substantively important.” These benchmarks may not be appropriate in every context but are a good starting point for interpretation. Another approach is to compare the effect size to observations published in scientific literature. For example, John Hattie’s Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement provides the effect sizes associated with many education interventions. One more approach is to compare school-level effect sizes to district-level or state-level effect sizes. Similarly, effect sizes can be compared across groups or from one outcome to another, such as from the Badger Exam to the ACT. How Does Effect Size Differ From Statistical Significance? Effect size helps users to evaluate the magnitude of a difference between groups. Statistical significance determines whether the difference is likely to be due to chance. For example, the What Works Clearinghouse (WWC) uses statistical significance to determine if the difference in outcomes between treatment and control groups in a study is likely to be due to chance, and uses effect size to determine if the difference is large enough to have a substantive impact on the treatment group. It is possible for a difference between groups to be significant without being substantively important. Statistical significance is dependent on sample size. If a comparison is poorly designed statistical significance can simply mean that a large sample was used. Effect size is independent of sample size. Definitions Comparison Group and Target Group: The performance of two groups must be compared in order to calculate a gap. Those two groups are the target group and the comparison group. The target group is the group that the user is interested in studying (students with disabilities, economically disadvantaged students, English language learners and students of the race/ethnicity groups not selected with the Comparison Race/Ethnicity filter). The comparison group is the group that the target group is compared to (students without disabilities, non-economically disadvantaged students, English proficient students, and students of the race/ethnicity group selected with the Comparison Race/Ethnicity filter). Effect Size: Effect size is a standardized measure of the difference between groups on a given outcome. Cautions Small Group Size: Be careful not to draw erroneous conclusions based on small numbers of students. By default, gaps are not calculated when either the target or comparison group has less than 20 students. To see gaps for all comparisons regardless of the number of students set the Small Groups filter to “Show.” Gaps cannot be shown when there are no students in either the target or comparison group.