Download The Effects of Class Failure and Course Repetition on

The Effects of Class Failure and Course Repetition on
University Outcomes
Enzo Brox∗ and Michael Dörsam†
September 2016
Abstract
Higher educational institutions have implemented mechanisms to test whether students have the required skills for a program. In this paper we use plausible exogenous
variation in class failure to assess the causal impact of class failure at the beginning of a
program on subsequent student performance. We apply a regression discontinuity design
to administrative data from a German university. Based on a sample of 2000 students we
find that class failure increases the probability to drop out of a program substantially but
does not affect the grades in later courses.
JEL-Code: I21, I23, J24
Keywords: higher education, course failure, academic achievement, regression discontinuity design
∗
University of Konstanz, Department of Economics, Box 146, 78457 Konstanz, Germany.
[email protected]
†
University of Konstanz, Department of Economics, Box D133, 78457 Konstanz, Germany.
[email protected]
Email:
Email:
Extended Abstract
Dropping out of university without a certificate is a widely discussed topic in western
societies. According to the DZHW (Deutsches Zentrum für Wirtschaftswissenschaften)
about one third of all students at German universities do not finish their program,
but drop out earlier (DZHW, 2014). The issue is at the forefront of many political
discussions as high dropout rates especially in later years of a program create a lot
of losers without having winners.1 First, students who drop out of a program either
have to decide for another educational program or may start working immediately.
Both decisions are associated with negative consequences for the labor market. If they
decide for a new kind of apprenticeship, they will need longer to become qualified candidates for the labor market. If they decide to start working immediately, the share
of high qualified employees diminishes. Second, public higher educational institutions
in OECD countries spend about 9000 US Dollar per student annually on educational
services (OECD, 2014). As it is one of the goals of higher educational institutions to
train qualified workers for the labor market, a late dropout coincides with a waste of
resources as the money could have been spent for other students.
To avoid late dropout, universities are forced to deal with two issues. They have to
select suitable candidates who meet the program requirements,2 and, they have to
help low performing students who may be in general suitable for a program to catch
up with their better performing colleagues. To manage both, universities in BadenWuertemberg have installed a specific institutional mechanism, so called ’Orientation
exams’, in the first year of a program. Orientation exams are exams students have to
pass in the first year to continue a program. If students fail the exam at their first
attempt, they have to pass the second attempt. Otherwise they lose the eligibility to
continue the same program at every German university. Orientation exams combine
a selection and an incentive mechanism. On the one hand, Orientation exams are
thought to test students eligibility for the requirements of the program and to select
suitable candidates. On the other hand, a failure at the first attempt is thought to
signal students that they have to spend more time on the covered material, which may
help them to catch up with their peers.
1
In 2015, the state Nordrhein-Westfalen started to pay universities 4000 Euro for every graduate, to create
an incentive for lower dropout rates, especially in later years of a program. Other states plan to follow this
approach recently.
2
The perfect strategy would be to select exactly those students who are able to meet the requirements
before the program start. Often this is not possible as the high-school grade point average (HSGPA) is the
only information available and universities are forced to accept a certain amount of students to receive funding
from the state. Therefore, in the best case students either dropout early or they pass the program. In this
way the mechanism would be efficient.
1
Both the selection mechanism and the incentive mechanism are rarely studied at the
higher educational level. Grade retention at high-school, which is the equivalent to
course repetition at the higher educational level, is a frequently discussed topic among
education researchers and politicians. Many papers have been conducted to evaluate
the effectiveness of retention at the high-school level with mixed results. Manacorda
(2012) finds that grade retention increases the probability to drop out. Schwerdt et al.
(2015) find a positive effect on subsequent achievement, while Jacob and Lefgren (2009)
conclude that the impact of grade retention depends on the grade in which a student
gets retained. At the university level empirical evidence is scarce. Furthermore, retention policies at the university level may have a different effect compared to the
high-school level. On the one hand, the additional effort of failing a course is lower
as university students only have to repeat those courses in which they failed. On the
other hand, the pressure after having failed a course is higher for university students
as they only have a limited number of attempts to pass a course before they are forced
to leave the program.
The theoretical literature offers explanations for both positive and negative effects of
course repetition. A positive effect on university student performance may arise as
course repetition is thought to increase the incentive to put effort into educational performance in order to avoid bad grades and to be separated from peers (Jacob, 2005;
Jacob and Lefgren, 2004). While the peer effect may not be that strong at the university level, the avoidance effect should be significant as university grades determine
subsequent labor market success (Arcidiacono, 2004). Furthermore, positive effects
may arise because of learning gains and a better match between program requirements
and students’ knowledge (Mc Coy and Reynolds, 1999; Jimerson et al., 2002). Negative effects may occur because of a loss of self-confidence and motivation (Alexander
and Kabbani, 2003) which are crucial in tertiary education, since university studies are
voluntary and students have the outside option to seek employment immediately.
Convincing quasi-experimental evidence for the effect of course repetition on student
performance at the tertiary level is scarce. So far, the study by Tafreschi and Thiemann
(2015) is the only one that uses data from higher educational institutions and accounts
for the potential endogeneity of failure rates. The authors investigate a very specific
institutional setting implemented at the University of St. Gallen where students who
do not meet a certain performance requirement after the first year are forced to repeat
all first year courses before they are allowed to move to the second year. Tafreschi and
Thiemann (2015) find that having to repeat the full first year curriculum increases the
2
dropout probability by about 10 percentage points and increases later grades by 0.5
standard deviations.
Our paper contributes to the empirical literature discussing the impact of grade retention policies. While Tafreschi and Thiemann (2015) investigate a very specific institutional setting at the University of St Gallen3 , we consider a different set up that
is common practice in most bachelor’s degree programs. To pass a course bachelor
students usually have to write an exam. If they do not achieve a certain amount of
points in the exam they fail, and have to write a retake exam to get the European
Credit Transfer System (ECTS) points for the course. Failing an Orientation exam
twice makes them loose the eligibility to continue the subject of study at every German
university. Therefore, students are highly incentivized to study hard before taking the
retake exam. The extra amount of time and effort spent for a subject is a reasonable
treatment for our study and different from the treatment in Tafreschi and Thiemann
(2015). To determine the causal impact of course repetition on subsequent university
outcomes, we apply a regression discontinuity design (RDD) to administrative data
from the University of Konstanz. The discontinuous relationship between test scores
and promotion in the Bachelor of Science Program in Economics allows us to estimate
the impact of class failure in tertiary education on dropout rates and student performance in later semesters.
We find large and persistent effects of course failure on dropout rates. Students who
have to repeat one of the Orientation exams in the first semester are more likely to
drop out of the program after the first year. This result is robust across specifications.
We also find positive effects on later performance in related subjects. These effects,
however, are not robust throughout all specifications.
3
First year students who do not achieve a certain performance requirement after the first year are forced
to repeat all first-year courses before they are allowed to move into the second year (Tafreschi and Thiemann,
2015, p.4).
3
References
Alexander, K.; Entwisle, D. and N. Kabbani (2003). Grade retention, social promotion,
and ’third way’ alternatives. Early Childhood Prograam for a New Century, 197–238.
Angrist, J. and J. Pischke (2009). Mostly Harmless Econometrics: An Empirical Companion. Princeton University Press.
Arcidiacono, P. (2004). Ability sorting and the returns to college major. Journal of
Econometrics 121 (1-2), 343–375.
Jacob, B. and L. Lefgren (2009). The effect of grade retention on high school completion.
American Economics Journal: Applied Economics 1, 33–58.
Jacob, B. A. (2005, June). Accountability, incentives and behavior: the impact of highstakes testing in the chicago public schools. Journal of Public Economics 89 (5-6),
761–796.
Jacob, B. A. and L. Lefgren (2004, February).
Remedial education and student
achievement: A Regression-Discontinuity analysis. Review of Economics and Statistics 86 (1), 226–244.
Jimerson, S. R., G. E. Anderson, and A. D. Whipple (2002). Winning the battle and
losing the war: Examining the relation betwen retention and dropping out of high
school. Psychology in the School 39, 441–457.
Manacorda, M. (2012). The cost of grade retention. The Review of Economics and
Statistics 94 (2), 596–606.
Mc Coy, A. and A. Reynolds (1999). Grade retention and school performance: An
extended investigation. journal of school psychology 37, 273-298. Journal of School
Psychology 37, 273–298.
OECD (2014). Education at a Glance 2014. OECD Indicators, Paris. OECD.
Schwerdt, G., M. R. West, and M. A. Winters (2015, August). The effects of test-based
retention on student outcomes over time: Regression discontinuity evidence from
florida. Working Paper 21509, National Bureau of Economic Research.
Tafreschi, D. and P. Thiemann (2015, February, 17). Doing it twice, getting it right?
the effect of grade retention and course repetition in higher education. Working
Paper , 1–37.
4