Download Developing Data-Driven Predictive Models of Student

1
PREDICTIVE ANALYTICS
FOR
STUDENT SUCCESS:
Developing Data-Driven Predictive Models of Student
Success
Final Report
University of Maryland University College
January 6, 2015
A Research Project Funded by the Kresge Foundation
2
Table of Contents
EXECUTIVE SUMMARY ............................................................................................................ 4
SECTION 1: INTRODUCTION ................................................................................................. 10
Grant Partnership................................................................................................................................... 11
Objectives and Milestones ...................................................................................................................... 11
SECTION 2: LITERATURE REVIEW ..................................................................................... 13
Theoretical Models of Community College Transfer Student Performance ..................................... 13
Educational Data Mining ....................................................................................................................... 15
Predicting Transfer Students’ First-Term GPA .................................................................................. 16
Predicting Transfer Student Re-Enrollment ........................................................................................ 18
Predicting Re-Enrollment for Non-Traditional Students .................................................................... 19
Literature Guiding Interventions .......................................................................................................... 20
Community College Transfer Students’ Transitioning ....................................................................... 22
Literature to Support Specific Interventions ....................................................................................... 22
Checklist ............................................................................................................................................. 22
Community College Mentor ............................................................................................................... 23
SECTION 3: RESEARCH SCOPE AND DESIGN ................................................................... 25
Research Questions ................................................................................................................................. 25
Student Population.................................................................................................................................. 26
SECTION 4: DATA SOURCES .................................................................................................. 27
SECTION 5: SURVIVAL ANALYSIS: REGISTRATION AND WITHDRAWAL IN THE
ONLINE CLASSROOM .............................................................................................................. 29
SECTION 6: PROFILES OF STUDENTS USING DATA MINING ...................................... 31
Profiles of Student Success ..................................................................................................................... 31
Further Findings from Data Mining ..................................................................................................... 33
SECTION 7: PREDICTIVE MODELING OF STUDENT SUCCESS ................................... 34
Initial Predictive Modeling ..................................................................................................................... 34
Predicting Successful GPA ................................................................................................................. 34
Predicting Re-enrollment .................................................................................................................... 35
Updated Predictive Modeling ................................................................................................................ 36
Population ........................................................................................................................................... 38
Predicting Earning a Successful First-term GPA ................................................................................ 39
Predicting Re-Enrollment ................................................................................................................... 40
Predicting Retention............................................................................................................................ 42
Predicting Graduation ......................................................................................................................... 44
Summary of Results from Predictive Modeling.................................................................................. 45
SECTION 8: GRADUATION RATES ....................................................................................... 48
SECTION 9: EXAMINING LEARNER BEHAVIOR IN THE ONLINE CLASSROOM ... 49
Online Classroom Behaviors and Class Performance ......................................................................... 51
3
Student Level Online Classroom Behaviors and Course Performance ............................................. 53
Engagement Profiles and Course Performance ................................................................................... 56
Modeling Retention................................................................................................................................. 57
SECTION 10: STUDENT MOTIVATION AND SELF-REGULATION ............................... 59
Population ........................................................................................................................................... 59
Methodology ....................................................................................................................................... 59
Results ................................................................................................................................................. 60
Key Findings ....................................................................................................................................... 63
SECTION 11: INTERVENTION IMPLEMENTATION AND EVALUATION ................... 64
Checklist .................................................................................................................................................. 65
Participants.......................................................................................................................................... 65
Results ................................................................................................................................................. 65
College Success Mentoring ..................................................................................................................... 66
Participants.......................................................................................................................................... 66
Results ................................................................................................................................................. 66
Jumpstart Summer ................................................................................................................................. 68
Results ................................................................................................................................................. 68
Accounting 220 and Accounting 221 ..................................................................................................... 69
Participants.......................................................................................................................................... 69
Results ................................................................................................................................................. 69
Key Findings ....................................................................................................................................... 69
SECTION 12: DISSEMINATION .............................................................................................. 70
Presentations at Conferences ................................................................................................................. 70
Publications ............................................................................................................................................. 70
Learner Analytics Summit ..................................................................................................................... 71
Success Calculator .................................................................................................................................. 73
SECTION 13: FINANCIAL SUPPORT ..................................................................................... 74
SECTION 14: CONCLUSIONS .................................................................................................. 75
Future Directions .................................................................................................................................... 76
REFERENCES .............................................................................................................................. 77
4
EXECUTIVE SUMMARY
The purpose of the Predictive Analytics for Student Success (PASS) project was to: (a) aggregate
data across multiple institutions to track the academic progress and completion of community
college transfer students, (b) identify factors associated with success, and (c) implement
interventions that promote student success. In completing the PASS project research and
interventions, University of Maryland University College (UMUC) partnered with two
community colleges, Montgomery College (MC) and Prince George’s Community College
(PGCC). This work was funded by a grant from the Kresge Foundation -- Developing DataDriven Predictive Models of Student Success.
The purpose of the grant was:
 To build an integrated database tracking students across institutions from community
college to UMUC.
 To use predictive statistical models and data mining techniques to track and model
students’ progress across institutions.
 To identify factors predictive of students’ success at UMUC
 To inform the development of interventions aimed to improve outcomes for undergraduate
students transferring from community colleges to UMUC or to other four-year institutions.
This report will summarize the data development, research, intervention implementation and
evaluation, dissemination, and application creation completed through the PASS project.
Phase 1
During the first 24 months (Phase 1) of the grant, UMUC and the partner institutions developed
and signed a Memorandum of Understanding (MOU) to ensure data security and establish
parameters for data use. The MOU allowed the PASS project team to conduct research using
individual student data while protecting student information and confidentiality. Once the MOU
was in place, researchers identified the population of interest, conducted an initial literature
review to identify variables of interest, and began data collection and exploratory analyses.
The research team identified over 250,000 students enrolled at UMUC between 2005 and 2012.
Of those, over 30,000 students transferred from MC and PGCC. Student demographics,
academic performance at the three institutions, behavior in the online classroom at UMUC, and
advising data were combined into an integrated, multi-institutional database: the Kresge Data
Mart (KDM).
The literature review covered student performance in online courses, successful course
completion, factors associated with re-enrollment and retention, and the use of data mining
techniques in higher education. Existing research showed that factors such as the number of
schools attended, the number of credits transferred, and community college GPA influenced
student success. Key measures of success included successful course completion and retention.
5
In Phase 1, initial data mining was conducted to identify variables that were associated with
success. Specific courses were identified as having predictive value in relation to success at
UMUC. Regression analyses determined that student online classroom activities prior to the
start of a class (i.e., entering the online classroom prior to the first day) and during the early
weeks of the course were predictive of successful course completion.
Phase 2
Phase 2 of the PASS project was completed in months 25 to 36 of the grant. The initial plan for
Phase 2 was to:





Secure external evaluators
Further develop collaboration with the community colleges
Identify the scope of the project
Clarify the research plan and conduct associated analyses
Begin initial dissemination of research findings
UMUC began meeting regularly with the community colleges to develop the Phase 2 research
plan and evaluate research findings and grant progress. Two external evaluators were selected to
conduct an independent evaluation of the research project. These collaborations proved to be
highly beneficial in developing the research program and designing interventions. As a result of
the collaborations, new data were identified for collection, and a full scope of the research was
outlined in the form of a research plan.
A research plan was developed to model students’ progress and performance from the
community college to graduation from a four-year institution. The research plan created a model
addressing the relation between students’ prior academic work and performance at UMUC to
include graduation. The full path model of students’ academic trajectory from community
college to UMUC is below.
Community
College
Data
UMUC First
Term GPA
Reenrollment
Retention
Graduation
The plan identified the following research goals for Phase 2:
1. To develop profiles of transfer students at UMUC
2. To identify factors from students’ community college academic backgrounds that predict
success at UMUC
3. To develop predictive models of student success based on demographic information,
community college course taking behaviors, and first-term factors.
4. To develop interventions designed to improve the success of students transferring from
community colleges to UMUC.
6
Phase 2 considered two primary outcomes of interest in predictive models: 1) earning a first-term
GPA of 2.0 or above at UMUC, termed successful first-term GPA, and 2) students’ re-enrollment
at UMUC within 12 months following their first academic term, termed retention.
Key findings from Phase 2 include:






Across studies, age and marital status were associated with success at UMUC. Older,
married students were found to be more likely to succeed.
Four profiles of student success at UMUC were identified based on students’ GPAs and
retention rates. The profiles differed in terms of community college course taking
preferences and course load and in the change in GPA when transferring to UMUC.
These results suggest that the degree of student preparedness, particularly in specific
target areas (e.g., accounting, economics), is predictive of success at UMUC.
Course efficiency, or the ratio of credits earned to credits attempted, in the community
college was determined to be a predictor of success at UMUC. The higher the course
efficiency, the more likely a student was to succeed.
A new factor, delta GPA, was introduced in these analyses, corresponding to the
difference between students’ GPA at the community college and at UMUC. While most
students experienced a decreased GPA when transferring to UMUC, the magnitude of
this decrease was predictive of students’ continued enrollment at UMUC beyond the first
term.
Students who took math or honors courses in community college were more likely to
succeed at UMUC, suggesting that rigor of community college courses may prepare
students to succeed at a four-year university.
Student behaviors in the online classroom indicated high variability in the extent to which
they engage in course content and course-related activities. A substantial percentage of
students accessed course content and course materials to a limited extent, thus impacting
successful course completion.
Phase 3
Phase 3, the final year of the Kresge Grant, focused on four goals:
1.
2.
3.
4.
Data enhancement
Extended research
Implementation and evaluation of the interventions
Continued dissemination of research findings and intervention results
As a result of continued collaboration with the community colleges, the KDM was expanded to
include additional variables from the community colleges as well as updated data from UMUC to
allow for expanded analyses of retention and graduation.
With the inclusion of new data, Phase 3 analyses focused on re-enrollment, retention, and
graduation from UMUC.
7
Key Findings from Phase 3 include:





Demographic Factors. Gender and marital status were associated with both performance
(earning a successful first-term GPA) and persistence (re-enrollment). These
characteristics may indicate students’ maturity and commitment to pursuing academic
goals. Interestingly, while African American status was negatively associated with
earning a successful first-term GPA, it was positively associated with persistence metrics.
This suggests that while not always successful in their first semester, African American
students are nonetheless committed to their educational goals.
Math at the Community College. Across models examining both persistence and
performance, variables associated with taking math at the community college were found
to be significant predictors. Within our sample, taking math at the community college
reflects academic abilities and may also reflect students’ commitment to meeting the
requirements necessary for transfer and graduation.
Community College Success and Completion. In models predicting first-term GPA, reenrollment, and graduation, students’ community college GPA was a significant
predictor. This suggests that, overall, performance at the community college matters for
success and persistence at a four-year institution.
First-Term Performance. As in findings from Phase 2, students’ performance in their first
semester at UMUC remains crucial in predicting re-enrollment, retention, and graduation.
In fact, across models, it was the strongest individual predictor of performance. Firstterm GPA may be an indicator of factors contributing to students’ success, beyond
academic abilities. Specifically, students who are better at acclimating to an online
university and the demands associated with a four year institution may have a higher
first-term GPA and may be more able to persist.
Online Classroom Engagement. A particularly rich finding from Phase 3 analyses is the
association between student online classroom engagement as measured in the learning
management system (LMS) and course performance. The general pattern was that
students earning higher grades in a particular course were also significantly more
engaged in the online classroom. Further, online course engagement, in combination
with students’ community college GPA, was predictive of overall course performance;
such a model linked students’ community college backgrounds with four-year
institutional experience.
Phase 3 also included the examination of the efficacy of four interventions aimed at promoting
community college transfer student success at UMUC. In addition, two interventions at the
community college were used to better prepare students for transfer.
Interventions undertaken at UMUC were:

Checklist. New student orientation checklist administered to community college transfer
students to aid them in navigating online resources at UMUC. Although no significant
differences were found, students responding to an evaluation survey found the checklist to
be a useful tool.
8



Mentoring. Eight week structured mentoring program, where new UMUC community
college transfer students were paired with a peer mentor -- a successful student at UMUC
who had transferred from the same community college. Each week, mentors emailed
mentees with study tips and information to support adjustment to UMUC. Although no
statistically significant improvements in semester performance were found for mentees,
unexpectedly, students serving as mentors had a significantly higher cumulative GPA and
a significantly higher rate of successful course completion when compared to the control
group of students who were invited to be mentors and elected not to participate. This
phenomenon may be due to the bias inherent in the self-selection process.
Jumpstart Summer. A program that paired mentoring with Jumpstart, a four-week
onboarding course, designed to support students’ goal setting and academic planning. Four
experimental conditions were examined: (a) a control group, (b) students only completing
the Jumpstart course, (c) students only participating in the mentoring program, and (d) a
Jumpstart Summer group, receiving both mentoring and enrolled in the Jumpstart course.
No significant differences in performance were found; however, students successfully
completing the Jumpstart course had a higher rate of successful course completion and reenrolled at a higher rate.
Accounting 220/221: The online tutoring intervention was developed by faculty for
students taking Accounting 220 and Accounting 221 -- courses with a disproportionally
high failure rate both at UMUC and nationally. Students who participated in the online
tutoring had a significantly higher GPA at the end of the semester and a significantly
higher rate of successful course completion, when compared to students not participating
in online tutoring.
Interventions developed at the community colleges were:


Diverse Male Student Initiative (DMS-I). DMS-I is a two-year program at Prince
George’s Community College that provides minority male students with role models and
academic and career mentoring. DMS-I held a two-day summer institute at PGCC that
featured keynote speakers and awarded book and tuition vouchers for early registration to
participants with the aim of improving academic planning and persistence. PGCC and
UMUC will track and evaluate the success and persistence of students who participated
in the program and who transfer to UMUC.
Women’s Mentoring, Boys to Men, TriO: Women’s Mentoring, Boys to Men, and TRiO
are comprehensive mentoring programs, developed at Montgomery College, that provide
minority students with comprehensive academic and social support throughout their
transfer pathways from high school to MC, and ultimately to a four-year institution. MC
and UMUC will identify students participating in these programs who transfer to UMUC
and will track them to evaluate their performance. UMUC will provide similar mentoring
and support if these students transfer to UMUC.
Findings from research and interventions were disseminated through ten conference
presentations and manuscripts. In addition, a website, http://www.umuc.edu/PASS, was created
to share project goals and results with a broad audience of stakeholders.
9
This report was produced by the UMUC Office of Institutional Research and Accountability and
contains 12 sections:
Section 1: Introduction
Section 2: Literature Review
Section 3: Research Scope and Design
Section 4: Data Sources
Section 5: Survival Analysis: Registration and Withdrawal for Online Courses
Section 6: Mining of Community College Data
Section 7: Predicting Student Success
Section 8: Graduation Rates
Section 9: Data Mining of Online Learner Behavior
Section 10: Students’ Motivation and Self-Regulation
Section 11: Implementation and Evaluation of Interventions
Section 12: Dissemination
Section 13: Financial Statement
Section 14: Conclusions
10
SECTION 1: INTRODUCTION
The purpose of this report is to present the results of research conducted by the University of
Maryland University College (UMUC), in partnership with two community colleges,
Montgomery College (MC) and Prince George’s Community College (PGCC) as part of the
PASS project. The scope of work was undertaken as part of a grant from the Kresge foundation
and includes: (a) data development, (b) research using data mining and predictive modeling to
examine community college transfer student success, and (c) intervention development,
implementation, and evaluation to provide academic, social, and institutional support to
community college students both prior to and after transfer. The project was broken out into
three phases. Each phase was built upon results from the previous phase, resulting in continued
research development and comprehensive analyses. A research plan identifying the scope of the
project began in Phase 1. The final research design and methods were finalized in Phase 3.
The research plan was developed to conceptualize students’ academic pathways from the
community college, to transfer to a four-year institution, to graduation from UMUC. In
developing the research plan, specific milestones in students’ academic pathways were modeled.
These included: (a) earning a successful first-term GPA, (b) re-enrolling in the immediate next
semester after transfer, (c) retention (re-enrollment within a 12-month window), and (d)
graduation within an eight-year period. Each of these milestones was predicted based on data
aggregated from the community college and UMUC. Specifically, students’ demographic
information, community college course taking behaviors, indicators of first-semester
performance at UMUC, and behaviors in the online classroom were used in predictive modeling
and in data mining. A model presenting students’ academic trajectories from transfer to
graduation that guided the research was developed.
PASS project goals included:











To develop a data sharing partnership and create a memorandum of understanding
between partner community colleges and UMUC
To build an integrated database tracking students across institutions, from community
college to UMUC
To integrate data from students’ community college backgrounds with UMUC
performance data for use in research
To use data mining to develop profiles of transfer student success at UMUC
To identify factors from students’ community college academic backgrounds that predict
success at UMUC
Develop predictive models of UMUC first-term GPA, re-enrollment, retention, and
graduation based on community college data
Examine graduation rates of community college transfer students at UMUC
Examine online classroom engagement as associated with course performance
Profile students’ motivational and self-regulatory attributes
Develop, implement, and evaluate interventions aimed at promoting community college
transfer students’ success
Disseminate research findings
11
Grant Partnership
UMUC is a four-year public university that offers online degree programs to a diverse population
of working adults. With support from the Kresge Foundation, UMUC established partnerships
with two Maryland community colleges that also serve large and diverse student populations.
Montgomery College (MC), established in 1946, enrolls over 60,000 students annually. Prince
George’s Community College (PGCC) enrolls more than 40,000 students from approximately
128 different countries. Both institutions serve the metro-D.C. area, but differ in that PGCC
serves more low-income students. Both institutions have endorsed the goals of this project and
are committed to working with UMUC to find ways to promote student success throughout their
academic careers.
Objectives and Milestones
Specific objectives and milestones were identified for each phase of the research project. These
objectives and milestones have been modified throughout the course of the project, but are
consistent with grant requirements. Table 1 presents the objectives and milestones for each
phase.
Table 1. Project objectives and milestones
Objectives
Milestones
Status
April 2011 – October 2012
Phase 1
Develop a
Develop a project action and collaboration plan with
Complete
Project Action
the partnering agencies.
Plan
Data Collection
Prepare a data ―universe‖ (integrated database system) Complete
and Preparation
on CC transfer students in the UMUC population
(KDM)
Understand variables; define student characteristics
Complete
and retention data; develop data dictionary.
Data Analysis
Conduct initial predictive analyses and employ data
Complete
mining techniques to identify factors contributing to
students’ success
Project
Conduct ongoing project evaluation. Take action on
Complete
Evaluation
identified areas for improvement.
November 2012 – October 2013
Phase 2
Develop and
Analyze data and identify factors that predict
Complete
Validate
success/failure.
Analytic Models Validate predictive analyses and models developed
Complete
of Student
through data mining techniques to predict students’
Success
success and retention at UMUC.
Build student profiles based on analyses.
Complete
Disseminate Key Discuss results with Kresge Workgroup and share
Complete
Findings
with advisory board.
Discuss results with Project Partners and obtain
Complete
feedback.
12
Objectives
Develop
Interventions
Project
Evaluation
Research Plan 3
Phase 3
Develop
Interventions
Implement Pilot
Interventions
Disseminate
Results on
Interventions
Phase 3 Analyses
Report Findings
Dissemination of
Results and
Resources
Project
Evaluation
Milestones
Present key findings at national conferences on higher
education
Work with stakeholders at UMUC and CC partners to
develop a list of potential interventions.
Conduct ongoing project evaluation. Take action on
identified areas for improvement.
Design and develop KDM2 to update and improve
data related to student success
Status
Complete
Complete
Complete
Complete
Plan Phase 3 analyses on expanded integrated data.
Complete
November 2013 – December 2014
Review relevant literature on interventions that
Complete
promote student success in online learning.
Develop an implementation plan and timeline for
Complete
piloting of interventions.
Implement and evaluate pilot interventions.
Complete
Develop and disseminate report on the pilot
interventions
Complete
Develop and execute Phase 3 research plan
Present key findings from Phase 3 analyses at national
conferences; publish research in journals
Prepare written report of both Phase 3 analyses and
full scope of Kresge grant work.
Develop website and repository for educational data
mining and student success.
Host a national convening on data mining and learner
analytics.
Deliver final project evaluation.
Complete
Complete
Complete
December
2014
Complete
December
2014
13
SECTION 2: LITERATURE REVIEW
The literature review was conducted over the course of the four-year project. This section
presents a review of the literature in the following areas:
 Theoretical models of community college transfer student performance
 Educational data mining
 Predicting transfer student first-term GPA
 Predicting transfer student re-enrollment
 Literature guiding interventions
 Community college student transitioning
 Literature to support specific interventions
Theoretical Models of Community College Transfer Student Performance
Two theoretical models of community college transfer student performance and persistence have
guided work in the field, as well as in the PASS project analyses. The first is Tinto’s (1975,
1987) Student Integration Model, which applies a psychological lens to understanding student
attrition. The Student Integration Model identifies four aspects of student-institution interactions
that have an effect on persistence. Specifically, these are the background characteristics and
academic goal commitments that students bring to a university setting, and in turn, their effects
on students’ academic and social integration at the transfer institution. Background
characteristics include students’ demographic attributes, family backgrounds, and experiences
prior to college (Tinto, 1975). Goal commitments include learners’ motivation for degree pursuit
and educational expectations as well as institutional commitment to a particular university.
Academic and social integration is based on students’ interactions with a variety of institutional
features over time. Tinto (1975) suggests that these interactions may be evaluated based on both
structural and normative considerations. Structural considerations refer to objective and explicit
social and academic standards that students may have to meet (e.g., a minimum GPA, meeting
with an advisor), whereas normative components of integration relate to students’ identifications
with these standards (e.g., earning a high GPA). Tinto emphasized the central importance of
students’ institutional integration, both academic and social, by saying, ―we learned that
involvement matters and that it matters most during the first critical year of college,‖ (Tinto,
2006, p. 3; Upcraft, Gardner, & Barefoot, 2005).
At the same time, academic and social integration into a transfer institution are not givens for
many students. Building on Tinto’s earlier work (1975), Bean and Metzner (1985) developed a
model of attrition, reflecting the experiences of non-traditional undergraduate students, termed
the Conceptual Model of Non-Traditional Student Attrition. In their definition, non-traditional
undergraduate students may be defined as those who are older (i.e., 25 and above, Stewart &
Rue, 1983), enrolled part-time, non-residential, commuting to campus, or representing some
combination of these characteristics (Bean & Metzner, 1985). Understandably, this population
of students is considered to undergo a different socialization process from that of traditional
students conceptualized in Tinto’s model (1975). Non-traditional students may have different
experiences with and potential for institutional commitment and social integration. Bean and
Metzner (1985) suggest that this may be because older students exhibit greater characteristics
14
associated with maturity and therefore may be less open to the socialization process and because
these students have more limited contact with socializing agents (e.g., faculty, peers, Chickering,
1974). More generally, non-traditional students may be less interested in institutions’ social
culture, and rather more concerned with academic offerings and credentials.
Juxtaposing the experiences of traditional and non-traditional students, for non-traditional
learners there is (a) more limited interaction with faculty and peers as well as with college
services (i.e., more limited social integration, as per Tinto, 1975), (b) similarity in academic
focus and experience (i.e., parallel classroom experience), and (c) much greater interaction with
the external, non-institutional environment (Bean & Metzner, 1985).
Based on the differences identified between traditional and non-traditional students, Bean and
Metzner (1985) conceptualize students’ decisions to drop out as predicated on four general types
of variables. The first of these are background factors, including students’ demographics, past
academic performance, and educational goals and expectations. The second group of
considerations is students’ academic performance, or factors reflecting learners’ grades, study
habits, and pursuit of major at the transfer institution. The third group of factors are students’
intent to leave, considered to be more psychological; these include students’ goal commitment,
perceived utility of a given degree, and institutional satisfaction. Finally, unique to this model,
the fourth group of factors are external factors that may have a direct effect on students’
decisions to drop-out; these include finances, out-of-school work, and family commitments
(Bean & Metzner, 1985).
There are two compensatory relationships between variables identified. First, if students’
academic outcomes are low, they may nonetheless persist, compensating with high levels of
psychological commitment. Further, when academic performance is low, students will persist if
environmental factors support their continued enrollment. Conversely, when environmental
factors do not support persistence, for non-traditional students, even high academic performance
may not be sufficiently compensatory. More generally, Bean and Metzner (1985) suggest that
for non-traditional students, environmental factors may have a much more pronounced effect on
attrition decisions than do academic factors, as non-traditional students are much more closely
affiliated with the non-institutional environments than are traditional students residing on
university campuses (Bean & Metzner, 1985; Metzner, 1984).
As such, understanding non-traditional student persistence may be particularly challenging at the
institutional level as, in large part, it may be attributed to environmental factors that the
institution may not be aware of or able to control. Indeed, for non-traditional students, the
primary point of institutional interaction has to do with academic factors; as such these areas
represent targets for intervention (Bean & Metzner, 1985).
As part of the PASS project, we were interested in gaining insight into community college
transfer students’ persistence at a four-year institution by looking longitudinally to consider
which background factors, including learner characteristics and community college experiences,
may impact student re-enrollment and continued pursuit of educational goals.
15
Educational Data Mining
Current literature on student success focuses on outcomes such as course success, course
withdrawal and retention. For example, variables such as student characteristics, previous course
work, grades, and time spent in course discussions and activities may be useful in predicting
course success (Aragon & Johnson, 2008; Morris & Finnegan, 2009; Morris, Finnegan & Lee
2009; Park & Choi, 2009). Course-level variables acquired from student login data from the
LMS may have predictive value in measuring course withdrawal rates (Willging & Johnson,
2008; Nistor & Neubauer, 2010). Variables such as student characteristics, number of transfer
credits, final grade in any given course, experience in online environments, and course load may
be useful in predicting re-enrollment and retention (Aragon & Johnson, 2008; Morris &
Finnegan, 2009; Boston, Diaz, Gibson, Ice, Richardson & Swan, 2011).
Although these studies showcase a variety of findings related to student success, the majority of
studies of retention in online learning environments use traditional statistical or qualitative
methods. Park and Choi (2009) point out that expansion of methods such as data mining may
have utility when student, course, program, and institutional level variables are well defined and
institutionally meaningful. Literature related to educational data mining focuses on exploratory
research.
Data mining is a method of discovering new and potentially useful information from large
amounts of data (Baker & Yacef, 2009; Luan, 2001). Educational data mining is a subset of the
field of data mining that draws on a wide variety of literatures such as statistics, psychometrics,
and computational modeling to examine relationships that may predict student outcomes
(Romano & Ventura, 2007; Baker & Yacef, 2009). In educational data mining, data mining
algorithms are used to create and improve models of student behavior in order to better
understand student learning (Luan, 2002).
Data mining methods are most helpful for finding patterns already present in data, not
necessarily in testing hypotheses (Luan, 2001). Baker and Yucef (2009) suggest that research in
higher education should use a variety of algorithms, such as classification, clustering or
association algorithms in determining relationships between variables. Although many
definitions of these techniques exist in data mining literature, Han and Kamber (2001) offer the
following definitions. Classification is the process of finding a set of models or functions that
describe and distinguish data classes or concepts to predict a class of objects whose class label is
unknown. Clustering analyzes data objects that are related to similar outcomes without
consulting a class label. Association is the discovery of rules showing attribute value conditions
that occur frequently together in a given set of data (Han & Kamber, 2001).
Recent research suggests that these data mining algorithms can be used to examine variables
related to student success. Yu, DiGangi, Jannach-Pennell, Lo, and Kaprolet (2010) used a
classification algorithm to explore potential predictors related to student retention in a traditional
undergraduate institution. In this study, the authors used a decision tree to explore demographic,
academic performance, and enrollment variables as they related to student retention. This study
revealed a predictable relationship between earned hours and retention, but also found that at this
institution, retention was closely related to state of residence (in-state/out of state) and living
16
location (on campus/off campus). The authors speculate that this finding points to the potential
utility of online courses in improving retention for out-of-state or off-campus students.
Despite these recent developments in exploring variables related to student success in traditional
higher education settings, research using data mining techniques to uncover relationships among
variables in online courses is limited in scope. The PASS project is designed to fill this gap in the
extant literature by utilizing data on online students who attended multiple institutions.
Predicting Transfer Students’ First-Term GPA
Generally, the transition from community college to a four-year university has been considered
to be a stressful period for students. In early examinations of this transitional period, Hills
(1965) determined that when students from junior college transfer to a four-year university they
might experience an ―appreciable loss in their level of grades‖ (p.209), termed transfer shock.
Transfer shock has been defined as a decrease in academic performance (i.e., GPA) experienced
by students in their first semesters at a four-year university, due to difficulties with adjustment
(Keeley & House, 1993). Since Hill’s (1965) initial exploration, a wealth of studies have
emerged examining transfer shock and students’ decreases in GPA when transitioning from
community college to a four-year university (e.g., Best & Gehring, 1993; Keeley & House, 1993;
Preston, 1993; Soltz, 1992).
However, recent research has painted a more nuanced picture of transfer shock. Cejda, Kaylor,
& Rewey (1998) determined that transfer shock is discipline specific. For instance, while
students transferring into mathematics and science majors did experience a drop, those majoring
in the fine arts and humanities actually experience an increase in GPA. Further, in a metaanalysis of 62 studies examining transfer shock, Diaz (1992) determined that while the majority
of studies did find that community college students experience a transfer shock, it was slight
(i.e., one half of a grade point or less); also, the majority of studies reviewed found that students
recovered from transfer shock over the course of their university careers. Nickens (1972)
skeptical of transfer ―shock‖ and ―recovery‖ suggests that transfer students’ GPAs cannot be
distinguished from the GPAs of their native counterparts. Specific decreases in GPA may be
explained by difference in institutional practices and any subsequent increases in GPA may be
explained by regression to the mean and the attrition of weaker students (Nickens, 1972).
Regardless of findings, across studies examining community college students’ performance
when transferring to a four-year university, first-term GPA has been a key outcome of study
(Carlan & Byxbe, 2000; Driscoll, 2007; Glass & Harrington, 2010; Hughes & Graham, 1992;
Townsend, McNerny, & Arnold, 1993). This may be because first-term GPA has been
considered to be a barometer of transfer students’ success and adjustment to a four-year
institution (e.g., Knoell & Medsker, 1965) as well as level of preparedness to meet the academic
demands of a four-year university (Carlan & Byxbe, 2000; Roksa & Calcagno, 2008). Further,
first-term GPA has been considered to be strongly associated with persistence or students’
retention and graduation from a four-year university (Gao, Hughes, O’Rear, & Fendley, 2002).
Indeed, there have been a number of studies examining predictors of first-term GPA for
community college transfer students (e.g., Graham & Hughes, 1994; Townsend et al., 1993).
Most commonly, demographic factors have been examined as potentially impacting community
17
college students’ transfer success. For example, Durio, Helmick, and Slover (1982) found that
demographic factors (i.e., gender and ethnicity) impacted transfer students’ success. Examining
an expanded pallet of variables predicting first-term GPA, Keeley and House (1993) considered
students’ age, gender, ethnicity, college major, residence status, as well as class standing (e.g.,
sophomore) as predictive of first-term GPA. In particular, age (i.e., being older) and gender (i.e.,
being female) were found to the associated with higher first-term GPA for transfer students, as
was having earned an associate degree prior to transferring. In addition to the focus on students’
demographic factors, GPA at the community college level has been found to be a key
determinant of first-term GPA when students transfer to a four-year institution (Baldwin, 1994;
Towsend, McNerny, & Arnold, 1993). However, more research is needed to identify predictors
of transfer students’ success at a four-year university (Johnson, 1987).
Course Taking Behavior at the Community College
In examinations of community college transfer students’ performance at four-year institutions, at
the forefront have been considerations of students’ preparedness to handle the challenges
associated with university-level course work (e.g., Berger & Malaney, 2003; Keeley & House,
1993; Townsend, 1995; Townsend et al., 1993). Despite concerns over community college
transfer students’ preparedness, limited research has examined the nature of community college
students’ course taking backgrounds to determine predictors of university success. Some studies
provide initial insights. For example, Phlegar, Andrew, and McLaughlin (1981) determined that
students fundamentally prepared in key subject areas (i.e, math, science, and English) at the
community college level performed better upon transferring. Deng (2006) determined that
students attending career-focused community college programs outperformed those attending
liberal-arts community college programs, when transferring to a four-year university. Rather
than considering specific courses of study, Pennington (2006) determined that students’
enrollment in developmental course work in community college was associated with a decreased
GPA upon transfer to a four-year institution.
Carlan and Byxbe (2000) found community college major to be significantly associated with
first-term GPA; for instance, students majoring in education and psychology had a higher GPA
after transfer than did students majoring in business and the sciences. However, it is unclear
whether these major-specific differences in GPA drop were associated with different levels of
students’ preparedness or with cross-institutional differences in the academic demands required
by these various programs of study.
Rather than examining community college majors, Cejda et al. (1998) found students’ first-term
GPA to be related to university major. Parallel to prior findings (i.e., Carlan & Byxbe, 2000)
students in the sciences, indeed, experienced a drop in first-term GPA, while students in the fine
arts, humanities, and social sciences experienced a GPA increase. This replicated findings that
students majoring in the sciences and mathematics (i.e., biology, chemistry, math, physics,
accounting, and economics) had a lower GPA than their fellow community college transfer
students (James Madison University, 1989). However, the nature of students’ preparedness for a
four-year university and the types of community college academic experiences that may support
transfer success have yet to be fully examined.
18
Studies examining community college students’ preparedness have primarily examined students’
academic backgrounds at the level of the major (e.g., Carlan & Byxbe, 2000). Institutional data
sharing as part of the PASS project, allowed the specific courses of community college students
to be examined as predictors of performance at the four-year institution.
Predicting Transfer Student Re-Enrollment
Historically, research on student retention largely focused on the experiences of traditional
students, until Tinto (1993) expanded on extant models of retention to consider which factors
may impact the retention of non-traditional students. For both traditional and non-traditional
students, retention was thought to be a consequence of students’ academic and social integration
(Tinto, 1993). Other research has echoed the central role of social factors in predicting retention
for non-traditional students, online, and distance learners (Boston, Diaz, Gibson, Ice, Richardson,
& Swan, 2009). At the same time, a number of demographic and community college factors
have been considered as predictive of community college transfer students’ persistence at a fouryear university.
Based on a comprehensive review of the persistence literature, Peltier, Laden, and Matranga
(1999) determined that gender, race and ethnicity, socioeconomic status, high school GPA,
college GPA and interaction variables are all related to persistence. In particular race/ethnicity
and prior academic achievement have been robust predictors of persistence (e.g., Astin, 1997;
Tross, Harper, Osher, & Knwidinger, 2000; Levitz, Noel, & Richter, 1999), whereas findings
with regard to gender have been more mixed, Reason, 2009; St. John et al., 2001). Wetzel,
O’Toole, and Peterson (1999) used logistic regression, with a dichotomous outcome variable,
retained or not. Retention was significantly predicted primarily based on academic factors,
including GPA, earning a low GPA which places students at low academic risk, and the ratio of
credit hours earned to the credits attempted.
Murtaugh, Burns and Schuster (1999) used survival analysis to examine the retention of
undergraduate students, enrolling in a university between 1991 and 1996; 25% to 35% of the
cohort examined had interrupted enrollment within this period. Specifically, 13.5% stopped out
for a single term, 10.8% had stopped out for two terms, and 1.8% had stopped out for three
terms, after which they were required to undergo a readmission process. Predictors of stopping
out were referred to as hazards. Hazards were examined for one year, two year, and four year
retention. Minority status had a higher rate of withdrawal than did white students; also associated
with withdrawal was age, high school GPA, first quarter GPA, area of study, and participation in
freshman orientation. In particular, Murtaugh et al. (1999) highlight the importance of precollege characteristics in predicting persistence.
Looking at a sample of traditional, first time freshman, Cabrera, Nora, and Castaneda (1993)
used structural equation modeling to analyze predictors from both Tinto’s (1975) and Bean and
Metzner’s (1985) models to predict student persistence. Cabrera et al. (1993) ranked variables
predicting persistence; the most important factor was psychological goal commitment, or intent
to persist, followed by GPA, institutional commitment, and encouragement from family and
friends. In turn, intent to persist was predicted by institutional commitment, encouragement
from family and friends, academic goal commitment, and academic integration – these factors
having an indirect effect on persistence.
19
Whereas the aforementioned studies focused on individual student factors predicting retention,
Moore and Fetzner (2009) addressed the institutional characteristics that fostered commitment in
non-traditional students. These factors included having a leadership culture that fosters
commitment to student success and institutional policies and practices that incorporate student
support services and technological support. For online learners, access to services and support
that meet their needs was found to be crucial (Moore & Fetzner, 2009). Further, student
satisfaction, defined as students happy with their progress and with support received for learning,
and with a perception that the knowledge they were learning was valuable, was predictive of
retention. Faculty satisfaction, stemming from involvement in curricular design and training in
the use of online technologies supporting learning, were found to be key to engagement and
contributors to retention (Moore & Fetzner, 2009).
Predicting Re-Enrollment for Non-Traditional Students
Based on theoretical work (Astin, 1975; Bean & Metzner, 1985; Tinto, 1975), we may expect
that community college transfer students’ persistence may be affected by different factors. First,
given that much of the literature examining community college students performance has
focused on the degree of student preparedness (Carlan & Byzbe, 2000), learners’ prior academic
experiences may be particularly important to examine, especially as they include not only high
school work, as for traditional students, but college-level course work at a two-year institution as
well. Further, to the extent that transfer students enter more connected to external factors beyond
their experiences at the transfer institution, it may be particularly important to examine learner
background characteristics and how these are related to academic factors at the transfer
university.
Wang (2008), using logistic regression, found that the probability of graduating with a bachelor’s
degree for students starting at community college was predicted by gender, socio-economic
status, high school curricula, educational expectations, community college GPA, college
involvement, and math remediation; while persistence, prior to graduation, was predicted by
community college GPA and locus of control. Just as in the Wang (2008) study, in the PASS
project, researchers looked to students’ demographics and community college factors, including
course taking behaviors as well as overall performance, to predict next-semester re-enrollment.
Kreig (2010) examined students at Western Washington University, an institution with a
substantial population of community college transfer students comprising each education level,
and found that native students were more likely to graduate, even after demographic
characteristics and prior academic performance were controlled. Krieg (2010) compares the
experience of community college students to that of freshmen at a four-year university. For new
community college students, there may be a difficult adjustment to a new learning context, which
may result in early attrition if students consider themselves to be incompatible with the new
environment. As such, first year retention is a particularly important factor to consider in
understanding students’ persistence and ultimate graduation.
While this tension in fit has most commonly been examined by considering the drop in
performance (i.e., GPA) that community college students experience upon transfer to a four-year
university, alternately termed transfer shock (Cejda, Kaylor, & Rewey, 199; Townsend,
20
McNemy, & Arnold, 1993), Krieg (2010) suggests that this may more profoundly manifest in
rapid attrition from the four-year institution. More generally, there may be an interaction
between transfer student status, first-term GPA, and drop-out rates (Spady, 1970). Specially,
those transfer students scoring a low GPA in the first quarter were twice as likely to drop out as
were native students (Krieg, 2010). Pascarella and Terenzini (1980) likewise conclude that the
majority of attrition occurs in the freshman year, when students are new to the university setting,
and further indicate that this marks a misalignment between theory and evidence. For instance,
Tinto’s model of academic attrition is better suited to modeling student attrition beyond the first
year.
The difference Krieg (2010) documents, is not specific to low performing students. Even high
performing community college transfer students are more likely to drop-out than are their native
counterparts. This may be because transfer students have less immediate affiliation and
integration into the transfer institution or because these transfer students are required to take
prerequisite courses before entering into a major (Krieg, 2010). This points to the importance of
looking beyond community college students’ prior academic performance, to look at specific
course taking behaviors at the community college as well as to consider first-term GPA at the
transfer institution – these factors were examined as part of the PASS analyses. More broadly,
these findings are aligned with the interactional relationships identified in Bean and Metzner’s
model (1985) that suggest that for non-traditional learners, academic success may not be a
sufficient factor to promote persistence.
Literature Guiding Interventions
Intervention with specific populations (e.g., community college transfer students) and in specific
contexts (e.g., online universities) are needed, as the majority of interventions have focused on
finding solutions that will have a general effect on a broad population of students (Pascarella &
Terenzini (1998).
Two prominent models of student retention have been proposed, however, both of these models
speak primarily to the needs of traditional students. Tinto proposed the Student Integration
Model, which identified attrition as resulting from a lack of congruency between students’ needs
and institutional offerings (1987). Specifically, Tinto points to the need for students’ academic
abilities and motivational orientations to match an institution’s academic and social
characteristics. In determining whether or not students will persist in post-secondary education,
Tinto (1987) suggests that two forms of commitments must be in place. The first is students’
commitment to educational goals and the second is students’ commitment to remain within a
particular institution.
From Tinto’s model of student retention, conclusions may be drawn regarding the types of
factors that interventions to promote retention ought to foster in students; specifically Tinto’s
model suggest the need for interventions that target students’ (a) academic abilities, (b)
motivational orientations, specifically with regard to the types of educations goals students adopt
in pursuing higher education, and (c) institutional connections. Intervention designs should
emphasize the correspondence between students’ abilities or goals and institutional offerings.
21
Yet more work is needed to understand how to adapt the Student Integration Model (Tinto,
1987), to reflect the experiences of non-traditional students, transfer students, and students
enrolled in online universities, such as UMUC. In particular, factors affecting students’ retention
may deviate from the proposed model based on differences in the type of institution students are
a part of as well as students’ gender and ethnicity (Pascarella & Terezini, 1997). To this end,
proposed interventions are geared not only with general UMUC student populations, but also
speak to the specific needs of female students (e.g., Girls to Women) and diverse learners (e.g.
mentors in the Community College Mentor and College Writing interventions are matched with
mentees according to demographic characteristics, including ethnicity.
Tinto’s (1987) model has further been critiqued for being limited in considering the role that
external factors, or considerations independent of students and institutions, may have on
retention (Pascarella & Terezini, 1997). These external factors, including financial and familiar
considerations, may be particularly important to consider when modeling retention of nontraditional students. Studies have found that often times these students do not persist in postsecondary education because of finances, employment demands, and taxing family
responsibilities (Bean & Metzner, 1985).
To expand Tinto’s model and to consider the needs of non-traditional students – those classified
as part-time, older, and non-residential (e.g., online learners at UMUC) – Bean and Metzner
proposed a Student Attrition Model (1985). This model suggests that students’ persistence and
academic outcomes can be understood as a result of four factors, namely: (a) background
variables, (b) academic variables, (c) psychological factors, and (d) environmental variables.
Background variables refer to students’ characteristics that may put them at a risk or deficit
relative to their peers. These factors include age, high-school performance, gender, and
ethnicity. Academic variables include students’ study habits, the role of advising, and students’
certainty in their major. Psychological factors reflect students’ motivation for engaging in postgraduate education – these include students’ goal commitment, the expected utility or value of a
degree, and psychological stress. Finally, environmental factors introduced in the model address
students’ responsibilities outside of the university and may represent constraints on students’
pursuit of educational goals. The role of social interaction is featured in this model, as previous
models have identified the importance of social integration in predicting students’ persistence
(e.g., Tinto, 1975; Pascarella & Chapman, 1983), however, the nature of social interactions may
differ for traditional versus non-traditional students.
In designing interventions, all four of the factors described in Bean and Metzner’s model were
considered. In particular, mentoring programs targeted students and matched mentees with
mentors according to background variables. Further, mentoring programs were intended to help
students in mitigating the effects of environmental variables; the intention of these interventions
was to provide students with role-models who have successfully persisted, despite limiting
external factors. In providing students with an Introductory Check-List and academic tutoring,
the interventions were designed to impact academic variables. Finally, psychological outcomes
were targeted by encouraging students to take advantage of the advising available through
UMUC and providing students’ with the opportunity to discuss their long term and professional
goals with mentors.
22
One of the unique challenges for non-traditional students is the identified lack of social
integration and social interaction (Bean & Metzner, 1985). A number of interventions were
geared toward connecting students with social resources. For instance, the checklist intervention
encouraged students to be involved with available student organizations. Further, to the extent
that persistence is marked by a match between a student and an institution (Cabrera, Nora, &
Castaneda, 1993), a number of the interventions were aimed specifically at helping students
recognize others like them as members of the UMUC community.
Community College Transfer Students’ Transitioning
Transferring from community college has been identified as a high-stress time for students,
presenting academic, psychological, and environmental challenges (e.g., Laanan, 2001). Flaga
(2006) identified five dimensions of transitioning. These are, learning resources, connecting,
familiarity, negotiating, and integration. The first two dimensions deal with the knowledge and
skills that students need in order to be successful, whereas the last three dimensions address how
these skills may develop over time.
Learning resources refer to the tools students may use to gain information about the university.
Three types of learning resources were specified; these were: (a) formal resources provided by
the university (e.g., orientation information), (b) informal resources provided by individuals
knowledgeable about the university but not officially affiliated (e.g. information from alumni),
and (c) initiative-based resources that students gather independently (Flaga, 2006). The second
dimension, connecting, refers to the relationships that students are required to form when
transferring to a new institution; including (a) academic connections (e.g., with faculty), social
connections (e.g., with other students), and physical connects (e.g., with the university
environment) (Flaga, 2006).
The third dimension, familiarity, emerges when students become more comfortable with their
new environment. The fourth dimension, negotiating, occurs when students adjust their
behaviors to better fit their new environment. Finally, the fifth dimension, integrating, does not
always happen, but involves students shifting their identities to reflect their new institution
(Flaga, 2006).
Literature to Support Specific Interventions
Checklist
The Checklist targeted the first two dimensions identified by Flaga (2006) as supporting
students’ transitioning. Specifically, through the checklist, students received support
encouraging them to connect with both formal and informal information resources with the intent
of forming academic, social, and physical connections. In completing the activities specified in
the checklist, students had the opportunity to exercise initiative in connecting with resources and
develop familiarity with UMUC as an institution and an academic community.
In a qualitative study of community college transfer students, one of the recommendations
transfer students proposed as a resource to help their transition was the creation of a transfer
23
checklist (Owens, 2007). Indeed, 27% of students expressed a desire for the introduction of a
checklist or guide to aid them in the transfer process (Owens, 2007). In describing the features
that would make checklists appeal to them, students expressed desires for ease-of-use, online
availability, and comprehensiveness (with information ranging from where to park to how to
register for classes); as well as checklists that break down complex processes in a step-by-step
manner and include necessary contact information (Owens, 2007).
This is a particularly important initiative given that surveys of community college students have
determined that students have a need for more information (e.g., Harbin, 1997; Andres, 2001)
and more assistance (Townsend & Wilson, 2006) as they move to their new institutions.
Community College Mentor
Peer-mentoring for community college students transferring to four-year schools has been underexamined in the literature. However, mentoring interventions have been broadly used as an
avenue to promote students’ retention (Good, Halpring, & Halprin, 2000; Hoyt, 2000).
Flaga (2006) suggests that benefits associated with peer mentoring are not only academic;
through mentoring, students gain access to informal learning resources and have the opportunity
to socially connect with their peers. Likewise, Good et al. (2000) found mentoring to confer
psychological and academic benefits to both mentors and mentees.
The mentoring relationship has been identified as supporting three types of outcomes, namely
psychosocial, vocational, and role-modeling (Ensher, Heun, & Blanchard, 2003). Psychosocial
support refers to mentors providing counseling, friendship, and, encouragement to their mentees
(Enscher et al., 2003). Vocational support is considered to be support that enhances the
professional lives of mentees (Enscher et al., 2003) and can be extended to include the academic
support provided by mentors to new students. Finally, role-modeling refers to mentors
demonstrating appropriate behaviors or expectations, either implicitly or explicitly (Enscher et
al., 2003). For example, role-models can offer examples of effective study strategies or describe
appropriate standards of communication when conferring with professors. Tinto (2001) further
suggests that peer mentor relationships can address both specific classwork and general skills
associated with successful college completion. Moreover, these benefits can affect mentees as
well as mentors (Good et al., 2000; Snowden & Hardy, 2012).
Mentoring has been found to be particularly beneficial for minority students (Good et al., 2000;
Redmond, 2000). Redmond (2000) suggests that mentoring programs must adopt the following
goals to meet the needs of diverse students: (a) promote greater student contact, (b) promote
students’ use of services for support with non-academic problems, (c) intervene quickly when
students encounter academic difficulties, and (d) develop culturally-sensitive psychosocial
environments.
A case study for mentorship in diverse communities is the ALANA (Asian, Latin, Africa, and
Native American) mentoring program, targeting minority community college students (Mueller,
1993). The stated goals of the ALANA program were to (a) provide social and academic
support for minority students, (b) engage in role-play to help students critically think through
24
challenging situations, and (c) assist students in making time-sensitive decisions (e.g., course
add/withdrawal). Mentors in the ALANA program seek to maximize social interaction with
their mentees as a mechanism for relieving students’ anxieties (Mueller, 1993).
Peer mentoring has been shown to benefit students transitioning from two- to four-year
institution and those in distance education programs. For instance, Lenaburg, Auirre, Goodchild,
and Kuhn (2012) reported on the impact of a program that oriented community college students
to a four-year institution. As part of the program, students were provided with peer mentors. At
the conclusion of the program, participants rated their peer mentor experience very highly,
commenting that peer mentors were instrumental in explaining the transfer process, providing
social support and helping them maintain interest in a four-year institution. Most recent results
suggest that peer mentors were instrumental in helping students transition from community
colleges to a four-year university. Peer mentoring has also benefitted students new to online
learning contexts (Boyle et al., 2012; Brown, 2011). A study of peer mentoring programs in three
distance education universities, for example, found evidence of improvement in mentees’ course
passage rates, retention, and sense of belonging (Boyle et al., 2012).
Though not evaluated in the empirical literature, the University of California at Berkeley has a
mentoring program for transferring community college students: the Starting Point Mentorship
Program. Through this program, transferring students are paired with mentors who offer: (a)
guidance, (b) motivation, and (c) access to campus and community resources. Specifically, the
benefits to mentees are outlined as: advice on study skills, time management and goal-setting,
information about the differences in academic and social culture between community college and
a four-year institution, encouragement to set and pursue academic goals, and the point-of-view of
a current student.
Despite the likelihood that peer mentoring can mitigate the shock of student transfer—either
from community college to a four-year institution or from face-to-face to online environments—
there have been few experimental studies directly assessing peer mentoring programs’ impact on
key student indicators (Boyle et al., 2010). Further, to date, there have been no such studies of
peer mentoring for students experiencing the double shock of transferring from a largely face-toface community college to an online, four-year institution.
25
SECTION 3: RESEARCH SCOPE AND DESIGN
In Phase 3, research was undertaken to expand on and validate initial findings from Phases 1 and
2. In particular, variables previously identified as potentially predictive of performance and
persistence at UMUC, as well as newly introduced factors, were used to predict key outcomes
throughout the path model of students’ academic trajectories. The path model identifies the
academic milestones along the path to completion for community college students. (See figure
1.)
Figure 1. Path model of students’ academic trajectory from community college to UMUC.
Community
College
Data
UMUC First
Term GPA
Reenrollment
Retention
Graduation
Research Questions
Predictive modeling was used to answer the following research questions related to students’
performance, persistence (re-enrollment and retention), and ultimate achievement of a credential
(graduation)
Performance
1. To what extent do demographic characteristics, community college course taking
behaviors, and community college performance metrics predict earning a successful
first-term GPA (2.0 or above) at UMUC?
Persistence
2. To what extent do demographic characteristics, community college course taking
behaviors, community college performance metrics, and UMUC first-term GPA predict
re-enrolling at UMUC in a semester immediately following the first semester of
transfer?
3. To what extent do demographic characteristics, community college course taking
behaviors, community college performance metrics, and UMUC first-term GPA predict
retention at UMUC, or re-enrollment within a 12-month window following the first
semester of transfer?
Graduation
4. To what extent do demographic characteristics, community college course taking
behaviors, community college performance metrics, and UMUC first-term metrics
predict graduation from UMUC?
5. What are the graduation rates of community college transfer students at UMUC?
26
Beyond building predictive models of key milestones along students’ academic trajectories,
students’ experiences while enrolled at UMUC were examined. In particular, two aspects
shaping students’ academic trajectories were examined.
First was an examination of students’ motivational and self-regulatory profiles as they relate to
socio-demographic characteristics (e.g., employment status, family structure). Examining
motivation and self-regulation as well as probing aspects of students’ background introduced a
deeper examination of students’ backgrounds that may shape their experiences at both the
community college and the transfer institution.
Further, data mining analyses were used to examine whether students’ engagement in the online
classroom, as measured by UMUC’s LMS, was associated with performance at UMUC. This
examined the extent to which students’ interactions within the online classroom was potentially
facilitative of meeting academic milestones (e.g., earning a successful GPA).
These in-depth, learner-focused analyses introduced two additional research questions:
Learner-Focused
6. What is the association between student motivational and self-regulatory characteristics,
socio-demographic factors (e.g., employment status, family structure) and performance at
UMUC?
7. What is the nature of students’ engagement in the online classroom environment and its
association with successful course completion?
Student Population
The population of interest for analyses was defined as first-term undergraduate students, whose
first semester of transfer to UMUC, from MC or PGCC, was between Spring 2005 and Spring
2012.
In this report, a number of outcomes reflecting student success and corresponding to key
academic milestones were examined. These are defined as:
Successful first-term GPA–earning a GPA of 2.0 or above in the first semester at UMUC
Re-enrollment–enrollment in the immediate next semester after initial enrollment
Retention–re-enrollment at UMUC within 12 months after initial enrollment
Graduation–earning a first bachelor’s degree from UMUC within a specified time period,
specifically within 4, 6, or 8 years of transfer
Models predicting each of the target outcome variables were developed, with results presented in
Sections 7 and 8. Further, learner-focused analyses were undertaken examining the relation
between online course engagement and performance and motivational and self-regulatory
profiles, socio-demographic characteristics and performance with results presented in Sections 9
and 10.
27
SECTION 4: DATA SOURCES
One of the key achievements of Phase 1 of the Kresge research grant was the development of the
KDM, an integrated multi-institutional database that aligns the academic work of transfer
students across institutions. Data for the KDM came from three student information systems:
1. Banner - Montgomery College’s student information system
2. Datatel - Prince George’s Community College’s student information system
3. PeopleSoft – UMUC’s student information system.
All data were made anonymous to protect students’ information. Demographic, academic,
transfer, and enrollment data were collected on each student from each institution. Demographic
data included students’ gender, age, marital status, and race/ethnicity. Enrollment data included
course registration, program of study or major, and student status. Community College academic
data included information about students’ academic history prior to transferring to UMUC, such
as course grades, repeated courses, and remedial coursework. Transfer data included the number
of courses transferred, transfer GPA, and prior degrees earned.
The standardization and alignment of data across institutions was accomplished in Phase 2. Due
to the institution-specific design of each student information system, a data dictionary was used
to document the name, definition, type, range and default value of each element as it existed in
its native system as well as its transformation and standardization in the KDM. As research
progressed, categorical and derived fields were developed and added to the data dictionary to
enable researchers to try different predictive models. For this research, over 300,000 course
records were collected and aligned across institutions.
In addition, online classroom behavior data from UMUC’s online LMS were added to the
database for analysis of student behavior in the online classroom. Classroom behavior is defined
by over 30 categories of actions taken in the LMS by a student. Examples of typical actions are:
login time, access to various modules within the classroom, and posting of or responding to a
conference note. Each action that a student made in the classroom was totaled for each day.
Daily actions were aggregated by week, enabling researchers to analyze student activity in a
class as it progressed over time. For this research, over 3 million rows of data were available for
data mining. Advising data from UMUC’s customer relationship management system (CRM),
Goldmine, were also added to the database for future analyses.
The KDM served as the primary resource for all the analyses and findings for this research grant.
A model of the data included in the KDM is presented in Figure 2.
28
Figure 2. The Kresge Data Mart
After reviewing the initial findings from Phases 1 and 2, the community colleges, in
collaboration with UMUC, agreed to provide additional data elements that would enrich the
research and analysis. As a result, the data in the database were enhanced, resulting in the second
iteration of the Kresge Data Mart (KDM2). These additional data included: students’ completion
of developmental education, whether or not students received financial aid, and students’
ACCUPlacer scores.
A total of 493 source and derived variables were analyzed for inclusion in the dataset. Over 300
variables were tested as part of data mining analyses. Forty key variables were examined in
predictive modeling.
All data are stored on secured servers and have restricted access for the Institutional Research
office, researchers doing analysis on student success, and developers working on the database.
29
SECTION 5: SURVIVAL ANALYSIS: REGISTRATION AND WITHDRAWAL IN THE
ONLINE CLASSROOM
Using social network analysis, Dawson (2010) found that visualizing classroom interaction
patterns could provide insights into the nature of interactions for high- versus low-achieving
students completing an online course. Dawson (2010) determined that high-performing students
primarily interacted with other high-performing students, and likewise, low-performing students
were more likely to have interactions with other low-performing students. More importantly, in
examining instructor-student interactions, instructors networked with high-performing students
(81.7%) at significantly higher rates than they did with low-performing students (34.61%).
Social connections in online learning may result in cognitive and learning gains as well. Rovai
(2002) found a correlation between levels of engagement in the classroom community and
increased levels of content learning and understanding; this was especially true for females.
When this type of social and academic engagement is not present, students may withdraw from
online learning.
Additionally, students’ academic withdrawal was analyzed using survival analysis. Analyses
were run on a dataset for this study containing 19,190 undergraduate UMUC students in OL1
(Online Session 1) in Fall 2011 in 278 distinct courses
An exploratory survival analysis was carried out using a Kaplan-Meier estimator. Survival
analysis is a statistical technique that can be used to model ―time-to-event‖ data. In this case,
this analysis examines the time it took for a student to withdraw from a particular course (in
weeks and days) reflected as a time-to-event. Survival analysis generates a table that indicates a
hazard (or withdrawal) rate during the semester. Table 2 presents the withdrawal rate for new
and returning students by day. Figure 3 presents the hazard function for withdrawal rates of new
and returning students.
Table 2. Withdrawal rate for new and returning students by day
Number of
Cumulative
Number of
Week
Student
Proportion of
students
Withdrawals
Withdrawal
1
19,190
407
0.98
2
18,783
284
0.96
3
18,499
251
0.95
4
18,248
217
0.94
5
18,031
295
0.92
6
17,736
132
0.92
Withdrawal rate
0.0031
0.0022
0.002
0.0017
0.0024
0.0011
30
Figure 3. Hazard function of withdrawal rates for new and returning students.
Students withdraw at a higher rate in Week 1 compared to any other week in the course session,
with the exception of Week 5, which is the academic withdrawal deadline. Student status, new or
returning, may significantly affect student withdrawal rate. New students withdraw at a higher
rate than returning students. These findings suggest that interventions targeting new students
with interventions in Week 1 may be appropriate.
At the conclusion of Phase 1, three goals for the completion of the grant were identified and
completed in Phase 2:
1. Validate the predictive models and data mining techniques explored in Phase 1 on an
expanded dataset.
2. Build profiles of successful students and their online learning behaviors.
3. Develop interventions to improve the success of students transferring from community
colleges to UMUC.
31
SECTION 6: PROFILES OF STUDENTS USING DATA MINING
Data mining models were used to examine community college transfer students’ performance at
UMUC. Data mining focused on exploratory analyses identifying potential predictors of
students’ success and retention at UMUC. The following questions were considered.
1. Which profiles of students at UMUC can be identified?
2. To what extent does community college course taking differentiate each success profile at
UMUC?
Data exploration was initially performed by using IBM Modeler, SPSS, SAS JMP 10 Pro, and
Excel. Data were transformed and new variables were created as needed. Transformations were
performed in Modeler, JMP, and Excel. A variety of black box algorithms were used to develop
profiles of students’ success. The black box algorithms employed were Neural Nets, Boosted
Trees, and Random Forests.
Profiles of Student Success
In addition to independently considering these two outcomes of student success – UMUC GPA
and retention at UMUC – researchers also examined these two predictors jointly. Thus, profiles
of student success at UMUC were determined that classified students based on successful GPA
and retention. All combinations of the two attributes were examined. Four quadrants were
formed with students evidencing a high or low GPA, and being retained or not. These four
Success Quadrants were named Stars, Strivers, Slippers, and Splitters. (See Figure 4.)
Figure 4. Success Quadrants
32
Success Quadrant
Stars
Strivers
Slippers
Splitters
Percent of population
59%
17%
15%
9%
Students in Top Courses
62%
16%
13%
9%
Examining the community college course taking behaviors of students belonging to each of these
four success profiles yielded a number of key conclusions:




Transfer students who took accounting, economics, or higher-level math classes in
community college were more likely to earn a first-semester GPA of 2.0 or above at
UMUC (i.e., classified as Stars or Splitters).
Students who took more classes in history, sociology, psychology, and similar social
sciences were more likely to earn a GPA of less than 2.0 at UMUC.
The two low-GPA groups, Strivers and Slippers were less likely to take courses in subject
areas that Stars took.
The Splitters, the smallest group, did not show a distinct pattern of course taking
behavior.
In likelihood analyses (i.e., comparing the general proportion of course enrollment to actual
enrollment for each cluster), Strivers and Slippers showed nearly identical preferences. The
average numbers of classes students took and passed in each subject area were compared for
Strivers and Slippers. On average, Slippers passed fewer classes in all of the subject areas
preferred by Stars than did Strivers.
In addition, Slippers are noticeably less likely than Strivers to take courses in several areas:




Developmental English
Business/management
Sociology
Psychology
The four student success profiles, Stars, Strivers, Slippers, and Splitters, provided a useful
framework for understanding students at UMUC and introduced a new outcome measure that
combined performance (first-term GPA) and retention (retention within a 12-month window).
Factors from the students’ academic profile at the community college, such as course taking
behavior, course load, and change in GPA between the community college and UMUC, were
predictive of which student success profile the student would fall in. These results suggest that
student preparedness, particularly in specific areas (e.g., accounting, economics) is important in
attaining success at UMUC. More exploration of additional outcome variables, such as reenrollment and graduation, are planned for phase 3 of this project.
33
Further Findings from Data Mining
Using both predictive models and data mining techniques to understand predictors of student
success at UMUC, a number of conclusions may be drawn based on analyses undertaken in
Phase 2. These findings emerged from looking across studies and across student sub-populations
and through the use of varied statistical methods.
1. Student success. Overall, students transferring from MC and PGCC are successful at
UMUC. Indeed, 60% of transfer students were classified as Stars, indicating that they
were earning a GPA of 2.0 or above in the first term at UMUC and re-enrolling in a
subsequent term. Data indicate that earning high grades at the community college
was an indicator of successful performance at UMUC.
2. Online Classroom Behavior. Patterns in students’ behaviors in the online classroom
have some value in predicting success. In the analysis of online classroom data,
students varied greatly in the extent to which they engaged in course content and
course-related activities, with a substantial percentage of students not accessing the
course or materials at all. Results from this research have indicated that online
classroom activity is tied to course success. Though demographic factors and factors
in students’ community college course-taking backgrounds were predictive of success
at UMUC and of students’ behaviors in the online classroom, more robust data are
needed to more fully understand the relationship between academic behaviors and
student success.
3. Change in GPA. A new factor, the change GPA between the community college and
the first-term GPA at UMUC, was introduced. Many students experienced a decrease
in GPA when transferring to UMUC; however, the magnitude of this decrease has
predictive value in determine whether or not students are retained at UMUC. More
research is needed to better understand the tradeoff between the difficulty of course
work and a higher GPA to help determine what strategies community colleges may
employ to better prepare students for their academic transition.
4. Transitional Period. Transferring from community college to a four-year institution
is a particularly challenging transition for students. For one, students’ GPAs tend to
suffer during the first semester at the four-year institution. The magnitude of the
change in GPA seems to have an effect on students’ retention, differentiating the
Strivers and Slippers. For another, indicators of students’ preparation, such as course
efficiency and subject areas, were predictors of success at UMUC. This suggests that
students need to prepare for the rigor of UMUC course work. Finally, the number of
credits students earned prior to transfer may serve as an indicator of students’
preparedness to pursue their study at UMUC.
34
SECTION 7: PREDICTIVE MODELING OF STUDENT SUCCESS
Initial Predictive Modeling
Based on exploratory analyses using data mining, predictive modeling, including cluster analyses
and logistic regression, were used to model student success using demographic and community
college course taking variables. The following questions were considered.
1. What are the demographic profiles of community college students transferring from
MC and PGCC to UMUC?
2. Which factors from students’ demographic profiles and course-taking backgrounds in
community college predict success at UMUC overall, and in specific courses?
3. What kinds of online learning behaviors do students transferring to UMUC engage
in?
In addition to considering variables used in data mining, a number of possible predictors of
success not previously considered were included. For example, students’ course efficiency in
community college (the ratio of credits completed to credits attempted) and change in GPA (the
difference between students’ community college and UMUC GPA) were used as predictors.
Building upon findings from data mining and particularly exploratory analyses of community
college course taking behaviors suggesting that students’ course taking behaviors at the
community college may predict performance at the transfer institution, demographic factors and
variables in students’ community college course-taking backgrounds were examined as
predictive of success at UMUC.
Predicting Successful GPA
Independent Variables. Three types of independent variables were considered.
Specifically, these were students’ demographic characteristics, community college course
taking behaviors, and course efficiency.
Course efficiency was introduced as a summative measure of community college
students’ course taking that was thought to reflect the real-world cost, both in terms of
time and tuition, of students’ not completing courses as intended.
Dependent Variables. Across models run, a dichotomous outcome variable was used.
Students’ first-term GPA at UMUC was the target dependent variable, with a GPA of 2.0
or above being indicative of successful first-term GPA and a GPA below 2.0 being
indicative of an unsuccessful first-term GPA. First term referred to students’ first
semester of transfer at UMUC.
Logistic regression was used to determine which independent variables might be predictors of
success in terms of first-term GPA at UMUC. (See Table 3.) Demographic factors, primarily
age, marital status and race, were found to be significantly related to success at UMUC.
Specifically, older or married students were found to have higher GPAs at UMUC. When
35
compared to white students, students self-identifying as African American, Hispanic, or with an
unspecified race/ethnicity tended to have a significantly lower GPA at UMUC.
Table 3. Results of Multivariate Logistic Regression Analysis of Success at UMUC (N=7615)
Variable
B
S.E.
Sig.
Exp(B)
Age
.268
.027
.000
1.308*
Gender
-.083
.060
.164
.920
Asian Ethnicity
-.055
.119
.643
.946
African American
-.876
.081
.000
.417*
Hispanic Ethnicity
-.380
.113
.001
.684*
Unspecified Race
-.470
.104
.000
.625*
Married
.422
.085
.000
1.525*
English Course Taken
-.187
.081
.021
.829*
Math Course Taken
.345
.072
.000
1.413*
Speech Course Taken
.078
.070
.269
1.081
Computer Course Taken
-.078
.063
.218
.925
Honors Course Taken
.467
.166
.005
1.594*
Remedial Course Taken
.029
.068
.674
1.029
Online Course Taken
-.175
.059
.003
.839*
Course Efficiency
.241
.012
.000
1.273*
Note: White was used as reference category for race/ethnicity variables thus not used in the logistic regression
model.
*Statistically significant
Table 3 also shows that prior coursework was related to success at UMUC. Math courses and
honors courses were related to success at UMUC, while online courses at the community college
level were inversely related to success. Finally, course efficiency at the community college was
found to be a significant predictor of success at UMUC.
Predicting Re-enrollment
Independent variables. A number of independent variables were used in these analyses:
students’ community college GPA, race/ethnicity, gender, and age were used as control
variables. Then, the predictor of interest, delta GPA, was entered into the model.
Dependent Variables. The outcome of interest in these analyses was retention at
UMUC, defined as a student’s enrollment in a course at UMUC within 1-year of the
entering semester. A binary coding (0 or 1) was used depending on whether or not
student was retained.
Overall, the majority of transfer students (76.35%) were retained at UMUC. After controlling
for demographic factors and community college GPA, students’ change in GPA upon
transferring to UMUC was nonetheless a significant predictor of retention. (See Table 4.)
36
Table 4. Results of Multivariate Logistic Regression Analysis of Retention at UMUC (N=12637)
Variable
B
S.E.
Sig.
Exp(B)
Age
-0.13
.002
0.00
1.19*
Gender
0.15
0.04
0.00
1.17*
Hispanic
0.19
0.08
0.03
1.21*
African American
0.34
0.05
0.00
1.40*
Asian
0.43
0.09
0.00
1.54*
Race/Ethnicity
0.17
0.07
0.02
1.19*
Unknown
Community College
0.65
0.02
0.00
1.91*
GPA
Delta GPA
0.64
0.02
0.00
1.89*
Note: Excluded from the model were students classified as Non-resident alien, American Indian, Hawaiian/Pacific
Islander, or Two or more ethnicities, as these were not significant predictors in the model.
*Statistically significant
1. Demographics. In various analyses, students’ age and marital status were repeatedly
found to be predictors of success at UMUC. Older, married students tended to earn
higher GPAs and be retained. These findings may be indicative of students’ greater
maturity or dedication to their education goals. At the same time, minority status (i.e.,
African American or Hispanic) was associated with lower performance at UMUC. More
investigation needs to be done to determine how best to reach these underserved
populations and improve success.
2. Community College Courses. Course efficiency in community college was determined
to be a predictor of success at UMUC. The higher the ratio, the more likely the student
will succeed. Similarly, students who took math or honors courses were more likely to
succeed. These results point to the importance of considering not only quantitative
measures of students’ course work (e.g., course load) but also qualitative aspects of
students’ work (e.g., honors and math).
Updated Predictive Modeling
Expanding on initial predictive modeling, models were enhanced to incorporate new data,
introduced as part of a second wave of data sharing with the community colleges. Predictive
models were further developed and validated in updated predictive modeling. Specifically,
models were constructed predicting key milestones in students’ successful completion of a fouryear institution after transferring from a community college. These were:
1)
2)
3)
4)
Earning a successful first-term GPA
Re-enrollment in the next semester after transfer
Retention within a 12-month window following
Graduation with a 4-year, 6-year, and 8-year timeframe.
37
Each of the final predictive models is presented in turn. In developing predictive models, over
35 demographic, community college course taking behavior, community college performance,
and UMUC first-term variables were examined as potentially offering predictive power. These
are listed in Table 5.
Table 5. Variables considered in predictive modeling
Type of Predictor
Listing of Variables Examined
Demographic Characteristics Age
Gender
Race/Ethnicity
Marital Status
Receiving a PELL Grant at the CC
Community College Course
Math Enrollment
Successful Speech Completion
Taking
English Enrollment
Successful Computer-related Crs
Speech Enrollment
Completion
Honors Enrollment
Dev Writing Completion
Dev Education Enrollment
Dev Reading Completion
Enrollment in an Online Course
Dev Math Completion
Successful Course Completion
Exempt from Dev English
Successful Math Completion
Exempt from Dev Math
Successful English Completion
Repeating a Course
Community College
Community College GPA
Summative Metrics
CC Credits Earned
CC Credits Attempted
Percentage of Courses Withdrawn From
Receiving an Associate Degree
UMUC First-term Metrics
UMUC First-term GPA
UMUC First-term Enrollment Full-Time/Part-Time
UMUC First-term Credits Attempted
UMUC First-term Credits Earned
UMUC First-term Credits Transferred
38
Models were constructed to maximize statistical fit while being as parsimonious as possible (i.e.,
including as few predictors as possible). Model fit was determined by examining the percentage
of variance in the outcome variable explained by predictors in the model as well as by
considering accuracy of classification (e.g., categorizing students as graduating or not).
In predictive models, hierarchical logistic regression was used. Logistic regression predicts the
probability of a dichotomous outcome being achieved. As such, all target outcome variables
were dichotomized – for example, students’ first-term GPA at UMUC was recoded as being
either successful (≥2.0) or not. In hierarchical regression, variables are entered as blocks or in
steps, so that variables entered in at a previous step are controlled for when additional predictors
are added to the model. Across models, order of entry was: demographic characteristics,
community college course taking, community college performance summative measures, and if
considered, first-term at UMUC indices.
Table 6 presents descriptives of each of the dichotomized outcome variables examined.
Table 6. Descriptives associated with each target outcome variable
Dependent Variable
Population Performance
Successful First-term GPA
76.3% earn a GPA ≥2.0 in their first semester
(n=6151)
Re-enrollment
66.7% of students re-enroll in a subsequent
semester (n=5376)
Retention
79.1% of students are retained within a 12-month
window (n=5376)
Graduation to Date (Spring 2014)
52.7% of students have graduated to date (n=5454)
Population
Predictive modeling was run on 8,058 transfer students from MC and PGCC. These were
students whose first semester of transfer to UMUC occurred between Spring 2005 and Spring
2012. Students enrolled in continuing education courses or earning a second bachelor’s degree
were excluded from these analyses. Demographic characteristics are presented in Table 7.
Table 7. Sample Demographic Characteristics (n=8058)
28.6 years old (SD=8.4)
Age
Female: 57.6% (n=4638)
Gender
Male: 41.2% (n=3323)
White: 24% (n=1956)
Race/Ethnicity
African American: 43.5% (n=3509)
Asian: 10.4% (n=839)
Hispanic/Latino: 10.2% (n=821)
American Indian: 0.9% (n=75)
Unspecified: 14.0%
Predictive models for earning a successful first-term GPA, re-enrollment, and retention were run.
39
Predicting Earning a Successful First-term GPA
Dependent variable. Successful first-term GPA was used as the dependent variable.
Independent variables. Three types of variables were used to predict successful first-term
GPA. These were: (a) students’ demographic characteristics, (b) community college course
taking behaviors, and (c) summative measures of community college performance.
Among the independent variables of students’ community college course taking behavior
examined, rate of successful course completion at the community college, both overall and in
specific subject areas was computed. Successful course completion was defined as the ratio
of courses students’ completed with a grade of C or above to the total number of courses in
which students were enrolled. In specific subject areas, successful course completion
referred to the ratio of courses in that subject area in which students earned a grade of C or
above to the total number of courses in that subject area.
While course taking behaviors focused on students’ specific academic experiences in
community college, summative measures (e.g., GPA) looked at students’ community college
careers, overall.
The model was overall significant, X2(21) = 756.43, p<0.001, correctly classifying 76.8% of
students as earning a successful first-term GPA or not. Cox and Snell’s R2 suggested that the
model explained 9.1% of variance in earning a first-term GPA, while Nagelkerke’s R2 suggested
that 13.7% of variance had been explained. (See Table 8.)
Race/ Ethnicity:
Compared to
White Students
Table 8. Predicting first-term GPA using demographic characteristics, community college
course taking behaviors, and summative measures of CC Background
β
SE(β)
Significance
β*
Demographic Characteristics
Gender*
0.12
0.06
0.043
1.13
Age**
0.01
0.00
0.001
1.01
Black***
-0.36
0.08
0.000
0.70
Hispanic/Latino
-0.10
0.11
0.367
0.91
Asian
-0.06
0.11
0.57
0.94
American Indian
-0.28
0.27
0.30
0.76
Race Not Specified*
-0.23
0.10
0.021
0.79
Marital Status**
0.25
0.08
0.001
1.29
PELL Grant Recipient***
-0.30
0.07
0.000
0.74
Community College Course Taking
Successful Course Completion
1.63
0.21
0.000
5.08
Overall***
Successful Math Completion**
0.20
0.06
0.004
1.22
Successful English Completion**
0.18
0.06
001
1.20
40
Developmental Math
0.27
Completion**
Developmental Writing
-0.08
Completion
Developmental Reading
-0.07
Completion
Developmental Math Exempt
-0.03
Developmental English Exempt
-0.11
Repeated Courses
-0.27
Summative Measure of CC Background
GPA***
0.22
Credits Earned
-0.001
Associates Received***
0.39
0.08
0.001
1.31
0.10
0.38
0.92
0.11
0.48
0.93
0.08
0.05
0.07
0.747
0.07
0.000
0.97
0.89
0.76
0.05
0.002
0.08
0.000
0.62
0.000
1.25
1.00
1.47
Note: *sig. at 0.05 level, ** sig. at 0.01 level, *** sig, at 0.001 level
In terms of demographic characteristics, gender, age, and marital status were all significant
predictors in the model. Specifically, students who were female, older, and married were
significantly more likely to earn a successful first-term GPA at UMUC. At the same time,
students’ reporting their race/ethnicity as African American or not designating a race/ethnicity
were less likely to earn a successful first-term GPA. Further, receiving a PELL grant at the
community college, as an indicator of financial need, decreased the likelihood of students
earning a successful first-term GPA.
In examining indicators associated with students’ community college course taking behaviors,
students’ overall rate of successful course completion and rate of successful math completion,
and successful English completion were all significant predictors in the model. Further,
students’ completion of developmental math was a significant predictor in the model.
Looking to summative measures of community college performance, cumulative GPA, credits
earned, and earning an Associate degree were all significant predictors.
As can be seen by examining the standardized beta, holding all else constant in the model,
students’ overall rates of successful course completion carry the most impact in increasing
students’ probability of earning a successful GPA. Standardized betas may be interpreted as, for
a 1 standard unit increase in successful course completion; students were 5.08 standard
deviations more likely to earn a successful first-term GPA.
Predicting Re-Enrollment
Dependent variable. Re-enrollment was defined as enrolling in a semester immediately
following the first term of transfer. Re-enrollment was binary coded as re-enrolled (1) or not
(0)
Independent variables. Four types of variables were used to predict students’ re-enrollment
and retention. These were: (a) students’ demographic characteristics, (b) community college
course taking behaviors, (c) summative measures of community college performance, and (d)
first-semester performance at UMUC. While course taking behaviors focused on students’
41
specific academic experiences in community college, summative measures (e.g., GPA)
looked at students’ community college careers, overall.
The overall model for re-enrollment was significant, X2(19) = 1063.24, p<.001. The model was
able to correctly classify 71.6% of students as re-enrolling or not. Pseudo R2 measures of effect
size ranged from an estimated 12.5% of variance in re-enrollment explained (Cox & Snell’s R2)
to 17.4% of variance (Nagelkerke’s R2) explained. (See Table 9.)
Race/ Ethnicity:
Compared to
White Students
Table 9. Predicting re-enrollment using demographic characteristics, community college course
taking behaviors, summative measures of CC backgrounds, and UMUC first-term indicators
β
SE(β)
Significance
β*
Demographic Characteristics
Gender***
0.20
0.05
0.000
1.22
Age
0.00
0.00
0.638
1.00
Black*
0.17
0.07
0.013
1.19
Hispanic/Latino
-0.02
0.10
0.83
0.98
Asian
0.07
0.10
0.492
1.07
American Indian
0.19
0.27
0.469
1.21
Race Not Specified*
0.05
0.09
0.60
1.05
Marital Status**
0.24
0.07
0.001
1.28
PELL Grant Recipient
0.13
0.07
0.065
1.14
Community College Course Taking
Repeated a Course**
0.17
0.06
0.005
1.19
Enrolled in a Developmental
0.21
0.06
0.001
1.23
Course***
Exempt from Developmental
0.22
0.08
0.004
1.25
Math**
Summative Measures of Community College Backgrounds
Community College GPA**
-0.11
0.04
0.005
0.89
Cumulative Credits Earned at CC
-0.00
0.00
0.208
1.00
Earned an Associate Degree
-0.13
0.07
0.059
0.88
First Term at UMUC
First-term GPA***
0.26
0.02
0.000
1.30
First-term Credits Earned***
0.14
0.01
0.000
1.14
Enrolled Full Time
-0.16
0.08
0.054
0.86
Cumulative Credits
0.01
0.00
0.000
1.01
Transferred***
Note: *sig. at 0.05 level, ** sig. at 0.01 level, *** sig, at 0.001 level
Examining demographic characteristics determined that gender and marital status were both
significant predictors in the model. Specifically, being female and married increased students’
probability of re-enrolling in a subsequent term at UMUC. Further, unlike with first-term GPA,
race/ethnicity designated as African American or unspecified were significantly positive
predictors of re-enrollment.
42
In examining students’ community college course taking behaviors, different predictors than
those found to be significant in predicting performance were identified. Specifically, students’
likelihood of re-enrollment increased if they either enrolled in a developmental course or were
exempt from developmental math at their community college. Surprisingly, repeating a course at
the community college was found to be a significant, positive predictor of re-enrollment; in other
words, re-taking a course in community college increased the likelihood that students’ would reenroll at UMUC. While this finding may appear to be counter-intuitive, it may reflect the fact
that students willing to retake courses may be more committed to achieving an academic
credential, despite challenges they may experience.
Summative measures of students’ community college background found only community college
GPA to be a significant predictor in the model. Further, despite being a positive predictor of
first-term GPA, community college GPA was a negative predictor of persistence or reenrollment. More work is needed to understand why this may be the case. In part, those
students earning a high GPA at community college, despite likewise earning a successful GPA at
UMUC, may be more averse to ―transfer shock‖ due to the new four-year context and associated
academic demands.
Looking at first-term UMUC indicators, as may be expected, first-term GPA and total number of
credits earned were significant predictors of re-enrollment. Further, the cumulative number of
credits transferred was a significant positive predictor in the model. Number of credits
transferred may reflect the pragmatic value of community college course work in helping
students’ meet four-year institutional academic requirements. Examining standardized beta
coefficients in the model reveals first-term GPA to be the strongest predictor of re-enrollment at
UMUC. Indeed, for every standard unit increase in UMUC GPA, the probability of reenrollment increases by 1.30 standard deviations.
Predicting Retention
Dependent variable. Retention was defined as re-enrolling within a 12-month window
following the first term of transfer. Retention was binary coded as students being retained
(1) or not (0)
Independent variables. Four types of variables were used to predict students’ re-enrollment
and retention. These were: (a) students’ demographic characteristics, (b) community college
course taking behaviors, (c) summative measures of community college performance, and (d)
first-semester performance at UMUC. While course taking behaviors focused on students’
specific academic experiences in community college, summative measures (e.g., GPA)
looked at students’ community college careers, overall.
The model was overall significant, X2(17) = 1271.59. 80.5% of cases were correctly
classified as retained or not. Effect size measures suggest that between 14.8%, according to
Cox and Snell’s R2, and 23.1%, according to Nagelkerke’s R2, of variance in retention was
explained by the model. (See Table 10.)
43
Table 10. Predicting retention using demographic characteristics, community college course
taking behaviors, summative measures of CC backgrounds, and UMUC first-term indicators
β
SE(β)
Significance
β*
.180
-.005
.231
.027
.017
-.206
.048
.246
.148
.063
.004
.081
.113
.115
.291
.104
.090
.084
.004
.170
.005
.810
.883
.480
.648
.006
.079
1.197
.995
1.259
1.028
1.017
.814
1.049
1.279
1.159
.223
.065
.001
1.249
.174
.084
.037
1.191
.075
.089
Summative Measures of Community College Backgrounds
Community College GPA**
-.127
.043
First Term at UMUC
First-term GPA***
.590
.022
First-term Credits
.160
.013
Attempted***
Enrolled Full Time***
-.716
.131
Credits Transferred First.013
.001
term***
.399
1.078
.003
.881
.000
1.803
.000
1.174
.000
.489
.000
1.013
Race/ Ethnicity:
Compared to
White Students
Demographic Characteristics
Gender***
Age at Transfer
Black**
Hispanic/Latino
Asian
American Indian
Race Not Specified*
Marital Status**
PELL Grant Recipient
Community College Course Taking
Repeated a Course**
Completed Developmental
Math*
Exempt from Developmental Math
Note: *sig. at 0.05 level, ** sig. at 0.01 level, *** sig, at 0.001 level
As with re-enrollment, a number of demographic characteristics proved to be significant. Again
these were gender and marital status, with married females being more likely to persist. African
Americans and those having an Unspecified race/ethnicity were both found to be positive
predictors of retention.
Examining students’ community college course taking determined that repeating a course, and
being exempt from or completing developmental math were all three positive predictors in the
model. The presence of math-related variables seems to suggest the important role of the math
subject area in determining students’ academic preparedness and predicting students’
persistence.
In examining summative measure of community college academic backgrounds, cumulative
GPA at the community college was again found to be a negative predictor in the model. As a
contrast, first-term GPA at UMUC was a significant and positive predictor of retention. The
total number of credits attempted in the first term, the dichotomized variables part-time or full
44
time status, and the number of credits transferred from the community college were all found to
be significant predictors of retention.
A number of additional factors as predictive of retention were considered. These included,
receiving an Associate degree at the community college, credits earned and attempted at the
community college, average community college course load. Further, community college course
taking behaviors were not significantly predictive of retention – these included enrollments in
Math, English, Computer, or Speech courses as well as enrollment in Honors, Developmental,
and Online courses. Successful course completion indices were also not found to be
significantly associated with retention at the transfer institution.
Predicting Graduation
Sample. While previous models predicting earning a successful first-term GPA, reenrollment, and retention were run on the full cohort of MC and PGCC transfer students
enrolled in their first semester at UMUC between Spring 2005 and Spring 2012 (n=8050),
the graduation model was run on a more limited sub-sample. As we were interested in
predicting students’ eight-year graduation rate, only data from cohorts enrolled from Spring
2005 – Spring 2006, reflecting six semesters of data, were used. The remaining student
cohorts were not examined as they do not yet have eight years since their first term of entry.
The graduation model was run on a reduced sample of 2040 students.
Dependent variable. Graduation was defined as earning a first credential from UMUC (i.e.,
Certificate, Associate, Bachelor’s) within an 8-year period, based on cohort of entry. The
eight-year graduation rate was chosen because the population in this study reflected nontraditional students, who may be part-time or stop-out for various personal, family, and
financial reasons; eight years provides a graduation window which gives students sufficient
time to earn a credential. Graduation was binary coded as students either graduating within 8
years (1) or not (0).
Independent variables. Four types of variables were used to predict students’ re-enrollment
and retention. These were: (a) students’ demographic characteristics, (b) community college
course taking behaviors, (c) summative measures of community college performance, and (d)
first-semester performance at UMUC. While course taking behaviors focused on students’
specific academic experiences in community college, summative measures (e.g., GPA)
looked at students’ community college careers, overall.
The model was overall significant, X2(17) = 1271.59. 69.6% of cases were correctly
classified as retained or not. Effect size measures suggest that between 20.0%, according to
Cox and Snell’s R2, and 26.7%, according to Nagelkerke’s R2, of variance in graduation was
explained by the model. Table 11 includes a model summary.
45
Table 11. Predicting graduation using demographic characteristics, community college
course-taking behaviors, measures of CC experience, and UMUC first-term indicators
β
SE(β)
Significance
Demographic Characteristics
Gender
.029
.106
.785
First_Term_Age***
-.023
.007
.000
Minority Status
-.169
.104
.104
Receiving PELL at CC
-.262
.167
.116
Community College Course Taking
Math Enrollment at CC*
.329
.135
.015
Percent Ws at CC
-.670
.381
.079
Summative Community College Measures
Receiving AA at CC
.127
.129
.325
CC CUM GPA*
.168
.081
.038
CC Credits Earned
.005
.003
.059
UMUC First-term Indicators
UMUC First-term GPA***
.482
.044
.000
UMUC First-term Credits Earned***
.021
.002
.000
β*
1.029
.977
.845
.770
1.390
.512
1.135
1.184
1.005
1.619
1.022
Note: *sig. at 0.05 level, ** sig. at 0.01 level, *** sig, at 0.001 level
Looking at individual predictors in the model, in terms of demographic traits, only first-term age
when transferring to UMUC was found to be significant. Further, first-term age was a negative
predictor, such that being younger increased students’ likelihood of graduating.
At the community college, enrolling in a course in the Math subject area was a significant
predictor of graduation. Although other math-related indices, including completing
developmental math and the rate of successful math course completion, were examined as
predictors, the dichotomized variable reflecting whether or not students had enrolled in a math
course proved to be a sufficient indicator predicting graduation.
In terms of summative community college course taking indicators, community college
cumulative GPA was a significant positive predictor.
First semester at UMUC indices were all significant and positive predictors. Specifically,
students’ GPA in the first semester and the number of credits earned in their first term were
significant positive predictors. In examining standardized beta coefficients, first-term GPA was
the strongest predictor of eight-year graduation, followed by students having taken a math course
at the community college. Again, taking a math course could be interpreted as a variable
indicative of students’ academic preparation or of students’ willingness to complete requirements
necessary for graduation.
Summary of Results from Predictive Modeling
Looking across predictive models, Table 12 presents information regarding which indicators
were significant predictors across models.
46
Table 12. Significant Predictors for First-term GPA, Re-enrollment, Retention, and Graduation
Predictor
First-Term GPA Re-Enrollment
Retention
Graduation
Demographic Factors
Gender
First-term Age
Race/Ethnicity
+
+
Black (-)
Unspecified (-)
+
-
Marital Status
PELL Grant
Community College Course Taking
Overall Successful Course
Completion
Successful Math Completion
Successful English Completion
Repeated a Course
Enrolled in a Developmental Course
Exempt from Developmental Math
Completed Developmental Math
Enrolled in Math at CC
Community College Summative Measures
Community College GPA
Credits Earned
Associates Degree Received
UMUC First Term Factors
First-term GPA
First-term Credits Earned
Cum Credits Transferred
Enrolled Full Time
First-term Credits Attempted
+
+
-
Black (+)
Unspecified (+)
+
Black (+)
Unspecified (+)
+
+
+
+
+
+
+
+
-
+
+
+
+
+
+
+
+
+
N/A
N/A
N/A
N/A
N/A
+
+
+
Looking across the models helped identify a number of predictors which seem to be associated
with both performance (e.g., first-term GPA) and persistence (i.e., re-enrollment, retention, and
graduation). Females and those who were married were more likely to both earn a successful
first-term GPA, as well as to persist to graduation. Interestingly, African American status,
though negatively associated with performance in the first semester, was positively associated
with persistence. This may suggest that although minority status has typically been considered
to be an at-risk factor for students’ success (Greene, Marti, McClenney, 2008), some students
may benefit from the flexibility offered by an online institution.
Repeating a course was significantly associated with persistence measures – re-enrollment and
retention. This may indicate that while students struggle, being persistent in completing
necessary course work is associated with persistence. Further, a variety of factors related to
students’ math course taking behavior (e.g., being exempt from developmental math, completing
47
math at the community college, enrolling in math) were found to be associated with persistence
as well as graduation.
Examining students’ overall community college performance, community college GPA was
significantly associated with both performance and persistence. At UMUC, first-term GPA was
associated with re-enrollment, retention, and graduation. Likewise, the number of credits earned
in the first term was associated with re-enrollment and graduation, as they may reflect students’
commitment to educational goals and credential completion; likewise, being enrolled full-time
was associated with retention. Altogether, students’ volume of course taking in the first semester
(i.e., credits attempted, credits earned, full-time enrollment) were associated with persistence.
While this section presented results from predictive modeling for first-term GPA, re-enrollment,
retention, and graduation, the subsequent section delves into specific aspects of students’
experience at UMUC. Specifically, Section 8 examines (a) the relation between students’ online
classroom engagement and course performance as well as (b) students’ motivational and selfregulatory profiles.
48
SECTION 8: GRADUATION RATES
In addition to predicting graduation, graduation rates for students in our target cohort (i.e., whose
first semester of transfer to UMUC was between Spring 2005 to Spring 2012) were examined.
Students were divided into cohorts of entry depending on their first semester of enrollment at
UMUC. Cohorts were determined by fiscal year, including the Summer, Fall, and Spring terms
of a given year (e.g., Summer 2005, Fall 2005, and Spring 2006).
Graduation rates were examined for students earning a first-time bachelor’s degree from UMUC.
Term of graduation was likewise determined by fiscal year, and graduation rates for 1 to 8 year
periods were calculated. Not all starting cohorts at UMUC had been enrolled for a full eight
years; graduation rates were computed for as many years as students were at UMUC.
Rates of transfer were computed for students, overall, as well as separately for students coming
from each of the community colleges. Specifically, graduation rates were computed for students
transferring from Prince George’s Community College (n=3220) and from Montgomery College
(n=4724).
Table 13. Graduation rates for MC and PGCC transfer students, FY 06 – FY 12
FY Cohorts
Year
FY06
FY07
FY08
FY09
FY10
FY11
FY12
#
1017
1164
1138
1212
1333
1300
780
Graduation Rates by Subsequent Fiscal Year
Year 1
61
6%
59
5%
49
4%
80
7%
77
6%
97
7%
79 10%
Year 2
217 21%
189 16%
210 18%
240 20%
264 20%
288 22%
186 24%
Year 3
305 30%
293 25%
316 28%
378 31%
407 31%
416 32%
Year 4
378 37%
366 31%
386 34%
463 38%
478 36%
Year 5
424 42%
439 38%
428 38%
507 42%
Year 6
456 45%
481 41%
460 40%
Year 7
481 47%
506 43%
Notes: N=7944; Bachelor Graduates for the entire population to Spring 2014 = 3051 (38%)
Examining graduation rates determined that, overall, community college transfer students were
successful in earning a credential at UMUC. The eight-year graduation rate was 49%, while the
6-year graduation rates ranged from 44% to 40% of students in each cohort graduating. These
are impressive numbers compared to national rates as well as to UMUC overall rate.
Year 8
498 49%
49
SECTION 9: EXAMINING LEARNER BEHAVIOR IN THE ONLINE CLASSROOM
In addition to using predictive modeling to predict key academic outcomes, data mining was
used to examine the relation between students’ online classroom engagement and performance.
The online classroom activities come from the LMS. The LMS data came from a proprietary
classroom management system call WebTycho. WebTycho was replaced with Desire2Learn
(D2L) in 2014. However, for the population of students in this study, WebTycho provided the
data on student interactions in the classroom between Spring 2011 and Fall 2013.
Data mining techniques included Neural Nets, Boosted Trees, and Bootstrap Forest.
A model’s misclassification rate (the proportion of wrong predictions) was used to evaluate the
effectiveness of the models. For neural nets, R-squared levels were recorded for both the training
subset (on which the model was developed) and the validation subset (on which the model was
tested).
The analytical focus was on undergraduate students who had transferred to UMUC from
Montgomery College or Prince George’s Community College. The LMS dataset contained
approximately 2.3 million rows, each one representing a unique student/class/term/day
combination.
The data were examined at two levels: (a) course work and (b) student-level. While at the course
work level each student’s enrollment was treated as a unique record, such that a student could
have been listed in the course work file multiple times, the student level file ensured that there
was one record per student.
The dataset included only 8-week undergraduate courses only and provided week-by-week totals
of each action taken by students in the online classroom. Students’ online classroom activities
were then matched to records in the KDM.
There were a total of 30 different online classroom behaviors, 22 of these behaviors were not
well represented in the student data. The remaining eight online classroom behaviors were
evaluated to determine which had the most variability and seemed to be key indicators of student
engagement. Four online classroom behaviors were selected for examination:




Open classroom
Create a response note
Launch a conference
Read a conference
These actions also served to differentiate students’ course performance. Although 8-week
courses were examined, the scope of analysis was restricted to the first 3 weeks of a class, as
there was limited variation throughout the remainder of the course duration and prediction of
engagement-related factors early in the course facilitated the possibility of intervention.
50
The focus was further narrowed to include only student/class/term enrollments that earned a
grade of A, B, C, D or F. (See Figure 5.) Grades of AU, P, S (audit), FN (non-attendance),
Incomplete, and Withdraw were excluded.
Figure 5. Distribution of course grades
40%
35.1%
35%
32.0%
30%
25%
20%
17.6%
15%
9.5%
10%
5.8%
5%
0%
A
B
C
D
F
All models were run on the final dataset using four classroom activities during weeks 1–3 of an
8-week course where the final grade earned was A, B, C, D or F, and only for students who were
found in the KDM. The number of rows in this dataset is 28,021 and the number of unique
students is 4,277. Table 14 presents the averages for these unique student/course/session
combinations.
Table 14. Mean and median values for actions in the online classroom
Action
Open Class
Create Response Note
Launch Conferencing
Read Conference Note
Median
25
10
22
206
Mean
30.67
12.63
27.52
310.36
The distribution of the aggregate online classroom behaviors (compared to the median of all
students) across 8 weeks by grade received are presented in Figure 6.
Figure 6. Overall level of online classroom engagement for students by grade.
51
Level of Student Engagement by Grade
4
Engagement Index
3
2
1
A grades
0
B grades
-1
C grades
-2
D grades
-3
F grades
-4
1
2
3
4
5
6
7
8
Week
Four sets of findings will be discussed. First, the relation between students’ online classroom
behaviors, at the course level, and course performance was examined. Second, the association
between online classroom behaviors and course performance at the student level, across all of
their enrollments was analyzed. Third, the relation between an overall online classroom
engagement measure and performance was considered. Fourth, modeling the potential relation
between online classroom performance and re-enrollment was considered; however, did not
prove to be a fruitful avenue of investigation.
Online Classroom Behaviors and Class Performance
Because these LMS actions have uneven distributions, the median was chosen as a representation
of the average value rather than the mean. In each row, ―≥ med‖ flags were generated indicating
whether the values of the key LMS actions were at or above the median or below the median.
Figure 7 displays the grades distributions at the course level for students based on whether their
online classroom engagement, across the four actions, was above or below the median.
52
Figure 7. Distribution of grades above and below the median level of engagement
100%
90%
25%
80%
45%
A
B
C
D
F
70%
60%
31%
50%
40%
30%
20%
10%
33%
22%
8%
14%
15%
4%
5%
Below median
Above median
0%
As can be seen in the figures above, there are almost twice as many ―A‖ grades in the ―above
median‖ category as in the ―below median‖ category, and almost three times as many ―F‖ grades
in the ―below median‖ category as in the ―above median‖ category.
Higher counts of key LMS actions are associated with the higher grades. Similarly, the lowest
grades are typically found along with the lowest LMS counts. In order to meaningfully measure
how far above or below the median a LMS action value is, however, the values needed to be
indexed to a consistent scale to compensate for the differing ranges. A median difference index
(MAD) was created to capture the deviation between a student’s behaviors in the online
classroom and the median number of such behaviors manifest in the overall sample.
Both absolute values and MAD (i.e., median difference) indices were used in predictive models.
Model summary information is presented below; however, model fit was modest with a high
misclassification rate. (See Table 16.)
Table 16. Model fit information for predicting successful course completion.
Response
Variable
Course Grade
Course Grade
Predictors
(Weeks 1-3, Grades ABCDF)
Open Class (OC)
Create Response Note (CRN)
Launch Conferencing (LC)
Read Conference Note (RCN)
OC ≥ median
CRN ≥ median
LC ≥ median
RCN ≥ median
Model Type
Bootstrap
Forest
Neural net
Model Performance
Validation Set Results
R-squared: 0.164
Misclassification rate: 59.6%
R-squared: 0.126
Misclassification rate: 61.9%
53
Response
Variable
Course Grade
Course Grade
Course Grade
Course Grade
Course Grade
Course Grade
Course Grade
Predictors
(Weeks 1-3, Grades ABCDF)
OC difference-from-median index
CRN difference-from-median index
LC difference-from-median index
RCN difference-from-median index
Sum of median difference indexes
Model Type
Neural net
R-squared: 0.155
Misclassification rate: 59.7%
OC difference-from-median index
CRN difference-from-median index
LC difference-from-median index
RCN difference-from-median index
Open Class (OC)
Create Response Note (CRN)
Launch Conferencing (LC)
Read Conference Note (RCN)
Open Class (OC)
Create Response Note (CRN)
Launch Conferencing (LC)
Read Conference Note (RCN)
OC difference-from-median index
CRN difference-from-median index
LC difference-from-median index
RCN difference-from-median index
Course level ≥ 300
OC difference-from-median index
CRN difference-from-median index
LC difference-from-median index
RCN difference-from-median index
Subject area
Bootstrap
forest
R-squared: 0.251
Misclassification rate: 56.5%
Neural net
R-squared: 0.155
Misclassification rate: 60%
Bootstrap
Forest
R-squared: 0.162
Misclassification rate: 60%
Bootstrap
Forest
R-squared: 0.150
Misclassification rate: 59.9%
Bootstrap
Forest
R-squared: 0.140
Misclassification rate: 60.1%
Neural net
Model Performance
Validation Set Results
R-squared: 0.178
Misclassification rate: 59.5%
Student Level Online Classroom Behaviors and Course Performance
In the next set of analyses, the analysis shifted from class-level to student-level. The
student/course/term dataset (28,021 rows) was rolled up to yield 4,277 rows, each representing a
unique student.
In the rolled-up dataset, the four key online classroom actions were represented by the mean of
their values across that particular student’s entries in the previous dataset. Similarly, each
student’s grades from each class were averaged. A ―0/1‖ flag was created to indicate if the
average grades were ≥ 2.0. This value was used as the response variable for most of the
predictive models. Because this variable was categorical and not continuous, each model’s
accuracy calculations could also show misclassification rates.
The engagement calculations for this dataset followed the same procedure as the course level
calculations, except with student-level figures.
54
Different combinations of the variables were tested as predictors of student success. The primary
modeling methods used were Bootstrap Forest, Boosted Tree, and Neural Net. (See Table 17.)
Table 17. Data mining results for online classroom activities and course performance
Predictors (3-wk LMS rolled up +
addl )
Model
type
Results - validation set
Grades_Mean >=
2.0
OPENCLASS_Mean
CREATERESPONSENOTE_Mean
LAUNCHCONFERENCING_Mean
READCONFERENCENOTE_Mean
Bootstrap
forest
R-squared: 0.300
Misclassification rate: 0.196
Grades_Mean >=
2.0
OPENCLASS_Mean
CREATERESPONSENOTE_Mean
LAUNCHCONFERENCING_Mean
READCONFERENCENOTE_Mean
Boosted
tree
R-squared: 0.349
Misclassification rate: 0.209
Neural net
R-squared: 0.304
Misclassification rate: 0.210
Bootstrap
forest
R-squared: 0.296
Misclassification rate: 0.209
Boosted
tree
R-squared: 0.276
Misclassification rate: 0.224
Neural net
R-squared: 0.309
Misclassification rate: 0.199
Bootstrap
forest
R-squared: 0.347
Misclassification rate: 0.195
Boosted
tree
R-squared:
0.370
Misclassification rate:
0.199
Neural net
R-squared:
0.371
Misclassification rate:
0.188
Bootstrap
forest
R-squared: 0.098
Misclassification rate: 0.212
Response variable
Grades_Mean >=
2.0
Grades_Mean >=
2.0
Grades_Mean >=
2.0
Grades_Mean >=
2.0
Grades_Mean >=
2.0
Grades_Mean >=
2.0
Grades_Mean >=
2.0
Grades_Mean >=
2.0
OPENCLASS_Mean
CREATERESPONSENOTE_Mean
LAUNCHCONFERENCING_Mean
READCONFERENCENOTE_Mean
Difference index - OC
Difference index - CRN
Difference index - LC
Difference index - RCN
Difference index - OC
Difference index - CRN
Difference index - LC
Difference index - RCN
Difference index - OC
Difference index - CRN
Difference index - LC
Difference index - RCN
Difference index - OC
Difference index - CRN
Difference index - LC
Difference index - RCN
CC_GRADE_POINT_AVERAGE
Difference index - OC
Difference index - CRN
Difference index - LC
Difference index - RCN
CC_GRADE_POINT_AVERAGE
Difference index - OC
Difference index - CRN
Difference index - LC
Difference index - RCN
CC_GRADE_POINT_AVERAGE
Sum of difference indexes
55
Predictors (3-wk LMS rolled up +
addl )
Model
type
Results - validation set
Grades_Mean >=
2.0
Sum of difference indexes
Boosted
tree
R-squared: 0.301
Misclassification rate: 0.206
Grades_Mean >=
2.0
Sum of difference indexes
Neural net
R-squared: 0.192
Misclassification rate: 0.200
Grades_Mean >=
2.0
Sum of difference indexes
CC_GRADE_POINT_AVERAGE
Bootstrap
forest
R-squared: 0.325
Misclassification rate: 0.207
Grades_Mean >=
2.0
Sum of difference indexes
CC_GRADE_POINT_AVERAGE
Boosted
tree
R-squared: 0.290
Misclassification rate: 0.185
Grades_Mean >=
2.0
Sum of difference indexes
CC_GRADE_POINT_AVERAGE
Neural net
R-squared: 0.356
Misclassification rate: 0.189
Grades_Mean >=
2.0
OPENCLASS_Mean
CREATERESPONSENOTE_Mean
Bootstrap
forest
R-squared: 0.330
Misclassification rate: 0.209
Grades_Mean >=
2.0
Difference index - OC
Difference index - CRN
Bootstrap
forest
R-squared: 0.302
Misclassification rate: 0.208
Grades_Mean >=
2.0
Difference index - OC
Difference index - CRN
CC_GRADE_POINT_AVERAGE
Bootstrap
forest
R-squared: 0.332
Misclassification rate: 0.203
Bootstrap
forest
R-squared: 0.337
Misclassification rate: 0.190
Boosted
tree
R-squared: 0.313
Misclassification rate: 0.213
Neural net
R-squared: 0.354
Misclassification rate: 0.191
Response variable
Grades_Mean >=
2.0
Grades_Mean >=
2.0
Grades_Mean >=
2.0
OPENCLASS_Mean
CREATERESPONSENOTE_Mean
LAUNCHCONFERENCING_Mean
READCONFERENCENOTE_Mean
CC_GRADE_POINT_AVERAGE
OPENCLASS_Mean
CREATERESPONSENOTE_Mean
LAUNCHCONFERENCING_Mean
READCONFERENCENOTE_Mean
CC_GRADE_POINT_AVERAGE
OPENCLASS_Mean
CREATERESPONSENOTE_Mean
LAUNCHCONFERENCING_Mean
READCONFERENCENOTE_Mean
CC_GRADE_POINT_AVERAGE
These findings suggest that indexed measures of student engagement based on key online
classroom actions, along with a student’s community college GPA, can help predict whether or
not that student will achieve course success at UMUC.
56
Engagement Profiles and Course Performance
For this analysis, behaviors were collapsed to identify engagement profiles. ―High engagement‖
was defined as having the sum of median differences be greater than or equal to zero. ―Low
engagement‖ meant the sum of differences was below zero.
The engagement factor was combined with grades and the students were divided into four
―engagement quadrants‖ as follows:
E+G+
E+G–
E–G+
E–G–
High engagement; grade of A, B, or C
High engagement; grade of D or F
Low engagement; grade of A, B, or C
Low engagement; grade of D or F
Grades were proportionate to engagement. Nearly three-fourths of ―A‖ grades were associated
with high engagement while nearly three-fourths of ―F‖ grades were associated with low
engagement. However, engagement seemed a less useful discriminator for ―B‖ and ―C‖ grades.
These quadrants were very suggestive at a qualitative level but were not good predictors in
models, most likely because of the large number of low-engagement students who get an ―A,‖
―B,‖ or ―C‖ grade, or who are retained. Figure 8 displays students’ membership by grade in each
engagement profile.
Grade
A
B
C
D
F
High Engagement
72%
57%
43%
35%
27%
Low Engagement
28%
43%
57%
65%
73%
57
Figure 8. Student engagement by course performance.
Examining overall engagement, students earning high grades also demonstrated higher
engagement and vice versa.
Modeling Retention
Models examining LMS behaviors as associated with retention did not prove to be fruitful
predictors. This was likely because the majority of students were retained within a 12-month
window, limiting variance. (See Figure 9.)
Figure 9. Retention of community college transfer students.
58
Retention (1st year)
for unique ID/class/term rows
100%
80%
60%
40%
93.1%
20%
6.9%
0%
YES
NO
Key conclusions from data mining work examining online classroom data were as follows:
1. Four behaviors (i.e., open class, create response note, launch conference, read conference
note) were found to differentiate students’ course performance
2. Students’ improved course performance was associated with higher engagement
3. Students could be profiled based on their overall online classroom engagement and
performance
4. A model using means of four online classroom behaviors and community college GPA
was able to explain 31.3% - 33.7% of variance in course performance, overall
59
SECTION 10: STUDENT MOTIVATION AND SELF-REGULATION
In addition to examining online classroom engagement, transfer students’ motivational and selfregulatory characteristics were examined as they were associated with and course performance.
The goals of the study included:
1. Examine the motivation and self-regulation psychosocial profiles of community
college transfer students to UMUC.
2. Explore the relationship between psychosocial characteristics and students’ GPA at
UMUC.
3. Determine the extent to which psychosocial factors differ by socio-demographic
profiles, specifically, family structure and employment status.
Motivation may be defined as the cognitive and affective components driving students’ behavior
(Ames, 1992), whereas self-regulation is defined as the ―self-directed processes and self-beliefs
that enable learners to transform their mental abilities…into an academic performance skill‖
(Zimmerman, 2008, p. 166).
Population
The population consisted of 344 community college transfer students enrolled at UMUC in
Spring 2014. Participants were on average 39 years old with 45 % female (45%). The sample
was racially and ethnically diverse: 51% White, 27% African American, 7% Hispanic, and 7%
identifying as two or more races. The remaining participants’ did not report race/ethnicity.
Methodology
A survey was developed with three primary parts: (a) a motivation scale, (b) a self-regulation
scale, and (c) family structure and employment status questionnaire. The motivation and selfregulation scales were based on the Motivation and Self-Regulated Learning questionnaire
(MSLQ, Pintrich, 1991) and adapted for the online learning context (e.g., Barnard et al., 2009;
Levy, 2007).
Motivation scale. The motivation scale included three subscales: (a) a six-item locus of control
subscale (e.g., ―The good grades I receive are the direct result of my efforts,‖ Cronbach’s α =
0.50); (b) a two-item intrinsic motivation subscale (e.g., ―I prefer course materials that really
challenge me so that I can learn new things,‖ Cronbach’s α = 0.62); and (c) a five-item selfefficacy scale (e.g., ―I expect to do well in my classes,‖ Cronbach’s α = 0.84). Cronbach’s α is a
statistical measure of reliability. Overall, the reliability for the 13-item motivation scale was
0.80.
Self-Regulation scale. The self-regulation scale included six sub-scales, considered to be
particularly pertinent to online learning (Barnard et al., 2009). Specifically, it included: (a) five
items on goal-setting (e.g., ―I set standards for my assignments in online courses,‖ Cronbach’s α
= 0.83); (b) four items on environment structuring (e.g., ―I find a comfortable place to study,‖
Cronbach’s α = 0.85); (c) four items on task strategies (e.g., ―I prepare my comments before
60
joining in conferences and discussions,‖ Cronbach’s α = 0.85); (d) three items on time
management (e.g., ―I allocate extra study time for my online courses because I know they are
time demanding,‖ Cronbach’s α = 0.65); (e) three items on help-seeking (e.g., ―I am persistent in
getting help from the instructor through email,‖ Cronbach’s α = 0.69); and (f) four items on selfevaluation (e.g., ―I communicate with my classmates to find out how I am doing in my classes,‖
Cronbach’s α = 0.79). Overall, the reliability of the 23-item self-regulation scale was 0.92.
Socio-demographic factors. In the third part of the survey, participants were asked to report a
variety of factors associated with their family structure and employment status. Specifically,
participants reported whether they were single or married/in a domestic partnership and whether
they had children under 18 who lived with them. Further, participants reported whether they
were employed in Spring 2014, the average number of hours per week they worked, and the
financial sources they used to finance their education.
The survey was sent to a random sample of undergraduate students (N=2,690), enrolled at
UMUC during Spring 2014 semester, and who had previously transferred from a community
college. The survey had a 12.8% response rate. For those completing the survey, demographic
and performance data (i.e., cumulative GPA) were identified based on student records. No
statistically significant demographic differences were found between those students who
completed the survey and those who did not.
Results
Motivational and self-regulatory profile-. Table 18 includes participants mean scores on each
motivation and self-regulation sub-scale. Overall, students reported both moderately high levels
of motivation and self-regulation. Students had the highest scores on the self-efficacy sub-scale
on the motivation scale and the goal-setting subscale on the self-regulation scale.
Table 18. Motivation and Self-Regulation Scores by Subscale
Scale
Motivation
Locus of Control
Intrinsic Motivation
Self-Efficacy
Self-Regulation
Goal Setting
Environment
Management
Task Strategies
Time Management
Help Seeking
Self-Evaluation
Mean
Standard Deviation
3.86
3.50
4.04
4.22
3.76
4.30
4.26
0.48
0.55
0.71
0.59
0.57
0.62
0.66
3.43
3.86
3.30
3.16
0.76
0.78
0.89
0.83
Self-regulation and motivation associated with GPA- A series of Pearson’s correlations were
conducted. Although GPA was not significantly associated with motivation and self-regulation
61
overall, it was correlated with a variety of self-regulatory subscales: goal setting, task strategy
use, help-seeking, and self-evaluation.
Table 19 displays students’ motivational and self-regulatory profiles by whether they earned a
high GPA (i.e., 3.0 or above) or not. A GPA of 3.0 or above was selected as a cut off because
the majority of students responding to the survey had a high GPA.
Table 19. Motivation and Self-Regulation Scores by GPA
Scale
Motivation
Locus of Control
Intrinsic Motivation
Self-Efficacy
Self-Regulation
Goal Setting
Environ Mgmt
Task Strategies
Time Management
Help Seeking
Self-Evaluation
GPA 3.0 or ≥
(n=243)
3.87
3.50
4.06
4.24
3.75
4.35*
4.29
3.36*
3.86
3.25*
3.10
GPA < 3.0
(n=101)
3.83
3.50
4.06
4.16
3.78
4.18*
4.16
3.59*
3.86
3.42*
3.30
Motivation and self-regulation and socio-demographic profiles- Table 20 presents descriptive
information regarding students’ reported status in various socio-demographic categories.
Table 20. Descriptives for socio-demographic data
Employment Status: (Are you currently employed?)
Yes, Full
Yes, Part
No, I am
No, and I am not
I have served in the
Time
Time
seeking
seeking
military/been a
employment
employment
military spouse
67.7%
7.6%
7.8%
3.8%
7.6%
(n=233)
(n=26)
(n=27)
(n=13)
(n=26)
Avg. GPA
3.30
3.23
3.10
3.58
3.35
Payment Method: (Check all that apply)
I used
My work
I took out
I received
I paid for I used Military
scholarships
provided tuition
loans
financial aid
UMUC
Benefits or the
assistance
myself
GI Bill
18.6%
29.9%
27.0%
39.0%
42.4%
23.3%
(n=64)
(n=103)
(n=93)
(n=134)
(n=146)
(n=80)
Avg. GPA
3.49
3.28
3.20
3.29
3.34
3.32
Parental Status: (Check all that apply)
I have children under 18
I have children under 18
I have children
I have no
who live with me
who do not live with me
over 18
children
43.9%
5.2%
23.5%
33.4%
(n=151)
(n=18)
(n=81)
(n=115)
Avg. GPA
3.32
3.09
3.25
3.31
Independent sample t-tests determined that those students working full-time or in the military
reported significantly lower levels of goal setting; time management; and environmental
62
management than did those working part- time or not working. Further, those students not
working full-time had significantly higher self-regulation scores overall than did working
students. Table 21 presents students’ mean motivation and self-regulation by employment status.
Table 21. Motivation and Self-Regulation Scores by Employment Status
Scale
Motivation
Locus of Control
Intrinsic Motivation
Self-Efficacy
Self-Regulation
Goal Setting
Environment Management
Task Strategies
Time Management
Help Seeking
Self-Evaluation
GPA
Working Full Time
(n=259)
3.86
3.51
4.04
4.22
3.72*
4.27*
4.22*
Other
(n=66)
3.90
3.53
4.10
4.26
3.90*
4.45*
4.40*
3.39
3.82*
3.26
3.14
3.31
3.58
4.04*
3.45
3.26
3.25
Note: Employment status determined based on students’ responses to the question, ―Are you currently employed?‖
(Responses: Working full time & Military =1; Working part time, Not Employed & Seeking employment=0;)
Likewise, learners’ differed in self-regulatory profiles according to the way they paid for their
education. Those students reporting that they paid for UMUC themselves reported significantly
lower levels of time management; and self-evaluation, while having significantly higher GPAs.
Table 22 presents mean motivation and self-regulation levels for students by payment method.
Table 22. Motivation and Self-Regulation Scores by Payment Method
Scale
Motivation
Locus of Control
Intrinsic Motivation
Self-Efficacy
Self-Regulation
Goal Setting
Environment Management
Task Strategies
Time Management
Help Seeking
Self-Evaluation
GPA
Paid for UMUC Myself
(n=146)
3.88
3.51
3.99
4.28
3.72
4.33
4.29
Other
(n=198)
3.84
3.50
4.07
4.17
3.78
4.28
4.23
3.36
3.76*
3.31
3.05*
3.34*
3.48
3.94*
3.29
3.25*
3.19*
Note: Payment method determined based on a yes/no coding of students’ endorsement to the item, ―I paid for
UMUC myself‖ (No significant differences were found in students’ endorsements of, ―I used scholarships to pay for
UMUC‖; ―My work provided tuition assistance to help pay for UMUC‖; ―I took out loans to pay for UMUC‖; ―I
received financial aid to pay for UMUC‖ and ―I used military benefits or the GI Bill to pay for UMUC‖)
Those reporting having children had significantly higher intrinsic motivation than did students
reporting having no children. (See Table 23.)
63
Table 23.Motivation and Self-Regulation Scores by Parental Status
Scale
Motivation
Locus of Control
Intrinsic Motivation
Self-Efficacy
Self-Regulation
Goal Setting
Environment Management
Task Strategies
Time Management
Help Seeking
Self-Evaluation
GPA
Parents
(n=229)
3.87
3.50
4.10*
4.22
3.78
4.32
4.27
Non-Parents
(n=115)
3.84
3.52
3.92*
4.21
3.70
4.26
4.24
3.49
3.91
3.28
3.19
3.23
3.32
3.76
3.32
3.10
3.31
Note: Parental status determined based on a yes/no coding of students’ endorsement to the item, ―I have no
children‖ (No significant differences were found in students’ endorsements of, ―I have children under 18 who live
with me‖; ―I have children under 18 who do not live with me‖; ―I have children over 18‖)
No significant differences in motivation and self-regulation were found between students
married and not, although married students did have a significantly higher GPA.
Key Findings



Students’ GPA at UMUC was positively associated with goal setting.
There was a negative correlation between students’ GPA and task-strategy use, helpseeking, and self-evaluation. Students with lower academic abilities (i.e., lower GPAs)
may be more reliant on these self-regulatory approaches as a compensatory factor.
Psychosocial characteristics (i.e., motivation and self-regulation) differed based on
students’ socio-demographic profiles (i.e., employment status and payment method).
o Those students working full-time, as compared to not, reported significantly lower
levels of goal-setting, environmental management, and time management as well
as lower overall self-regulation.
o Those students paying for UMUC by themselves, at least partially, had
significantly higher GPAs while still reporting lower levels of time management
and self-evaluation.
o Those students reporting having children had significantly higher levels of
intrinsic motivation.
64
SECTION 11: INTERVENTION IMPLEMENTATION AND EVALUATION
Based on Kresge research, insights from the literature, and through discussion with community
college partners, a number of interventions were conceived, implemented, and evaluated.
Collectively these interventions aimed to offer students’ academic and social support through a
variety of mediums and targeted the unique issues faced by online, non-traditional learners.
Each intervention undertaken is briefly described below.
Student Resource Checklist- First-term community-college transfer students were randomly
assigned to a control (n=100) or test group (n=240). Students in the test group were sent a
Student Resource Checklist by their advisors. The goal of the checklist was to orient students to
the academic and social support resources available from the university, both online and face-toface. To complete the checklist, students had to use the university online resources to find
information about advisors, discipline-specific academic tutoring, writing assistance, and library
resources.
College Success Mentoring Program- First-term community college students, transferring from
MC and PGCC, were randomly assigned to a control (n=33) and test group (n=90). Students in
the test group were each paired with a peer mentor, who had transferred from their same
community college and had been successful at the UMUC. Mentors sent weekly emails to
mentees with advice and study tips and supported new students throughout their first semester.
JumpStart Summer- Jumpstart was a course offered free to all new students, intended to serve
as an orientation to online learning and to specifically address the needs of adult, career-oriented
students. As a part of the course, students completed academic diagnostic measures, developed
school- and career-related goals, a course plan, and were taught to use a variety of online tools,
including a course planner and resume-builder. In addition to examining the efficacy of the
Jumpstart course, we were interested in examining the effects of Jumpstart compared to and in
combination with mentoring. Students were randomly assigned to one of four conditions:
1. Control (n=44): Students received no intervention
2. Jumpstart (n=74): Students were assigned to complete the four-week Jumpstart course
3. Mentoring (n=74): Students were assigned a mentor for 8-weeks, parallel to the
College Success Mentoring Program
4. Jumpstart Summer (n=74): These were students were assigned to complete the fourweek JumpStart course as well as assigned a mentor for 8-weeks.
Accounting 220 and Accounting 221- An online tutoring intervention implemented in two
introductory accounting courses at UMUC, ACCT 220 and ACCT 221, during the Fall 2013
term. To support students’ success in these challenging courses, an online live tutoring program
was developed and offered by course instructors. Tutoring was offered during a three-month
period and students could choose to attend any number of sessions during that time.
A description and results from each intervention are presented, in turn. Across interventions,
three key outcome measures were considered.
65
Checklist
Participants
Participants in the Checklist intervention were identified through a data pull of students enrolled
at UMUC during the Spring semester of 2014; the data pull occurred in January 10th of 2014.
All Maryland community college transfer students, excluding those from MC and PGCC who
were participating in another intervention, enrolled in their first semester at UMUC were
isolated. From these, 241 were randomly selected to receive the Checklist and 103 were
randomly selected to serve as control participants.
Results
No significant differences in GPA and successful course completion were found between
students receiving the checklist and the control group. Table 24 compares the performance of
the test group (i.e., those receiving the checklist) and the control group.
Table 24. Average GPA and Successful Course Completion for Checklist Completers
Completed the
Test
Control
Received the
Checklist
Did Not Receive the
Checklist
(n=59)
Checklist
(n=240)
(n=103)
Term GPA
2.87
3.00
2.91
Successful Course Completion
73%
77%
77%
Re-Enrollment
67%
72%
67%
No significant differences in GPA and successful course completion were found between
students who completed the checklist and the control group.
Further, an evaluative survey was sent to all students receiving the checklist, both completing it
and not.
Of those who did complete the checklist, 42.37% responded (n=25), whereas only 4 students
(2.21%) who received the checklist but did not complete it, responded to the evaluation survey.
The overall response rate was 12.08%.
Of students responding, 85% of students reported that they would recommend completing the
checklist to other students.
Anecdotally, the goals of the checklist in familiarizing new UMUC transfer students with
resources and social support at UMUC proved to be successful. As one student explained, ―it
helped me compile information and learn how to use UMUC’s website.‖ This type of navigation
may be particularly important in helping to familiarize students with online resources at UMUC.
Another respondent reported, ―I had all my instructors emails listed on one sheet.‖ Indeed, a
goal of the checklist was to better connect students with both their instructors and advisors.
66
Finally, one student reported, ―it helped me get back into school after being out for 6 years,‖
getting at the ultimate goal of the checklist – to ease students’ transition to a four-year online
university.
College Success Mentoring
Participants
Participants in the study included mentor, mentee and control participants, all of whom had
transitioned to UMUC from MC or PGCC.
A total of 80 mentors and 761 control participants were included in the study. Selection and
recruitment of mentors is described in the Procedures section. In all but one case, mentors
retained for analysis were identified in the mentor-mentee matching phase of the program (n =
79). One additional mentor was added during remediation. The control participants were those
individuals who were recruited for the mentoring program, but were not selected as mentors.
A total of 90 mentees and 24 control participants were included in the study. Mentees were those
who received the mentoring treatment, while control participants were those that did
not. Selection and recruitment of mentees and control participants is described in the Procedures
section.
Results
No significant differences in GPA and successful course completion were found between
students in receiving mentoring and the control group. Table 25 compares the performance of
mentees in the test group and the control group, not receiving mentoring.
Table 25. Average GPA and Successful Course Completion for Mentoring Groups
Mentees
Test
Control
(n=90)
(n=34)
GPA
Successful Course Completion
Re-Enrollment
2.70
78%
74%
2.66
69%
75%
Table 26 compares the performance of mentors to a comparison group of students, eligible to
serve as mentors and invited to do so, who nonetheless elected not to participate. Although
students eligible to serve as mentors were overall successful, those who indeed served as mentors
had significantly higher cumulative GPA and rates of successful course completion. Further,
while term GPA, corresponding to the semester in which students served as mentors, did not
significantly differ across the test and control groups, mentors did have a 0.20 point higher GPA.
67
Table 26. Average GPA and Successful Course Completion for Mentor Test and Control Group
Mentors
Test
Control
Served as mentors
Invited but did not serve as
(n=70)
mentors
(n=117)
GPA
3.56
3.34
Successful Course Completion
95%
89%
Mentees- The mentoring evaluation survey was sent to both mentor and mentees. Among
mentees, 20% (n=18) responded to the survey. Of those responding, 82% reported that they
would recommend the mentoring program to other students.
Mentees reported receiving both academic and social support from their mentors. For example,
one mentee reported, ―They had previous experience with the format of UMUC classes; gave
insight to what [the classes] would be like.‖ This suggests that mentors offered support adjusting
to idiosyncratic aspects of UMUC’s courses, including content delivered online and an 8-week
compressed schedule. Mentees also connected with their mentors, ―She is very caring and very
down to earth. She made it very easy to communicate with her.‖ Indeed, part of the goal of the
mentoring program was to connect students with peer support at the transfer institution.
From an institutional perspective, the mentoring program supported students adjusting to UMUC
culture. As one student explained, ―He helped me the most in getting accustomed to the 8 week
sessions and how to set up my schedule throughout the week to be successful.‖ It is hoped that
specific skill building, like teaching mentees to set up a schedule, will support students’
performance in subsequent semesters at UMUC.
Finally, mentees discussed the importance of having role models of success who shared their
community college background. One student reported, ―Having someone that went through the
same process helped me get one step closer to my goal.‖ In this way, mentoring may have
promoted student success not only in the first semester but beyond.
Mentors- For the mentors, 48% (n=43) responded to the evaluation survey. Among themes
explored, was the benefit mentors gained from serving as role models and leaders. One student
explained, ―It put me in a responsible position. Not only did I have to help [him] succeed, I have
to [prove] to him that what I’m teaching him is working by passing myself.‖ This suggests an
intersection between the ways in which mentors and mentees viewed the program, as providing
students with role models of success. Another student further expanded, ―What I found to be
most valuable is my ability to learn more about myself as a leader and being able to improve my
communication skills.‖ As such, the mentoring program may have provided benefits to both
mentees and mentors in terms of skill development.
Serving as a mentor also served to increase mentors’ connections to UMUC. A mentor reported,
―Having the opportunity to give back to UMUC and have others learn from my experience,‖ as a
key benefit of the mentoring program. This type of institutional commitment may be difficult to
foster at online universities. Another student discussed the motivational benefits of helping their
68
peers, ―I like the idea of helping others. College is not always easy and the idea and act of
helping others is highly motivating.‖
Jumpstart Summer
Participants
All students transferring from community college to UMUC in Summer 2014 were targeted.
Students were randomly assigned to one of four groups:
a.
b.
c.
d.
Control (n=44)
Jumpstart (n=75)
Mentoring (n=75)
Jumpstart Summer (n=74)
The control group received no interventions. The Jumpstart group was assigned to take the fourweek Jumpstart onboarding course. The Mentoring group received 8-weeks of mentoring
through the College Success program. The Jumpstart Summer group both participated in the
four-week onboarding course and received eight weeks of mentoring.
Those students who did not want to take the Jumpstart course were allowed to opt-out of
participation; however, these students are still included in group comparison.
Results
Students’ average GPA and percentage of courses successfully completed will first be presented
across each of the four conditions. (See Table 27.)
Table 27. Comparing four conditions on GPA and successful course completion
Control
(n=44)
GPA
Successful Course Completion
2.46
75%
Mentoring
Program
(n=75)
2.16
74%
Jumpstart
(n=75)
2.13
64%
Jumpstart +
Mentoring
(n=74)
2.52
73%
A number of students elected to withdraw from the Jumpstart course. In the Jumpstart condition,
27 students withdrew (36.0%); in the Jumpstart Summer condition 22 students withdrew
(29.7%). Analyses were run excluding those students withdrawing from the Jumpstart course.
(See Table 28.)
Table 28. Comparing four conditions, excluding those students who dropped Jumpstart
GPA
Successful Course Completion
Control
(n=44)
Mentoring
Program
(n=75)
Jumpstart
(n=48)
Jumpstart +
Mentoring
(n=52)
2.46
75%
2.16
74%
2.23
59%
2.40
67%
69
Accounting 220 and Accounting 221
Participants
In two sessions of Fall 2013, 1,191 students enrolled in Accounting 220 or Accounting 221.
These students were divided into two groups:
Test Group: Students who participated in at least one live tutoring session.
Control Group: Students who did not attend any live tutoring sessions.
Sixty-seven students were placed into the test group because they attended the online tutoring
sessions and were registered for either ACCT 220 or ACCT 221. Sixteen students participated in
tutoring but were not matched with the course records for the two courses and were removed
from the analysis. The remainder of the students was placed in the control group. Demographics
of each group were examined. In addition, a standard T-test was conducted to determine if the
performance between the test and control groups were significantly different.
Results
Successful course completion, term GPA, change in GPA, and re-enrollment in the subsequent
term were compared as outcomes for the test and control groups. Change in GPA refers to the
difference between students’ GPA in the semester prior as compared to the GPA at the end of the
current semester. Table 29 provides results for both the test and control groups.
Table 29. Test and Control group performance on target outcome variables
Accounting 220 & Accounting 221
Successful Course Completion
Term GPA
Reenrollment
Change in GPA
Test
72%*
Control
58%*
2.52*
2.10*
78%
0.31
72%
0.07
*Indicates statistically significant differences between the test and control groups.
Key Findings




Students participating in tutoring (test group) had a significantly higher rate of successful
course completion when compared to those who did not participate (control group).
Students in the test group had a significantly higher term GPA than students in the
control group.
The re-enrollment rate of the test group was six percentage points higher than the control
group, but this was not statistically significant.
The change in cumulative GPA was .24 points higher for the test group than the control
group. While the difference in the change in GPA was not statistically significant, the test
group did demonstrate a greater increase in GPA than the control group.
70
SECTION 12: DISSEMINATION
A number of pathways have been taken to share results of the research and interventions
conducted for the PASS project. In particular, four types of initiatives were undertaken: (a)
presentations at conferences, (b) publications, (c) the Learner Analytics Summit, and (d)
development of the Student Success Calculator.
Presentations at Conferences
An ambitious conference schedule was adopted to disseminate findings of the research grant as
well as results based on the Kresge Grant overall. A summary of the presentations is included in
Table 30.
Publications
A number of publications are in-process or planned. The abstract of two of the manuscripts are
presented below.
1. Bridging the Great Divide: Examining Predictors of First-Term GPA for
Community College Students Transferring to a Four-Year Online University
While a variety of individual factors (e.g., age, gender) have been considered in predicting firstterm university GPA of community college transfer students, little has been done to consider
how students’ community college backgrounds may impact post-transfer success. In part,
community college factors, beyond GPA, have been neglected in the research literature due to
limitations in available data tracking students’ progress from community college to university.
In the present study, students’ demographic characteristics and community college course taking
behaviors (e.g., enrollment in math courses) are examined as predictive of first-term university
GPA. Further, a new variable, course efficiency, or the ratio of credits earned to credits
attempted, is introduced as a summative index of community college course taking and as
predictive of first-term university GPA.
2. Predictors of Retention for Community College Students Transferring to a FourYear Online University
This paper takes a longitudinal approach to modeling students’ continued educational enrollment
from community college to a four-year university. While much work has examined models
predicting transfer students’ retention at a four-year university, limited work has considered how
factors in students’ community college backgrounds may impact their retention upon transfer.
The present study seeks to inform these gaps by using demographic factors and community
college course taking behaviors (e.g., enrollment in math courses) to predict retention at a fouryear university. Further, course efficiency, or the ratio of credits earned to credits attempted, is
included as a summative index of students’ community college course taking behaviors and as
predictive of students’ retention at a four-year university.
71
Learner Analytics Summit
UMUC hosted a two-day convening to bring together leaders and practitioners of data analytics
to discuss issues facing both two-year and four-year institutions.
Session topics featured as part of the Learner Analytics Summit include:





A Review of the Data Analytics Toolkit
The Rise of Learner Success Scientists
Approaches to Predicting College Student Success
Using Analytics to Support Organization Change
Developing Institutional Capacity to Support Learner Analytics
As part of the Learner Analytic Summit, findings from the collaborations undertaken as a part of
the Kresge grant were presented. An abstract of the presentation is below:
This presentation will feature a panel discussion from administrators and researchers at UMUC
and partnering community colleges, MC and PGCC. The presentation will focus on three aspects
of the Kresge partnership: data sharing, research, and intervention development. Specifically,
the development of the memorandum of understanding and the Kresge Data Mart will be
discussed as will key research findings regarding the associations between students’ community
college course taking behaviors and performance at a four-year institution. Finally, findings
from interventions undertaken at UMUC and at the two partnering community colleges,
undertaken to promote transfer student success will be introduced. The presentation will
conclude with a description of the value added for each institution with time for questions and
panel discussion.
Table 30. Summary of Conference Presentations Delivered
Conference
Description
2013
AACRAO
Association for
Institutional
Research
WCET
AACRAO’s
Technology &
Transfer Conference
Multi-Institutional Data Predicting Transfer Student Success
A multi-institutional data base was developed to track the progress and success of
students who transferred from a community college to a 4-year institution. The study
identified risk factors through data mining.
Integrating Multi-Institutional Data for Predicting Student Success
Integrating multi-institutional data using detailed variable examination, data mining,
and statistical modeling predict student success and develop actionable interventions.
Using Learner Analytics Across Institutions
Overview of Kresge grant, partnerships, integrated data, factors that predict course
success, first-term GPA and retention, success quadrants, likelihood of community
college subject choices, possible interventions.
Mining for Success: A community college and four-year joint project on student
success
Review the goals of the study, results to date and plans for the future. Research so far
has included survival analysis, predictive models and clustering algorithms. The
72
results of this research will help this collaborative team to identify student success
initiatives that will be piloted and evaluated in Fall 2013.
2014
AACRAO’s
Technology &
Transfer Conference
Interventions to Promote Community College Transfer Student Success at a
Four-Year, Online University
Presentation on the effectiveness of two interventions aimed at promoting community
college students’ success when transferring to a four-year primarily online university
(i.e., Checklist, Mentoring).
SLOAN-C
Blended Interventions to Aid Transfer Students’ Transitioning from Face-ToBlended Learning Face to Online Courses
Conference
Presentation on the effectiveness of four interventions, delivered through various
mediums, in helping community college students transition to a four-year, online
university (i.e., Checklist, Mentoring, Jumpstart, CUSP)
Learning Analytics Cross Institutional Collaborations: Building Partnerships for Student Success
Summit
This panel presentation with MC and PGCC will have three purposes: 1) present key
goals for the Kresge partnership; 2) share research and intervention outcomes initiated
through the grant; 3) consider lessons learned and future directions for work on
promoting community college students’ success and persistence.
UPCEA MidProject Jumpstart: A Systemic Approach to Onboarding Adult Students
Atlantic
This presentation offers insights into the development and evaluation of Project
Jumpstart, an academic readiness course offered to new students at UMUC. Three
semesters of program implementation and improvements based on feedback from
administrators, teachers, and students are presented.
SLOAN-C
Jumpstart to Success: Creating a Personal Learning Plan to Improve Retention
International
and Success for Adult Students
Conference on
Development, implementation, and evaluation of JumpStart Mentoring program aimed
Online Learning
at improving on-boarding and promoting academic planning for community college
transfer students.
Examining the Relations Between Online Learning Classroom Behavior and
Student Success
We present descriptive and trend analysis of community college transfer students’
online classroom behaviors. The relation between patterns in online classroom
behaviors and course success and persistence will be examined.
Northeast
Community College Transfer Student Success at an Online University:
Association for
Conclusions from a Kresge Foundation Project
Institutional
This presentation introduces an overview of Kresge grant key goals, research and
Research
intervention initiatives, and future directions in promoting community college transfer
student success at a four year university.
Decision Sciences Online Live Tutoring Enhances Student Success
Institute
This research presents results from an evaluation of an online live tutoring
intervention implemented in two introductory accounting courses at a 4-year online
university.
73
Success Calculator
The Success Calculator was developed as an advising tool, based on predictive modeling of
student success. Two calculators were developed: 1) success in the first semester at UMUC, and
2) graduation from UMUC. The First Term Success Calculator uses students’ demographic data
and course taking behaviors to predict the probability of earning a GPA of 2.0 or above in their
first semester at UMUC. This calculator is intended to be used as an advising tool to support
students’ successful transition to UMUC.
The Graduation Calculator used models predicting the 8-year completion of community college
transfer student data. Predictors included demographic factors, community college factors, and
performance at UMUC in the first semester. An image of the Graduation Calculator is presented
in Figure 10.
Figure 10. Image of success calculator
UMUC Success Calculator
Student Information
Gender
Age At Transfer
Race/Ethnicity
PELL Grant Recipient
Female
25
Asian
No
Predicting Graduation
Math at CC
Percentage of Courses Withdrawn From
Received an Associated Degree
CC Cum GPA
CC Cum Credits Earned
First Term GPA at UMUC
UMUC First Term Credits Earned
Probability of Graduating in Eight-Year Period
No
30%
No
3.5
60
2.5
12
Calculate
55%
The calculator was developed in an Excel application. The community college partners
expressed an interest in piloting the calculator. An initial pilot was conducted with an advisor at
Prince George’s Community College. In order to better disseminate the calculator, a password
protected website has been developed to present the calculator for the community colleges to
use.
UMUC intends to adapt the calculator for each community college. In addition, the process for
the development of the calculator will be shared with 4-year and community colleges that are
interested in creating a similar collaboration.
74
SECTION 13: FINANCIAL SUPPORT
The Kresge Foundation awarded UMUC a $1.2 million grant to explore ways to improve student
success for transfer students by partnering with community colleges to track student progress.
The grant provided funding to build an integrated database, explore data mining techniques,
build predictive models of student success, implement and evaluate intervention strategies that
are designed to improve student success, and disseminate the results of this research to national
constituents.
In Phase 1 of the research study, approximately 41% of total grant funds were expended on
purchasing hardware and software for the development of the database, collecting data from the
community colleges, and hiring a data mining specialist and a graduate assistant. Additional staff
resources were provided in kind by UMUC. In Phase 2, funds were expended for additional data
collection, data mining consulting, and conference presentations. In the final stages of the grant,
expenses spent on collecting additional data from the community colleges, data mining
consultation, implementing interventions, and hiring an intervention coordinator. All tasks
within the grants were completed as planned. Any additional funds were used to support a
national convening on learner analytics.
75
SECTION 14: CONCLUSIONS
Work completed as part of Kresge Data Mining grant satisfied and exceeded the goals outlined
for the grant. Specifically, there were three stated goals for the grant:
1.
To build an integrated database tracking students across institutions, from
community college to UMUC.
2.
To use predictive statistical models and data mining techniques to track and model
students’ progress across institutions.
3.
To identify factors predictive of students’ success at UMUC that may inform the
development of interventions aimed to improve outcomes for undergraduate students
transferring from community colleges to UMUC or other four-year institutions.
To build an integrated database tracking students across institutions, from community
college to UMUC
Two iterations of the Kresge Data Mart (KDM) have been developed including data from the
community college partners as well as from UMUC’s student information system, customerrelationship management (CRM) advising system, and online classroom learning management
system (LMS). Two base extractions of data from the community colleges have been completed
and matched to UMUC students’ records.
To use predictive statistical models and data mining techniques to track and model
students’ progress across institutions
Predictive modeling was used to build models associated with key milestones in students’
academic trajectories including (a) earning a successful first-term GPA, (b) re-enrollment, (c)
retention, and (d) graduation. Across models demographic factors (gender, marital status), math
taking at the community college, CC GPA were all predictors of first-term GPA, re-enrollment,
retention, and graduation. First-term GPA at UMUC was also a significant predictor of reenrollment, retention, and graduation.
Data mining methods were used to identify patterns in students’ online classroom behaviors in
the LMS. Students’ were profiled based on course performance and level of engagement in the
LMS. Additionally, a predictive model using community college GPA and four online
classroom behaviors (i.e., opening a classroom, launching a conference, reading a conference
note, creating a response note) were found to predict successful course completion at the student
level. Data mining proved to be a fruitful technique for exploring the complexity of the variables
included in the KDM.
76
To identify factors predictive of students’ success at UMUC that may inform the
development of interventions aimed to improve outcomes for undergraduate students
transferring from community colleges to UMUC or other four-year institutions
Based on research, literature reviews, and collaborative partnerships, six interventions were
developed, implemented, and evaluated at UMUC and at the community colleges. Collectively,
the interventions targeted student academic achievement and social and institutional integration.
A number of interventions targeted community college students in the first-semester of transfer
to aid with the transition as well as acclimation to a four-year and online climate.
In addition, this research provided the opportunity to develop the Success Calculator, an
application tool predicting students’ probability of earning a successful first-term GPA at
UMUC. This tool represents a real-world extension of the research.
Sharing and disseminating research findings are being disseminated through conference
presentations and publications.
Future Directions
Future directions include expanding the research study to include other community colleges.
This would allow for the validation of models developed based on a larger and more diverse
sample and for the examination into how various predictors function across institutions. In
addition, the MC and PGCC are committed to continuing the data collection and analysis to get
feedback on how their students are performing at UMUC. This information has informed and
will continue to inform practices and policies at both the community colleges and at UMUC.
Future research directions include the following:

Evaluate long term effects of students participating in the interventions to determine if
the interventions influence retention or completion.

Evaluate the math performance of transfer students by facilitating meetings between
curriculum designers, program directors, and instructors to better align curriculum across
institutions and to ensure students’ academic preparedness.

Evaluate the accounting performance of transfer students by determine the extent to
which accounting courses across institutions are aligned and students are academically
prepared.

Evaluate the developmental math curriculum and transfer performance. Both MC and
PGCC are now offering modularized developmental math courses. UMUC will evaluate
the performance of these students and compare their performance with students who did
not receive the modularized math courses.

Share the UMUC Success Calculator with other institutions interested in developing
similar data sharing partnerships.
77
REFERENCES
Abrams, H. G., & Jernigan, L. P. (1984). Academic support services and the success of high-risk
college students. American Educational Research Journal, 21(2), 261-274.
Adelman, C. (2005). Moving into town-and moving on: The community college in the lives of
traditional-age students. Washington, DC: Office of Vocational and Adult Education, US
Department of Education.
Alfonso, M. (2006). The impact of community college attendance on baccalaureate
attainment. Research in Higher Education, 47(8), 873-903.
Ali, N. S., Hodson-Carlton, K., & Ryan, M. (2004). Students' perceptions of online learning:
implications for teaching. Nurse Educator, 29(3), 111-115.
Allen, T. D., Poteet, M. L., & Burroughs, S. M. (1997). The mentor’s perspective: A qualitative
inquiry. Journal of Vocational Behavior, 51, 70–89.
Altstadt, D., Schmidt, G., & Couturier, L. K. (2014). Driving the direction of transfer pathways
reform. Retrieved from:
http://www.jff.org/sites/default/files/publications/materials/Driving-the-DirectionPathways-Reform-042414.pdf
Ames, C. (1992). Classrooms: Goals, structures, and student motivation.Journal of educational
psychology, 84(3), 261.
Angelino, L. M., Williams, F.K., & Natvig, D. (2007) Strategies to engage online students and
reduce attrition. Journal of Educators Online, 4(2), 3-14.
Aragon, S.R. & Johnson, E.S. (2008). Factors influencing completion and non-completion in
online community college courses. American Journal of Online Education, 22(3), 146-158.
Armor, D. A., & Taylor, S. E. (1998). Situated optimism: Specific outcome expectancies and
self-regulation. Advances in experimental social psychology, 30, 309-379.
Bailey, T. R., Leinbach, D. T., & Jenkins, P. D. (2006). Is student success labeled institutional
failure? Student goals and graduation rates in the accountability debate at community
colleges.
Bailey, T., Jenkins, D., & Leinbach, T. (2005). Graduation Rates, Student Goals, and Measuring
Community College Effectiveness. CCRC Brief Number 28. Community College
Research Center, Columbia University.
Baker, R.S. & Yacef, K. (2009). The state of educational data mining: A review and future
visions. Journal of Educational Data Mining, 1(1), 3-17.
78
Baldwin, G. (1994). The student as customer: the discourse of ―quality‖ in higher
education. Journal for Higher Education Management, 9(2), 131-139.
Barnard, L., Lan, W. Y., To, Y. M., Paton, V. O., & Lai, S. L. (2009). Measuring self-regulation
in online and blended learning environments. The Internet and Higher Education, 12(1),
1-6.
Bean, J. P. & Metzner, B. S. (1985). A Conceptual Model of Nontraditional Undergraduate
Student Attrition. Review of Educational Research, 55(4) 485–540. JSTOR. Retrieved
from http://www.jstor.org/stable/1170245
Bean, J. P., & Metzner, B. S. (1985). A conceptual model of nontraditional undergraduate
student attrition. Review of educational Research, 55(4), 485-540.
Bean, J. P., & Metzner, B. S. (1985). A conceptual model of nontraditional undergraduate
student attrition. Review of educational Research, 55(4), 485-540.
Berger, J. B., & Malaney. G. D. (2003). Assessing the transition of transfer students from
community colleges to a university. NASPA Journal, 40(4), 1-23.
Bers, T. H., & Smith, K. E. (1991). Persistence of community college students: The influence of
student intent and academic and social integration. Research in Higher Education, 32(5),
539-556.
Best, G. A., & Gehring, D. D. (1993). The academic performance of community college transfer
students at a major state university in Kentucky. Community College Review, 21(2), 3241.
Boston, W., Diaz, S.R., Gibson, A.M., Ice, P., Richardson J., & Swan, K. (2011). An exploration
of the relationship between indicators of the community of inquiry framework and
retention in online programs. Journal of Asynchronous Learning Networks, 13(3), 67-83
Boyle, F., Kwon, J., Ross, C., & Simpson, O. (2010). Student-student mentoring for retention
and engagement in distance education. Open Learning, 25(2), 115-130.
Brown, K. (2001). Mentoring and the retention of newly qualified language teachers. Cambridge
Journal of Education, 31(1), 69–88.
Carlan, P. E., & Byxbe, F. R. (2000). Community colleges under the microscope: An analysis of
performance predictors for native and transfer students. Community College
Review, 28(2), 27-42.
Cejda, B. D., Kaylor, A. J., & Rewey, K. L. (1998). Transfer shock in an academic discipline:
The relationship between students' majors and their academic performance. Community
College Review, 26(3), 1-13.
79
Chen, P.S. D., Lambert, A. D. & Guidry, K. R. (2010). Engaging online learners: The impact of
Web-based learning technology on college student engagement. Computers & Education,
54(4), 1222–1232. Elsevier. http://dx.doi.org/10.1016/j.compedu.2009.11.008
Clinedinst, M. E., & Hawkins, D. A. (2010). 2010 State of college admission. Arlington, VA:
National Association for College Admissions Counseling.
Cohen, A. M., & Brawer, F. B. (1987). The collegiate function of community colleges. San
Francisco, CA: Jossey-Bass Inc., Publishers.
Cohen, A. M., & Brawer, F. B. (2003). The American community college (4th ed.). San
Francisco, CA: Jossey-Bass.
Conway, K. M. (2010). Educational aspirations in an urban community college: differences
between immigrant and native student groups. Community College Review, 37(3), 209242.
Dawson, S. (2010). ―Seeing‖ the learning community: An exploration of the development of a
resource for monitoring online student networking. British Journal of Educational
Technology, 41(5), 736–752. Wiley Online Library. http://dx.doi.org/10.1111/j.14678535.2009.00970.x
Deng, H. (2006). Are there differences between transfers from community college careeroriented programs and liberal arts programs? IR Applications: Using Advanced Tools,
Techniques, and Methodologies, 11, 1-11.
Diaz, P. E. (1992). Effects of transfer on academic performance of community college students
at the four-year institution. Community/Junior College Quarterly of Research and
Practice, 16(3), 279-291.
Dougherty, K. J. (1992). Community colleges and baccalaureate attainment. The Journal of
Higher Education, 188-214.
Dowd, A. C., & Coury, T. (2006). The effect of loans on the persistence and attainment of
community college economics students. Research in Higher Education, 47(1), 33-62.
Doyle, W. R. (2009). The effect of community college enrollment on bachelor's degree
completion. of Education Review, 28(2), 199-206.
Driscoll, A. K. (2007). Beyond access: How the first semester matters for community college
students aspirations and persistence. Policy brief 07–2. Retrieved from Policy Analysis
for California Education: http://www.edpolicyinca. org/sites/default/files/PB.
Duby, P., and Schartman, L. (1997). Credit hour loads at college onset and subsequent college
performance: A multi-institution pilot project. Paper presented for the Annual Forum of
the Association for Institutional Research, Orlando.
80
Durio, H.F., Helmick, C.K., & Slover, J.T. (1982). A comparison of aptitude and achievement
between transfer engineering students and students entering engineering as freshmen at a
major university. Educational Research Quarterly, 7, 42-50.
El Mansour, B., & Mupinga, D. M. (2007). Students' positive and negative experiences in hybrid
and online classes. College Student Journal, 41(1), 242-248.
Ely, E.E. (1997). The non-traditional student. Paper presented at the American Association of
Community Colleges annual conference, Anaheim, CA.
Ensher, E. A., Heun, C., & Blanchard, A. (2003). Online mentoring and computer-mediated
communication: New directions in research. Journal of Vocational Behavior, 63, 264-288.
Finnegan, C., Morris, L.V., and Lee, K. (2009). Differences by course discipline on student
behavior, persistence, and achievement in online courses of undergraduate general
education. Journal of College Student Retention, 10(1), 39-54.
Fisher, M., & Baird, D. E. (2005). Online learning design that fosters student support, selfregulation, and retention. Campus-Wide Information Systems, 22(2), 88-107.
Flaga, C. T. (2006). The process of transition for community college transfer students.
Community College Journal of Research and Practice, 30 3-19.
Gao, H., Hughes, W. W., O'Rear, M. R., & Fendley Jr, W. R. (2002). Developing structural
equation models to determine factors contributing to student graduation and retention:
are there differences for native students and transfers? Paper presented at the Forum of
the Association for Institutional Research, Toronto, Canada.
Glass, J. C. Jr., & Harrington, A.R. (2002). Academic performance of community college
transfer students and "native" students at a large state university. Community College
Journal of Research and Practice, 26(5), 415-430.
Goldrick-Rab, S. (2010). Challenges and opportunities for improving community college student
success. Review of Educational Research, 80(3), 437-469.
Good, J. M., Halprin, G. & Halprin, G. (2000). A promising prospect for minority retention:
students become peer mentors. The Journal of Negro Education, 29(4), 375-383.
Graham, S. W., & Hughes, J. C. (1994). Moving down the road: community college students’
academic performance at the university. Community College Journal of Research and
Practice, 18(5), 449-464.
Grubb, W. N. (1991). The decline of community college transfer rates: Evidence from national
longitudinal surveys. The Journal of Higher Education, 62(2), 194-222.
81
Han, J., & Kamber, M. (2001). Data mining: Concepts and techniques. China Machine Press, 8,
3-6.
Herzog, S. (2006). Estimating student retention and degree completion time: Decision trees and
neural networks vis a vis regression. New Directions for Institutional Research, 131, 17-33.
Hills, J. R. (1965). Transfer shock-the academic performance of the junior college transfer.
Journal of Experimental Education, 33, 201-215.
Ho Yu, C., DiGangi, S., Jannasch-Pennell, A., Lo, W., & Kaprolet, C. (2007, February). A datamining approach to differentiate predictors of retention. Paper presented at the Educause
Southwest Conference, Phoenix, AZ.
Hoachlander, G., Sikora, A. C., Horn, L., & Carroll, C. D. (2003). Community college
students. Education Statistics Quarterly, 5(2), 121-128.
Horn, L. J., & Carroll, C. D. (1996). Nontraditional undergraduates: Trends in enrollment from
1986 to 1992 and persistence and attainment among 1989-90 beginning postsecondary
students. US Government Printing Office, Superintendent of Documents, Mail Stop:
SSOP, Washington, DC 20402-9328.
Horn, L., Nevill, S., & Griffith, J. (2006). Profile of undergraduates in US postsecondary
education institutions, 2003-04: With a special analysis of community college students
(NCES 2006-184). U.S. Department of Education. Washington, DC: National Center for
Education Statistics.
Hoyt, J. E. (1999). Remedial education and student attrition. Community College Review, 27(2),
51- 72.
Hughes, J. A., & Graham, S. W. (1992). Academic performance and background characteristics
among community college transfer students. Community/Junior College Quarterly of
Research and Practice, 16(1), 35-46.
James Madison University. (1989). The academic performance of transfers from Virginia's
public two-year institutions at James Madison University. Harrisonburg, VA: James
Madison University Office of Planning and Analysis.
Johnson, N. T. (1987). Academic factors that affect transfer student persistence. Journal of
College Student Personnel, 28(4), 323-329.
Kane, T. J., & Rouse, C. E. (1999). The community college: educating students at the margin
between college and work. The Journal of Economic Perspectives, 13(1), 63-84.
Kearsley, G. (2000). Online education: Learning and teaching in cyberspace (Vol. 91). Belmont,
CA: Wadsworth.
82
Keegan, D. (1996). Foundations of distance education. Psychology Press.
Keeley, E. J., & House, J. D. (1993). Transfer shock revisited: A longitudinal study of transfer
academic performance. Paper presented at the annual forum of the Association for
Institutional Research, Chicago.
Kember, D. (1989). A longitudinal-process model of drop-out from distance education. The
Journal of Higher Education, 278–301. JSTOR. Retrieved from
http://www.jstor.org/stable/198225
Knoell, D. M., & Medsker, L. L. (1965). From junior to senior college: A national study of the
transfer student. Washington, D.C.: Community College Press, American Association of
Community Colleges.
Kuh, G. D. (2003). What we’re learning about student engagement from NSSE: Benchmarks for
effective educational practices. Change,35(2), 24-32.
Laanan, F. S. (1996). Making the Transition: Understanding the Adjustment Process of
Community College Transfer Students. Community College Review, 23(4), 69-84.
Laanan, F. S. (2000). Community college students' career and educational goals. New Directions
for Community Colleges, 2000(112), 19-33.
Laanan, F. S. (2001). Transfer student adjustment. In F. S. Laanan (Ed.), Transfer students:
Trends and issues. No. 114. New Directions for Community Colleges. San Francisco, CA:
Jossey-Bass.
Lenaburg, L., Aguirre, O., Goodchild, F., Kuhn, J. W. (2012). Expanding pathways, A bridge
program for community college STEM students. Community College Journal of Research
and Practice, 36(3), 153-168.
Levy, Y. (2007). Comparing dropouts and persistence in e-learning courses. Computers &
Education, 48(2), 185-204.
Lin, Y. G., McKeachie, W. J., & Kim, Y. C. (2001). College student intrinsic and/or extrinsic
motivation and learning. Learning and Individual Differences, 13(3), 251-258.
Luan, J, (2002). Data mining and its applications in higher education. New Directions for
Institutional Research, 113, 7-36.
Luan, J. (2001). Data mining as driven by knowledge management and higher education:
Persistence clustering and prediction. Paper presented at the SPSS Public Conference,
University of California at San Francisco.
Luan, J. & Zhao, C-M. (2006). Data mining: Going beyond traditional statistics. New Directions
for Institutional Research, 131, 7-16.
83
Metzner, B. S., & Bean, J. P. (1987). The estimation of a conceptual model of nontraditional
undergraduate student attrition. Research in Higher Education, 27(1), 15-38.
Miller, M. T., & Lu, M. Y. (2002). Barriers and challenges to serving non-traditional students in
e-learning environments. Retrieved from: http://files.eric.ed.gov/fulltext/ED468117.pdf
Miller, M., & Lu, M. Y. (2003). Serving non-traditional students in e-learning environments:
Building successful communities in the virtual campus. Educational Media
International, 40(1-2), 163-169.
Moneta, G. B., & Kekkonen-Moneta, S. S. (2007). Affective learning in online multimedia and
lecture versions of an introductory computing course. Educational Psychology, 27(1), 5174.
Montondon, L., & Eikner, A. E. (1997). Comparison of community college transfer students and
native students in an upper level accounting course. Community College Review, 25(3),
21-38.
Moore, J. C. & Fetzner, M. J. (2009). The road to retention: A closer look at institutions that
achieve high course completion rates. Journal of Asynchronous Learning Networks, 13(3),
3–22. Retrieved from http://sloanconsortium.org/jaln/v13n3/road-retention-closer-lookinstitutions-achieve-high-course-completion-rates
Morris, L.V. & Finnegan, C.L. (2009). Best practices in predicting and encouraging student
persistence and achievement online. Journal of College Student Retention, 10(1), 5-34.
Muilenburg, L. Y., & Berge, Z. L. (2005). Student barriers to online learning: a factor analytic
study. Distance education, 26(1), 29-48.
National Survey of Student Engagement (2008). Promoting engagement for all students: The
imperative to look within—2008 results. Washington, DC: Author.
Nickens, J. M. (1972). " Transfer shock" or "transfer ecstasy". Paper presented at the annual
meeting of the American Education Research Association, Chicago.
Nistor, N., & Neubauer. K (2010). From participation to dropout: Quantitative participation
patterns in online university courses. Computers in Education, 55, 663-672.
Nora, A., & Rendon, L. I. (1990). Determinants of predisposition to transfer among community
college students: a structural model. Research in Higher Education, 31(3), 235-255.
Okun, M. A., Weir, R. M., Richards, T. A., & Benin, M. H. (1990). Credit load as a moderator of
the intent-turnover relation among community college students. Journal of Experimental
Education, 58(3), 213-222.
84
Palmer, J. C., & Eaton, J. S. (1991). Building the national agenda for transfer: a background
paper. In American Council on Education, Setting the national agenda: Academic
achievement and transfer (pp. 17-46). Washington, DC: American Council on Education.
Park, J-H., & Choi. H-J. (2009). Factors influencing adult learners’ decision to drop out or
persist in online learning. Educational Technology and Society, 12(4), 207-217.
Pennington, R. (2006). Rethinking grade transfer shock: examining its importance in the
community college transfer process. The Journal of Applied Research in the Community
College, 14(1), 19-33.
Phlegar, A. G., Andrew, L. D., & McLaughlin, G. W. (1981). Explaining the academic
performance of community college students who transfer to a senior institution. Research
in Higher Education, 15(2), 99-108.
Pincus, F. L., & DeCamp, S. (1989). Minority community college students who transfer to four
year colleges: a study of a matched sample of BA recipients and non recipients.
Community/Junior College Quarterly of Research and Practice, 13(3), 191-219.
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components
of classroom academic performance. Journal of Educational Psychology, 82(1), 33.
Preston, D. L. (1993). Interfacing two-year and four-year transcripts for transfer students. Paper
presented at the annual Forum of the Association for Institutional Research, Chicago.
Ragins, B. R., Cotton, J. L., & Miller, J. S. (2000). Marginal mentoring: The effects of type of
mentor, quality of relationship, and program design on work and career attitudes. Academy
of Management Journal, 43(6), 1177-1194.
Redmond, S. (Nov./Dec. 1990). Mentoring and cultural diversity in academic settings. American
Behavioral Scientist, 34(2), 188-200.
Roderick, M., Nagaoka, J., & Coca, V. (2009). College readiness for all: the challenge for urban
high schools. The Future of Children, 19(1), 185-210.
Roksa, J., & Calcagno, J. C. (2008). Making the transition to four-year institutions: Academic
preparation and transfer. (CCRC Working Paper No. 13). New York: Columbia
University, Teachers College, Community College Research Center.
Romero, C., Ventura, S., Espejo, P.G. and Hervas, C. (2008, June). Data mining algorithms to
classify students. Paper presented at the1st International Conference on Educational Data
Mining, Montreal, Canada.
Rovai, A. P. (2002). Building sense of community at a distance. The International Review of
Research in Open and Distance Learning, 3(1). Retrieved from
http://www.irrodl.org/index.php/irrodl/article/view/79/152
85
San Jose, D. L., & Kelleher, T. (2009). Measuring ecoshock and affective learning: A
comparison of student responses to online and face-to-face learning ecologies. MERLOT
Journal of Online Learning and Teaching, 5(3), 469-476.
Shapiro, D., Dundar, A., Chen, J., Ziskin, M., Park, E., Torres, V., & Chiang, Y. C. (2012).
Completing college: A national view of student attainment rates. Retrived from:
http://content.nwacc.edu/publicrelations/CostContainment/NSC_Signature_Report_4.pdf
Shapiro, D., Dundar, A., Ziskin, M., Chiang, Y. C., Chen, J., Harrell, A., & Torres, V. (2013).
Baccalaureate attainment: a national view of the postsecondary outcomes of students
who transfer from two-year to four-year institutions. National Student Clearinghouse
Research Center. Retrieved from: gttp://nscresearchcenter.org/wpcontent/uploads/SignatureReports5.pdf
Smith, P. J., Murphy, K. L., & Mahoney, S. E. (2003). Towards identifying factors underlying
readiness for online learning: An exploratory study. Distance Education, 24 (1), 57-67.
Soltz, D. F. (1992). JCCC transfer students: Their destinations and achievements. Overland
Park, KS: Johnson County Community College Office of Institutional Research.
Szafran, R. F. (2001). The effect of academic load on success for new college students: Is lighter
better? Research in Higher Education, 42(1), 27-50.
Szafran, R. F., & Austin, S. F. (2002). The Effect of academic load on success for new college
students: is lighter better? NACADA Journal, 22(2), 26-38.
Terenzini, P. T., Springer, L., Yaeger, P. M., Pascarella, E. T., & Nora, A. (1996). Firstgeneration college students: Characteristics, experiences, and cognitive
development. Research in Higher education, 37(1), 1-22.
Tham, C. M., & Werner, J. M. (2005). Designing and evaluating e-learning in higher education:
a review and recommendations. Journal of Leadership & Organizational Studies, 11(2),
15-25.
Tinto, C. (2001). Rethinking the first year of college. Syracuse University: Higher Education
Monograph Series.
Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent
research. Review of educational research, 89-125.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.).
Chicago: University of Chicago Press.
Townsend, B. (1995). Community college transfer students: a case study of survival. The Review
of Higher Education, 18(2), 175-193.
86
Townsend, B. K., & Wilson, K. (2006). "A hand hold for a little bit": factors facilitating the
success of community college transfer students to a large research university. Journal of
College Student Development, 47(4), 439-456.
Townsend, B. K., McNerney, N., & Arnold, A. (1993). Will this community college transfer
student succeed? factors affecting transfer student performance. Community College
Journal of Research and Practice, 17(5), 433 444.
Volkwein, J. F., & Lorang, W. G. (1996). Characteristics of extenders: full-time students who
take light credit loads and graduate in more than four years. Research in Higher
Education, 37(1), 43-68.
Walpole, M. (2003). Socioeconomic status and college: How SES affects college experiences
and outcomes. The Review of Higher Education, 27(1), 45-73.
Wassmer, R., Moore, C., & Shulock, N. (2004). Effect of racial/ethnic composition on transfer
rates in community colleges: implications for policy and practice. Research in Higher
Education, 45(6), 651-672.
Willging, P.A. & Johnson, S.D. (2009). Factors influencing adult student decisions to dropout of
online courses. Journal of Asynchronous Learning Networks, 13 (3), 115-127.
Williams, P. E., & Hellman, C. M. (2004). Differences in self-regulation for online learning
between first-and second-generation college students. Research in Higher Education,
45(1), 71-82.
Zakir Khouz, A. M., & Ahmed Shami, M. A. (1982). Relationship of students’ course loads and
their grade point average scores. Mecca, Saudi Arabia: Umm Al-Quara University
College of Education.
Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview.
Educational psychologist, 25(1), 3-17.
Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background,
methodological developments, and future prospects. American Educational Research
Journal, 45(1), 166-183.
Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-motivation for academic
attainment: The role of self-efficacy beliefs and personal goal setting. American
educational research journal, 29(3), 663-676.