Download Assessment of JiTT on Student Learning

Assessment of JiTT on
Student learning
Kathleen A. Marrs, Ph.D.
Department of Biology
IUPUI, Indianapolis IN
Research Goals
Question: Can a classroom environment using constructivist-based
JiTT activities improve student performance in Biology?
1.
2.
3.
4.
5.
Identify misconceptions students bring to Biology courses.
Create interactivity and active learning in large lecture class
Improve student study habits and class attendance
Increase relevance of studying Biology to everyday life
Enhance content knowledge in Biology
Method: leverage technology to realize these goals!
Outcome: Collect data to determine if this approach is effective in improving
student study habits and content knowledge in Biology.
JiTT Improves Study Habits
Warm Ups made a significant difference in student study skills.
We asked students to report whether they ‘crammed’ for exams in
Biology N100 (JiTT) vs. their other, non-JiTT classes: (n = 426)
‘Crammed’ in
Biology N100
‘Crammed’ in
other courses
A students
16%
44%
B students
34%
63%
C students
41%
65%
D students
64%
71%
F students
68%
69%
JiTT Improves Graduate Student
Study Habits
1) Did you put off studying for Biotech 540 and as a result ‘cram’ for
Biotech 540 tests (saving virtually all studying for the day/night
before the test)?
(34.4%) Yes
2) Do you ‘cram’ for other courses that you have this semester?
(62.0%) Yes
3) Do you feel that the Warm Up assignments and web notes made you
stay ‘caught up’ on class material?
(96.5%) Yes
4) Do you feel that you are more, less, or equally well ‘caught up’ on
your other graduate courses this semester?
(17.6%) Equal, (19.8%) More, (62.6%) Less
Biology 540 (Graduate Biotechnology), n=37
JiTT Improves Class Preparation
1) Do you read the web notes before class?
2) Do you do the readings from the text before class?
3) Do you do read the textbook or the class notes (if provided)
before your other classes?
A students:
B students:
C students:
D students:
F students:
(1) Yes
78%
75
63
45
46
Biology N100,
n=162
(2) Yes
65%
72
61
40
42
(3) Yes
53%
48
51
40
42
Effect of Cooperative Learning
Exercises on Student Learning
a) Do you think the Cooperative Learning exercises help you to learn the class
material?
Yes (67%)
b) Do you prefer doing the Cooperative Learning exercises to listening to lecture
for 75 minutes?
Yes (72%)
c)Any positive or negative things you would like to say about them?




Q4 = Cooperative Learning is a good thing to have. No offense, Dr. Marrs, but ..…if it
wasn't for the Coorperative Learning, I'd never come here.
Q4 = I like the cooperative learning exercises because they're the only thing that we do in
the class to participate, and it gets boring just listening. I have to stay active.
Q4 = I think the exercises help me to understand the material a little better. I think maybe we
do too many, but it definitely keeps me from getting too comfortable and falling asleep in
class when I'm tired. They also take some of the pressure off of those horrible tests.
Q4 = I think that the Cooperative Learning exercises help me to learn the material. They are
a mini review for me!! They definitely help break up the tedium of lecture. They are
wonderful!! Plus I like the fact that they can kind of take attendance for those of us who
show up to class everyday!
More measures assessing JiTT’s
effectiveness in teaching and learning

Increased retention rates: (lower DFW rates)
Examples: (Biology N100) 29% DFW lowered to 21% DFW.

Increased cognitive gains: Examples: Measurable gains in Force Concept
Inventory, a standardized test in Physics, and on Pre-vs. Post class tests in
JiTT Biology classes.

Increased classroom interactivity: based on reports from trained classroom
observers.

Students prefer JiTT: to traditional lecture classes, based on course attitude
surveys, anonymous end-of-course evaluations, and student focus groups.
“The web page was great. I’ve never had a class run from the web
before and it was really neat. All of the different categories (Warm
up, help, notes) were really great and made the class fun and
interesting and different.”
JiTT and Content Knowledge

…but do N100 students ‘learn’ the material any better with this
approach? Evidence from Pre- and Post tests, with and without JiTT:
Pre-test
(% correct)
Warm Up
CL Exercise
Post test (%
correct)
Mitosis
15%
18%
Yes
Yes
62%
78%
Genetics
12%
20%
Yes
Yes
48%
64%
Natural
selection
10%
14%
Yes
Yes
25%
56%
Human
Population
Growth
15%
17%
Yes
Yes
65%
85%
Pre-vs-Post: Mitosis

Pre-test: In human cells, which of the following happens during mitosis?

the duplication of all 46 chromosomes
(23%)

the replication of DNA
(20%)

the pairing of homologous chromosomes
(11%)

the fragmentation and disappearance of the nuclear envelope
(18%)

all of the above
(28%)

Warm Up question: What kind of ‘work’ does a cell in your liver do during Interphase? What
happens to the DNA of that cell during the S-phase of Interphase? What kind of ‘work’ is that
same cell doing during mitosis? What happens to the DNA of a cell during mitosis?

Cooperative Learning: Mitosis and Cell Replacement: Drawing cells in different phases of
the cell cycle.

Post Test: In human cells, which of the following happens during mitosis?

the duplication of all 46 chromosomes
( 5%)

the replication of DNA
( 9%)

the pairing of homologous chromosomes
( 6%)

the fragmentation and disappearance of the nuclear envelope
(78%)*

all of the above
( 2%)

*(62%) without Warm Ups and Cooperative Learning
Pre-vs.-Post: Genetics

Pre-test: A couple has just found out that they are both carriers for cystic fibrosis, a
recessive metabolic disorder (carriers = Cc). What are the chances that their child will
have the disorder cystic fibrosis (cc)?

a)0%
(12%)

b)25%, or 1/4
(20%)

c)50% or 2/4
(28%)

d)75% or 3/4
(21%)

e) 100% or 4/4
(19%)

Warm Up: A couple who are in good health have a baby and are shocked to find that he has
cystic fibrosis, an inherited, recessive disease. How did the child inherit a life-threatening disease
from healthy people? Why weren't the parents affected by cystic fibrosis? Can this couple give
birth to a child without cystic fibrosis? Why or why not?

Cooperative Learning: Genetics problems.

Post-test: A couple has just found out that they are both carriers for cystic fibrosis, a
recessive metabolic disorder (carriers = Cc). What are the chances that their child will
have the disorder cystic fibrosis (cc)?

a)0%
( 8%)

b)25%, or 1/4
(64%) *(48%) without Warm Ups and Cooperative Learning

c)50% or 2/4
(14%)

d)75% or 3/4
( 6%)

e) 100% or 4/4
( 8%)
Pre-vs.-Post: Natural Selection

Pre Test: According to evidence gathered by Charles Darwin, evolution and natural
selection is based on the _________ found in populations of organisms.
• a)
acquired characteristics
(15%)
• b)
weakest members
(16%)
• c)
strongest members
(45%)
• d)
normal variations
(14%)
• e)
similarities
(10%)

Warm Up: “What does the term Survival of the Fittest mean? How does the environment
influence the survival of an organism?”

Cooperative Learning: Graphing evolution of antibiotic resistance over time.

Post Test :According to evidence gathered by Charles Darwin, evolution and natural
selection is based on the _________ found in populations of organisms.
• a)
acquired characteristics
( 4%)
• b)
weakest members
(10%)
• c)
strongest members
(23%)
• d)
normal variations
(56%)*
• e)
similarities
( 6%)
*(25%) without Warm Ups and Cooperative Learning
Summary: JiTT Improves Content
Knowledge in Biology
% Gain (Posttest-pretest) Average Normalized Gain
Questions on pretest with no additional interventions
during semester
Questions on pretest tied to additional back of the
book homework problems during semester
Questions on pretest tied to Warm Up or cooperative
learning questions during semester
Questions on pretest tied to Warm Up and
cooperative learning questions during semester
Average of 4 questions
%G = 15% (25%-10%)
Average of 4 questions
%G = 17% (35%-18%)
Average of 4 questions
%G = 45% (59%-14%)
Average of 4 questions
%G = 56% (68%-12%)
Average of 4 questions
<g>= 15%/90 = 16.66
Average of 4 questions
<g>= 17%/82 = 21.00
Average of 4 questions
<g>= 45%/88 = 52.00
Average of 4 questions
<g>= 17%/88 = 63.00
%Gain (%G) = Percent of class correct on post-test - % of class correct on pre-test
Average normalized gain: <g> = % correct on post-test - % correct on pre-test / 100-(%Gain possible)
We used the results from a 20-question pre-class and post-class test, calculating the average normalized gain as
described (Hake 2002, reviewed in Dancy and Beichner 2002). We found that students tended to show an average
normalized gain of about 15% on test questions about concepts that were discussed in class, but not necessarily reinforced
by any additional class activities, and showed an average normalized gain of about 21% on test questions that were
reinforced by back of the book homework problems.
However, students tended to show an average normalized gain of ~52% on test questions that were reinforced by
either Warm Up questions or Cooperative Learning activities, and showed an average normalized gain of ~60% on
test questions reinforced by both Warm Up questions or Cooperative Learning activities (Marrs 2005, manuscript
submitted).