Download Sadler - Math Science Partnership

Preparing for Success in College Science:
The Dance of Mathematics, Misconceptions,
Teacher Knowledge, and the Advanced
Placement Program
Philip M. Sadler, Director
Science Education Department
Harvard-Smithsonian Center for Astrophysics, Cambridge, MA
1
Why someone from the
Harvard-Smithsonian
Center for Astrophysics?
2
Harvard-Smithsonian
Center for Astrophysics
•
•
Largest astronomical research
institution in the world
A partnership between:
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
 Harvard’s Department of Astronomy
 Harvard College Observatory
 Smithsonian Astrophysical Observatory
•
•
•
More than 250 scientists in a staff of
over 800
Telescopes on earth and in space
Search for earth-like planets
MSP Regional Conference
QuickTime
TIFF (Uncompressed
are needed to see
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
March 30-31, 2006
Why listen?
•
•
•
Education
– MIT B.S. in Physics
– Harvard Ed.M., Ed.D.’92
Teaching
– middle school science and
math
– Harvard University
• Astronomy
• Ed Research
• Ed Methods
• 200 teachers
Developer of
– Starlab Planetarium
– Project STAR
– MicroObservatory
•
Publications
– 5 texts
– 38 papers and book chapters
– 4 award-winning videos
•
Editorial Board
•
•
MSP Regional Conference
– 2 ed journals
Sponsored research
– $56m, $5m/yr
– 40 staff
Honors
–
–
–
–
ASP Brennan Prize
Project ASTRO Education Award
3 AIP Computers in Physics
1999 JRST Award
March 30-31, 2006
How do you rigorously
measure the conceptual
understanding of teachers
and students in science?
5
How do you rigorously
measure the conceptual
understanding of teachers
and students in science?
6
Psychological Foundations
“The unlearning of preconceptions
might very well prove to be the
most determinative single factor
in the acquisition and retention of
subject-matter knowledge.”
David Ausubel 1978
MSP Regional Conference
March 30-31, 2006
Psychological Foundations
“The unlearning of preconceptions
might very well prove to be the
most determinative single factor
in the acquisition and retention of
subject-matter knowledge.”
David Ausubel 1978
MSP Regional Conference
March 30-31, 2006
Clinical Interviews
Minds of Our Own consists of 3-one hour
programs broadcast on PBS in 1997-98. It
explores the ideas of students as they come to
understand scientific concepts
On-on-one with students
A Private Universe documents
students’ ideas through their
own drawings and explanations
www.learner.org
MSP Regional Conference
March 30-31, 2006
Students and teachers
have preconceptions
•
•
•
•
•
Exist prior to instruction
At odds with accepted scientific
thought, “misconceptions”
Commonly held, not
idiosyncratic
embedded in larger knowledge
structures, not just an “error”
resistant to change
MSP Regional Conference
March 30-31, 2006
MOSART – Misconception Oriented Standardsbased Assessment Resource for Teachers
Our criteria for conceptual understanding –
Students and teachers must:
– Prefer accepted scientific explanations over
widely-held misconceptions
– Apply their knowledge to make accurate
predictions
MSP Regional Conference
March 30-31, 2006
Our criteria for conceptual understanding
Students and teachers must:
– Prefer accepted scientific explanations over widelyheld misconceptions
– Apply their knowledge to make accurate predictions
For assessments to do this, test items must:
–
–
–
–
Include the scientifically correct answer
Include the most popular misconceptions
Be easy to score and use
Value predictive over “why” questions
MSP Regional Conference
March 30-31, 2006
5-8 Physical Science:
Motions and Forces
• The motion of an object can be
described by its position, direction of
motion, and speed. That motion can be
measured and represented on a graph.
MSP Regional Conference
March 30-31, 2006
The Problem
108. Kevin starts walking from a store a certain
distance from his home. Which sentence is a
correct description of Kevin’s motion as shown
on the graph?
MSP Regional Conference
March 30-31, 2006
The Correct Answer
108. Kevin starts walking from a store a certain
distance from his home. Which sentence is a
correct description of Kevin’s motion as shown
on the graph?
b. He walks toward home,
stops for a while, then walks
away from home.
MSP Regional Conference
March 30-31, 2006
The Fraction Who Choose Correctly
108. Kevin starts walking from a store a certain
distance from his home. Which sentence is a
correct description of Kevin’s motion as shown
on the graph?
b. He walks toward home,
stops for a while, then walks
away from home. 30%
MSP Regional Conference
March 30-31, 2006
Other answers that students give?
108. Kevin starts walking from a store a certain
distance from his home. Which sentence is a
correct description of Kevin’s motion as shown
on the graph?
a. He walks toward home
down a hill, then walks along a
level path, then walks up a hill.
b. He walks toward home,
stops for a while, then walks
away from home. 30%
c. He walks away from
home, stops for a while, then
walks toward home.
d. He walks toward home
down a hill, stops for a while,
then walks up a hill.
e. He walks down a hill and
gets trapped in a valley.
MSP Regional Conference
March 30-31, 2006
Which answers do your students give?
108. Kevin starts walking from a store a certain
distance from his home. Which sentence is a
correct description of Kevin’s motion as shown
on the graph?
MSP Regional Conference
a. He walks toward home
down a hill, then walks along a
level path, then walks up a hill.
28%
b. He walks toward home,
stops for a while, then walks
away from home. 30%
c. He walks away from
home, stops for a while, then
walks toward home. 18%
d. He walks toward home
down a hill, stops for a while,
then walks up a hill. 20%
e. He walks down a hill and
gets trapped in a valley. 5%
March 30-31, 2006
Research Questions
•
To what degree have students who completed
science courses mastered the NRC standards?
– At grade level
– At prior grade levels
•
Are there patterns of strength and weakness?
MSP Regional Conference
March 30-31, 2006
Research Questions
•
To what degree have students who completed
science courses mastered the NRC standards?
– At grade level
– At prior grade levels
•
•
•
•
Are there patterns of strength and weakness?
Have primary, middle, and high school science
teachers mastered the standards that they teach?
How well can teachers predict the knowledge state of
their students (including misconceptions)?
What is the impact of professional development
activities on teacher content knowledge?
MSP Regional Conference
March 30-31, 2006
Mining the Research Literature
76. An electric cord runs from a wall
outlet along the floor to a lamp. The
lamp’s light is on. You carefully
stack books, one at a time, on top of
each other on the wire until you
have 100 pounds of books.
Assuming the wire does not break,
what do you think would happen to
the brightness of the light?
a)
b)
c)
d)
e)
The brightness of the light would decrease gradually as more books
were added to the stack.
The light would dim all at once at some point, then remain dim.
The light would go out as soon as the first book was placed on the
wire.
The light would flicker or briefly dim as each book was added, then
return to normal.
The light would not change in brightness. 14%
MSP Regional Conference
March 30-31, 2006
Mining the Research Literature
76. An electric cord runs from a wall
outlet along the floor to a lamp. The
lamp’s light is on. You carefully
stack books, one at a time, on top of
each other on the wire until you
have 100 pounds of books.
Assuming the wire does not break,
what do you think would happen to
the brightness of the light?
a)
b)
c)
d)
e)
The brightness of the light would decrease gradually as more books
were added to the stack.
The light would dim all at once at some point, then remain dim.
The light would go out as soon as the first book was placed on the
wire.
The light would flicker or briefly dim as each book was added, then
return to normal.
The light would not change in brightness. 14%
MSP Regional Conference
March 30-31, 2006
Student Preference
76. An electric cord runs from a wall
outlet along the floor to a lamp. The
lamp’s light is on. You carefully
stack books, one at a time, on top of
each other on the wire until you
have 100 pounds of books.
Assuming the wire does not break,
what do you think would happen to
the brightness of the light?
a)
b)
c)
d)
e)
The brightness of the light would decrease gradually as more books
were added to the stack. 39%
The light would dim all at once at some point, then remain dim. 10%
The light would go out as soon as the first book was placed on the
wire. 6%
The light would flicker or briefly dim as each book was added, then
return to normal. 11%
The light would not change in brightness. 14%
MSP Regional Conference
March 30-31, 2006
Test Item Development
Breakdown of the NRCs
What concepts are the standards really asking kids to know?
What are the relevant misconceptions reported in the literature?
Item Construction
Items (M/C for ease) that represents the standard and captures kids’
knowledge based on research protocols.
Validation
Are the questions accurate in terms of the science? Readable?
Pilot Testing (N=100/item)
selection of core items that represent the most variance
Large scale sample (Physical Science Example, N=1000/item)
Item characteristics for 100-200 items/domain
Characterization of domains: 7,000+ students, 50+ teachers
Finalization of Instruments
Made available for evaluation of programs like yours
MSP Regional Conference
March 30-31, 2006
National Data:
MSP Regional Conference
March 30-31, 2006
Pr
op
er
IB
t
Ch ie s
of
em
ica Ma
IC
tte
lR
Ch
r
e
em
ac
tio
ic
al
IIA
ns
El
G
e
m
ra
en
ph
IIB
ts
i
n
1s
g
IIC
tL
M
ot
U
a
io
w
ni
n
di
o
fM
m
III
en
A
sio otio
Pr
n
na
op
lf
er
or
t ie
ce
s
s
of
En
er
III
III
B
gy
C
H
Be
ea
tf
ha
III
lo
D
vi
w
or
El
ec
of
tri
Li
ca
gh
III
E
lC
t
En
ir c
er
ui
ts
gy
III
Tr
F
an
So
sf
er
la
rE
ne
rg
y
IA
Grade 7/8 Physical Science Students
After Taking a Year of Physical Science
1.00
0.90
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
grade 7-8 physical
science students
n=6503
0.00
MSP Regional Conference
March 30-31, 2006
Adding HS Chemistry and Physics Students
1.00
0.90
grade 10-11
chemistry students
n=209
0.80
0.70
0.60
0.50
0.40
0.30
grade 11-12 physics
students n=178
0.20
0.10
IA
Pr
op
er
IB
t
Ch ie s
of
em
ica Ma
IC
tte
lR
Ch
r
e
em
ac
tio
ic
al
IIA
ns
El
G
e
m
ra
en
ph
IIB
ts
i
n
1s
g
IIC
tL
M
ot
U
a
io
w
ni
n
di
o
fM
m
III
en
A
sio otio
Pr
n
na
op
lf
er
or
t ie
ce
s
s
of
En
er
III
III
B
gy
C
H
Be
ea
tf
ha
III
lo
D
vi
w
or
El
ec
of
tri
Li
ca
gh
III
E
lC
t
En
ir c
er
ui
ts
gy
III
Tr
F
an
So
sf
er
la
rE
ne
rg
y
0.00
MSP Regional Conference
grade 7-8 physical
science students
n=6503
March 30-31, 2006
Adding HS Science Teachers
1.00
HS Chemistry and
PhysicsTeachers
n=16
0.90
0.80
0.70
0.60
0.50
grade 10-11
chemistry students
n=209
0.40
0.30
0.20
0.10
grade 11-12 physics
students n=178
grade 7-8 physical
science students
n=6503
MSP Regional Conference
March 30-31, 2006
IA
Pr
op
er
IB
t
Ch ie s
of
em
ica Ma
IC
tte
lR
Ch
r
e
em
ac
tio
ic
al
IIA
ns
El
G
e
m
ra
en
ph
IIB
ts
i
n
1s
g
IIC
tL
M
ot
U
a
io
w
ni
n
di
o
fM
m
III
en
A
sio otio
Pr
n
na
op
lf
er
or
t ie
ce
s
s
of
En
er
III
III
B
gy
C
H
Be
ea
tf
ha
III
lo
D
vi
w
or
El
ec
of
tri
Li
ca
gh
III
E
lC
t
En
ir c
er
ui
ts
gy
III
Tr
F
an
So
sf
er
la
rE
ne
rg
y
0.00
Adding MS Physical Science Teachers
HS Chemistry
and
PhysicsTeachers
n=16
1.00
0.90
0.80
0.70
grade 10-11
chemistry
students n=209
0.60
0.50
0.40
grade 11-12
physics students
n=178
0.30
0.20
0.10
IA
Pr
op
er
IB
t
Ch ie s
of
em
ica Ma
IC
tte
lR
Ch
r
e
em
ac
tio
ic
al
IIA
ns
El
G
e
m
ra
en
ph
IIB
ts
i
n
1s
g
IIC
tL
M
ot
U
a
io
w
ni
n
di
o
fM
m
III
en
A
sio otio
Pr
n
na
op
lf
er
or
t ie
ce
s
s
of
En
er
III
III
B
gy
C
H
Be
ea
tf
ha
III
lo
D
vi
w
or
El
ec
of
tri
Li
ca
gh
III
E
lC
t
En
ir c
er
ui
ts
gy
III
Tr
F
an
So
sf
er
la
rE
ne
rg
y
0.00
MSP Regional Conference
grade 7-8
physical science
students n=6503
MS Physical
ScienceTeachers
n=35
March 30-31, 2006
MSP Regional Conference
II I
F
H
C
ns
f
y
er
ne
rg
rE
t
ts
ig
h
ui
irc
Tr
a
So
la
gy
al
w
gy
En
er
s
ea
tf
lo
of
ce
fo
r
io
ro
fL
ric
er
ec
t
En
El
ha
v
II I
B
er
t ie
s
Be
II I
E
II I
D
II I
C
op
n
ot
io
n
ot
io
M
en
ts
of
M
g
on
al
aw
en
si
tL
ph
in
ns
at
te
r
tio
ea
c
of
M
El
em
lR
es
al
ic
ica
rti
G
ra
1s
di
m
Pr
Un
i
II I
A
II C
m
he
m
II A
C
Ch
e
op
e
Pr
II B
IC
IB
IA
Teacher Content and Predictive Knowledge
1.00
0.90
0.80
0.70
grade 7-8
physical
science
students
n=6503
0.60
0.50
MS Physical
ScienceTeach
ers n=35
0.40
0.30
0.20
0.10
MS
Predictions
0.00
March 30-31, 2006
Patterns in Classroom Data
MSP Regional Conference
March 30-31, 2006
Comparison of Item Formats
with misconception
without misconception
76. Electric Cord
68. Refraction of laser
61. Weight Loss
21. Mass of Baking
Soda + Vinegar
13. Candle wax
0.00
0.10
0.20
0.30
0.40
MSP Regional Conference
0.50
0.60
0.70
0.80
0.90
1.00
March 30-31, 2006
Teacher Content and Predictive Knowledge
Across 38 Classrooms
Student Performance
60%
50%
Low Content
Knowledge
High Content
Knowledge
Over-Prediction
40%
30%
20%
Accurate
Prediction
10%
0%
Content
Knowledge
Predictive
Knowledge
MSP Regional Conference
March 30-31, 2006
Teacher Content Knowledge and
Teacher Prediction Accuracy across 38 Classrooms
Student
Performance
60%
50%
40%
Low Content
Knowledge
High Content
Knowledge
30%
20%
10%
0%
OverPrediction
Accurate
Prediction
MSP Regional Conference
March 30-31, 2006
Patterns
For each standard at each level
–
–
–
–
Students have not achieved mastery
Teachers generally overestimate student knowledge.
Teachers know far more than their students
Teacher knowledge is a not a guarantee of student
knowledge
– Subject do much better on items if misconceptions are not a
choice
•
Teachers’ knowledge of student ideas is associated
with higher performance than content knowledge
MSP Regional Conference
March 30-31, 2006
Patterns in
Professional Development Data
MSP Regional Conference
March 30-31, 2006
Which factors predict teacher content
knowledge of the curriculum concepts?
1.
2.
3.
4.
5.
6.
7.
8.
Grade level
Gender
Years Teaching
Years Teaching science subject
Certification in the science subject
Degrees (BS, BA, MS, PhD)
Grad Courses taken in domain
Professional development in science
teaching/content
MSP Regional Conference
March 30-31, 2006
Predicting Teacher Mastery
linear models with significant factors
Model B 36% of variance
Source
Const
Grade band
Gender
Years Teaching subject
Certification
Yrs Teaching subject * Cert
Error
df
1
2
1
1
1
1
73
∑sq
57.56
0.14
0.06
0.08
0.03
0.06
0.61
Total
79
0.95
MSP Regional Conference
F-ratio Prob
6836.20 ≤ 0.0001
8.70
0.0004
8.29
0.0052
10.58 0.0017
4.49
0.0374
7.90
0.0063
March 30-31, 2006
Which factors predict teacher content
knowledge of the curriculum concepts?
1.
2.
3.
4.
5.
6.
7.
8.
Grade level
Gender
Years Teaching
Years Teaching science subject
Certification in the science subject
Degrees (BS, BA, MS, PhD)
Grad Courses taken in domain
Professional development in science
teaching/content
MSP Regional Conference
March 30-31, 2006
Interaction of
Years Teaching Subject and Certification
100%
Certified
Not Certified
95%
90%
85%
80%
75%
0
1
2
3
4
5
6
7
8
# of Years Teaching Earth and Space Science
MSP Regional Conference
March 30-31, 2006
2-Week Astronomy Institute
•
•
•
•
•
•
MSP Regional Conference
Basics
To boost astronomy
background
General astronomy
test
Speakers
Activities
Observing
March 30-31, 2006
2-Week Astronomy Institute
Astronomy Form 611 Pre-
•
1.00
.90
•
.80
Posttest
.70
•
.60
.50
.40
•
.30
.20
•
.10
.00
.00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0
0
•
Basics
To boost astronomy
background
General astronomy
test
Speakers
Activities
Observing
Pretest total
MSP Regional Conference
March 30-31, 2006
2-Week Astronomy Institute
Astronomy Form 611 Pre-
•
1.00
.90
.80
•
Posttest
.70
.60
.50
•
.40
.30
.20
Moderate initial
knowledge
Gains at all levels of
teacher knowledge
Few teachers with
no or negative
growth
.10
.00
.00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0
0
Pretest total
MSP Regional Conference
March 30-31, 2006
1-Week Astronomy Institute
•
•
•
•
•
•
MSP Regional Conference
Instrumentation
Earth-Sun
connection only
Only relevant items
Speakers
Activities
Observing
March 30-31, 2006
1-Week Astronomy Institute
Astronomy Form 611 Pre-
•
1.00
.90
.80
Posttest
.70
•
.60
.50
.40
•
Learn to use
professional
instrumentation
Disciplinary domain
focus
Speakers
.30
.20
.10
.00
.00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0
0
Pretest total
MSP Regional Conference
March 30-31, 2006
1-Week Astronomy Institute
Astronomy Form 611 Pre-
•
1.00
.90
•
.80
Posttest
.70
.60
.50
.40
•
.30
.20
.10
High initial
knowledge
No gains at highest
level of teacher
knowledge
Many teachers with
no or negative
growth
.00
.00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0
0
Pretest total
MSP Regional Conference
March 30-31, 2006
Comparison of 2 MSP Institutes
Astronomy Form 611 Pre-
1.00
1.00
.90
.90
.80
.80
.70
.70
Posttest
Posttest
Astronomy Form 611 Pre-
.60
.50
.60
.50
.40
.40
.30
.30
.20
.20
.10
.10
.00
.00
.00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0
0
.00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0
0
Pretest total
Pretest total
MSP Regional Conference
March 30-31, 2006
Patterns
•
•
•
Some teacher content weakness at all grade levels:
weakest at MS levels
Content knowledge grows very slowly for the noncertified teacher
Professional development can make a difference in
teacher content knowledge
– Length of program
– Focus on content knowledge at grade level vs. “science
apprenticeships”
– Must evaluate the fulfillment of goals
•
Content knowledge at higher levels does not
translate to knowledge at lower levels
MSP Regional Conference
March 30-31, 2006
Seeking Research Partners
•
Professional Development
– Increase in teacher content knowledge
– Increase in teacher pedagogical content knowledge
– Customized assessment instruments
•
Linking to Student Pre-Post Assessment
– Curricular and Pedagogical Innovation
– Impact of professional development
• Teacher Subject Matter Knowledge
• Accuracy of Teacher Prediction
– Breakout session tomorrow
MSP Regional Conference
March 30-31, 2006
Overview of Research
$3M, 4-year IERI study to investigate the kinds
of high school courses that best prepare
college students for:
– introductory courses in biology, chemistry, or
physics
MSP Regional Conference
March 30-31, 2006
Overview of Research
$3M, 4-year IERI study to investigate the kinds
of high school courses that best prepare
college students for:
– introductory courses in biology, chemistry, or
physics
Drawing hypotheses from
– research literature
– high school teachers
– Professors
MSP Regional Conference
March 30-31, 2006
Overview of Research
$3M, 4-year IERI study to investigate the kinds of high
school courses that best prepare college students for:
– introductory courses in biology, chemistry, or physics
Drawing hypotheses from
– research literature
– high school teachers
– Professors
67 items survey, sample of
– 18,000 college students
– 1st and 2nd semester
– 63 randomly-chosen colleges
MSP Regional Conference
March 30-31, 2006
FICSS Study Goals
Identify the HS pedagogy and curriculum that
prepare students for college science:
1.
•
•
•
From HS science teachers
From college professors
From educational researchers
MSP Regional Conference
March 30-31, 2006
FICSS Study Goals
Identify the HS pedagogy and curriculum that
prepare students for college science:
1.
•
•
•
From HS science teachers
From college professors
From educational researchers
Collect evidence that supports or refutes these
beliefs concerning:
2.
•
•
•
Physics First
Block Scheduling
Advanced Placement
MSP Regional Conference
•Labs and Demos
•Mathematics
•Project Work
March 30-31, 2006
Comparison of Teacher and Professor Views of
Factors Predicting Success in College Science
<10%
Fraction of Teachers
>100
<10%
Student-designed labs
Multi-sensory approach
10-30%
Use tech software
Classroom discussions
>30% Multiple exposures
Fraction of Professors
10-30%
Independent projects
Improve reading skills
Humanize science
Mimic scientists
Teach a specific topic
(cell/mol-bio, genetics)
Teacher enthusiasm
Science is cool
Estimation without a
calculator
Improve writing skills
Parental encouragement
MSP Regional Conference
>30%
Develop thinking skills
Avoid memorization
Conceptual learning
Improve study skills
Real life contexts
Cooperative learning
Math in context
Multiple problems and
contexts
Alternative representations
Math proficiency
March 30-31, 2006
Comparison of Teacher and Professor Views of
Factors Predicting Success in College Science
<10%
Fraction of Teachers
>100
<10%
Student-designed labs
Multi-sensory approach
10-30%
Use tech software
Classroom discussions
>30% Multiple exposures
Fraction of Professors
10-30%
Independent projects
Improve reading skills
Humanize science
Mimic scientists
Teach a specific topic
(cell/mol-bio, genetics)
Teacher enthusiasm
Science is cool
Estimation without a
calculator
Improve writing skills
Parental encouragement
MSP Regional Conference
>30%
Develop thinking skills
Avoid memorization
Conceptual learning
Improve study skills
Real life contexts
Cooperative learning
Math in context
Multiple problems and
contexts
Alternative representations
Math proficiency
March 30-31, 2006
Comparison of Teacher and Professor Views of
Factors Predicting Success in College Science
<10%
Fraction of Teachers
>100
<10%
Student-designed labs
Multi-sensory approach
10-30%
Use tech software
Classroom discussions
>30% Multiple exposures
Fraction of Professors
10-30%
Independent projects
Improve reading skills
Humanize science
Mimic scientists
Teach a specific topic
(cell/mol-bio, genetics)
Teacher enthusiasm
Science is cool
Estimation without a
calculator
Improve writing skills
Parental encouragement
MSP Regional Conference
>30%
Develop thinking skills
Avoid memorization
Conceptual learning
Improve study skills
Real life contexts
Cooperative learning
Math in context
Multiple problems and
contexts
Alternative representations
Math proficiency
March 30-31, 2006
Comparison of Teacher and Professor Views of
Factors Predicting Success in College Science
<10%
Fraction of Teachers
>100
<10%
Student-designed labs
Multi-sensory approach
10-30%
Use tech software
Classroom discussions
>30% Multiple exposures
Fraction of Professors
10-30%
Independent projects
Improve reading skills
Humanize science
Mimic scientists
Teach a specific topic
(cell/mol-bio, genetics)
Teacher enthusiasm
Science is cool
Estimation without a
calculator
Improve writing skills
Parental encouragement
MSP Regional Conference
>30%
Develop thinking skills
Avoid memorization
Conceptual learning
Improve study skills
Real life contexts
Cooperative learning
Math in context
Multiple problems and
contexts
Alternative representations
Math proficiency
March 30-31, 2006
Predictor Categories
•
Background
– Parents Ed
– SES
– Type of
physics
– School
– Class
attributes
– Course choice
• Grades
• SAT’s
MSP Regional Conference
March 30-31, 2006
Predictor Categories
•
Background
– Parents Ed
– SES
– Type of
physics
– School
– Class
attributes
– Course choice
• Grades
• SAT’s
•
Pedagogy
– Instructional
approach
– Demos
– Labs
– Autonomy
– Technology
– Homework/text
– Teacher
– Tests/assignments
– Discipline
MSP Regional Conference
March 30-31, 2006
Predictor Categories
•
Background
– Parents Ed
– SES
– Type of
physics
– School
– Class
attributes
– Course choice
• Grades
• SAT’s
•
Pedagogy
– Instructional
approach
– Demos
– Labs
– Autonomy
– Technology
– Homework/text
– Teacher
– Tests/assignments
– Discipline
MSP Regional Conference
•
Content
–
–
–
–
–
–
–
–
–
–
Facts
Concepts
Skills
Mechanics
Electricity
Stoichiometry
Periodic table
Genetics
Evolution
Dissection
March 30-31, 2006
Views on Factors
Block Scheduling
QuickTime™ and a
DV/DVCPRO - NTSC decompressor
are needed to see this picture.
Graphing by Hand
QuickTime™ and a
DV/DVCPRO - NTSC decompressor
are needed to see this picture.
MSP Regional Conference
Teacher Quality
QuickTime™ and a
DV/DVCPRO - NTSC decompressor
are needed to see this picture.
Mathematics
QuickTime™ and a
DV/DVCPRO - NTSC decompressor
are needed to see this picture.
March 30-31, 2006
…our most recent findings.
63
Predictor
College or University
B
SE B
b
Included
(Constant)
Demographic and General Educational Background
Highest Parental Educational Level
College Enrollment Status
Graduate Student
Non-degree Student
Race/Ethnicity
Native American
Asian
Black
Latino
Multi-racial
Not Reported
a
SAT Verbal
b
SAT Ma thematics
Last HS Mathematics Grade
Calculus (Non Ğ Advanced Placement)
Advanced Placement Calculus AB
40.87
***
2.64
0.69
**
.23
0.07
8.26
4.77
***
**
2.33
1.70
0.08
0.06
-0.01
-0.03
0.01
-0.07
-0.03
0.02
0.06
0.17
0.20
0.06
0.14
-1.57
-0.98
0.48
-3.04
-1.52
2.65
0.01
0.02
2.79
1.58
3.24
*
***
***
*
***
2.56
0.86
1.18
1.09
1.37
2.44
0.003
0.003
0.35
0.70
0.61
Advanced Placement Calculus BC
5.18
***
0.97
0.13
Last HS Science Grade
Advanced Placement Chemistry
Science as a Means to a Better Career
1.72
2.15
1.48
***
***
**
0.37
0.63
0.51
0.12
0.08
0.07
No Encouragement t o take Science
0.99
*
0.49
0.05
MSP Regional Conference
**
March 30-31, 2006
Predictor
College or University
B
SE B
b
Included
(Constant)
Demographic and General Educational Background
Highest Parental Educational Level
College Enrollment Status
Graduate Student
Non-degree Student
Race/Ethnicity
Native American
Asian
Black
Latino
Multi-racial
Not Reported
a
SAT Verbal
b
SAT Ma thematics
Last HS Mathematics Grade
Calculus (Non Ğ Advanced Placement)
Advanced Placement Calculus AB
40.87
***
2.64
0.69
**
.23
0.07
8.26
4.77
***
**
2.33
1.70
0.08
0.06
-0.01
-0.03
0.01
-0.07
-0.03
0.02
0.06
0.17
0.20
0.06
0.14
-1.57
-0.98
0.48
-3.04
-1.52
2.65
0.01
0.02
2.79
1.58
3.24
*
***
***
*
***
2.56
0.86
1.18
1.09
1.37
2.44
0.003
0.003
0.35
0.70
0.61
Advanced Placement Calculus BC
5.18
***
0.97
0.13
Last HS Science Grade
Advanced Placement Chemistry
Science as a Means to a Better Career
1.72
2.15
1.48
***
***
**
0.37
0.63
0.51
0.12
0.08
0.07
No Encouragement t o take Science
0.99
*
0.49
0.05
MSP Regional Conference
**
March 30-31, 2006
Predictor
College or University
B
SE B
b
Included
(Constant)
Demographic and General Educational Background
Highest Parental Educational Level
College Enrollment Status
Graduate Student
Non-degree Student
Race/Ethnicity
Native American
Asian
Black
Latino
Multi-racial
Not Reported
a
SAT Verbal
b
SAT Ma thematics
Last HS Mathematics Grade
Calculus (Non Ğ Advanced Placement)
Advanced Placement Calculus AB
40.87
***
2.64
0.69
**
.23
0.07
8.26
4.77
***
**
2.33
1.70
0.08
0.06
-0.01
-0.03
0.01
-0.07
-0.03
0.02
0.06
0.17
0.20
0.06
0.14
-1.57
-0.98
0.48
-3.04
-1.52
2.65
0.01
0.02
2.79
1.58
3.24
*
***
***
*
***
2.56
0.86
1.18
1.09
1.37
2.44
0.003
0.003
0.35
0.70
0.61
Advanced Placement Calculus BC
5.18
***
0.97
0.13
Last HS Science Grade
Advanced Placement Chemistry
Science as a Means to a Better Career
1.72
2.15
1.48
***
***
**
0.37
0.63
0.51
0.12
0.08
0.07
No Encouragement t o take Science
0.99
*
0.49
0.05
MSP Regional Conference
**
March 30-31, 2006
Mathematics Preparation
QuickTime™ and a
Sorenson Video 3 decompressor
are needed to see this picture.
MSP Regional Conference
March 30-31, 2006
Student Comments: math
•
•
•
•
•
•
Fewer topics, more in-depth. Make honors physics calculus based. I
was in honors physics in HS and it was hardly math-based at all, much
less calculus-based.
The high school course I took gave me a good conceptual basis, but the
mathematics was not stressed as much as in college.
More focus on the mathematical side of physics
My high school teacher taught us step by step methods to obtaining
the answers mathematically, this was very beneficial when doing word
problems in college.
High school students should be learning to think about physical
situations mathematically, and gaining familiarity with the kinds of
problems we would do in college.
More of the mathematical transformations needed to properly do
physics at the college level is required.
MSP Regional Conference
March 30-31, 2006
High School Science Laboratory Experiences
Some examples of predictors used in this analysis:
Full Understanding (=5) versus Memorization (=1)
Labs Frequently Addressed Student’s Beliefs
Labs for Improving Conceptual Understanding
Time Discussing Labs
Analyzing Pictures or Illustrations
Draw/Interpret Graphs by Hand
Student-Designed Projects
Read & Discuss Labs a Day Before
Labs Frequently Built Upon Previous Experience
Understanding of Lab Procedure
Freedom in Designing & Conducting Labs
Use of Computer Simulations
30 Variables were studied in this analysis
MSP Regional Conference
March 30-31, 2006
What Appears to:
Help:
•
Often Analyzed Pictures or Illustrations
•
Often Draw/Interpret Graphs by Hand
•
Quantitative problems
•
Labs Addressed Student’s Beliefs
•
More Freedom in Designing & Conducting Labs
(high math achievers)
•
Testing for facts
•
Mastery of select foundational concepts
•
Physics
–
–
–
–
•
more mechanics
more history of physics
less relativity
More prediction, less demo discussion
Chemistry
–
–
More stoichiometry
Less nuclear chemistry
MSP Regional Conference
March 30-31, 2006
What Appears to:
Help:
•
Often Analyzed Pictures or Illustrations
•
Often Draw/Interpret Graphs by Hand
•
Quantitative problems
•
Labs Addressed Student’s Beliefs
•
More Freedom in Designing & Conducting Labs
(high math achievers)
•
Testing for facts
•
Mastery of select foundational concepts
•
Physics
–
–
–
–
•
more mechanics
more history of physics
less relativity
More prediction, less demo discussion
Chemistry
–
–
More stoichiometry
Less nuclear chemistry
Hinder:
•
•
•
•
•
•
•
•
•
MSP Regional Conference
Read & Discuss Labs a Day
Before
Greater Understanding of Lab
Procedure
Student-Designed Projects
More Freedom in Designing &
Conducting Labs (low math
achievers)
Coverage of entire domain
Standardized exam prep
Testing on labs
Using class time to teach facts
and vocabulary
Reading the textbook
March 30-31, 2006
Lab Experience
QuickTime™ and a
Video decompressor
are needed to see this picture.
MSP Regional Conference
March 30-31, 2006
Student Comments: labs
•
•
•
•
In a basic high school physics course I would advise a lot of hands on
activities and labs to help students understand the basic concepts of
kinematics which tend to hinder a lot of students at the college level.
I would change the labs. Although the labs completed were excellent
in high school, detailed lab reports were not required and did not
prepare me for college physics lab reports.
Also, less physics labs in high school would be better and more focus
on the math and free body diagram aspect of physics.
I would suggest that the labs should be more challenging and less
emphasis should be placed on memorization and more emphasis on
comprehension.
MSP Regional Conference
March 30-31, 2006
Physics First
•
Leon Lederman’s Project ARISE
• A physics-chemistry-biology sequence leads the student
from the simple to the complex, an approach which is in
harmony with current understanding of how the brain
learns.
• Understanding modern biology, for example the function
of DNA, requires a background in chemistry, physics, and
mathematics.
• Moreover, chemistry is based upon the charge structure
of atoms and the forces between these charges,
concepts learned in physics.
MSP Regional Conference
March 30-31, 2006
Testing Physics First Hypotheses
1.
2.
3.
4.
Taking HS physics will have a positive impact
on chemistry performance
Taking HS chemistry will have a positive effect
on college biology
Students who take HS physics before HS
chemistry (2%) will perform better in college
chemistry (4%)
Students who now take HS chemistry before
HS biology (6%) will perform better in college
biology
MSP Regional Conference
March 30-31, 2006
College Performance in Biology, Chemistry and
Physics Based on HS Coursework
College Grade
90
High School Biology
High School Chemistry High School Physics
85
College
Biology
College
Chemistry
80
College
Physics
75
none Reg AP Reg none Reg AP Reg none Reg AP Reg
only only & AP
only only & AP
only only & AP
MSP Regional Conference
March 30-31, 2006
College Performance in Biology
based on high school coursework
College Grade
90
High School Biology
High School Chemistry High School Physics
85
College
Biology
80
75
none Reg AP Reg none Reg AP Reg none Reg AP Reg
only only & AP
only only & AP
only only & AP
MSP Regional Conference
March 30-31, 2006
College Performance in Biology and Chemistry
Based on Amount of HS Coursework
College Grade
90
High School Biology
High School Chemistry High School Physics
85
College
Biology
80
College
Chemistry
75
none Reg AP Reg none Reg AP Reg none Reg AP Reg
only only & AP
only only & AP
only only & AP
MSP Regional Conference
March 30-31, 2006
The Advanced Placement Program
•
AP began as a way for “exceptional students”
at elite private schools:
–
–
–
To take rigorous courses in HS
No planned impact on college admissions (1952)
No planned impact on GPA
MSP Regional Conference
March 30-31, 2006
The Advanced Placement Program
•
•
AP began as a way for “exceptional students”
at elite private schools:
– To take rigorous courses in HS
– No planned impact on college admissions (1952)
– No planned impact on GPA
Expanded to >2.1M exams/yr in 35 subjects
MSP Regional Conference
March 30-31, 2006
The Advanced Placement Program
AP began as a way for “exceptional students” at elite
private schools:
•
–
–
–
To take rigorous courses in HS
No planned impact on college admissions (1952)
No planned impact on GPA
Expanded to >2.1M exams/yr in 35 subjects
Benefits to the student (other than learning):
–
–
–
–
–
Higher HS Grade Point Average
Taking college courses in High School
High probability of getting into college and financial aid
Higher college grades if repeated
College credit (advanced standing), cost savings
•
•
MSP Regional Conference
March 30-31, 2006
What the public hears
“It is better to take a tougher course and get a
low grade than to take an easy course and
get a high grade.”
Clifford Adelman, Senior Research Analyst,
U.S. Dept. of Ed.
MSP Regional Conference
March 30-31, 2006
Our Research Questions
For students taking introductory college biology,
chemistry, and physics:
•
What grades do students earn based on high
school AP performance?
•
What is the predicted advantage taking AP when
controlling for student background, preparation,
and SES?
MSP Regional Conference
March 30-31, 2006
College Performance in Introductory
Science Courses
100
90
wit h controls
raw scores
85
80
5
4
3
2
1
ex
am
AP
.
N
o
no
AP
AP
o
ho
no
rs
,
re
gu
la
r,
n
ta
ke
n
75
no
t
College Science
95
MSP Regional Conference
March 30-31, 2006
Covariates with AP Score: The need for regression
models to model the unique contribution of AP courses.
90
80
Highest
Science
Course Type
Honors or
Regular
Course
Grade
SAT Mat h Score
MSP Regional Conference
Highest Math
Course Level
Highest Math
CourseGrade
AP 5
AP 4
AP 3
AP 2
AP 1
A
B
C
D
Calculus BC
Calculus AB
Calculus
precalculus
ŠAlgebra II
750
650
550
450
350
250
A
B
C
D
AP
honors
regular
70
not taken
College Science
100
AP Exam Score
March 30-31, 2006
Modeling the Impact of AP Courses
After controlling for covariates
100
90
wit h controls
raw scores
85
80
5
4
3
2
1
ex
am
AP
.
N
o
no
AP
AP
o
ho
no
rs
,
re
gu
la
r,
n
ta
ke
n
75
no
t
College Science
95
MSP Regional Conference
March 30-31, 2006
Modeling the Impact of AP Courses
After controlling for covariates
100
90
wit h controls
raw scores
85
80
5
4
3
2
1
ex
am
AP
.
N
o
no
AP
AP
o
ho
no
rs
,
re
gu
la
r,
n
ta
ke
n
75
no
t
College Science
95
MSP Regional Conference
March 30-31, 2006
Modeling the Impact of AP Courses
Students who do not take the exam perform at the
same level as those earning a 3
100
90
wit h controls
raw scores
85
80
5
4
3
2
1
ex
am
AP
.
N
o
no
AP
AP
o
ho
no
rs
,
re
gu
la
r,
n
ta
ke
n
75
no
t
College Science
95
MSP Regional Conference
March 30-31, 2006
Earning a “5” predicts increasing college grade
by 5 points over honors
100
90
wit h controls
raw scores
85
80
5
4
3
2
1
ex
am
AP
.
N
o
no
AP
AP
o
ho
no
rs
,
re
gu
la
r,
n
ta
ke
n
75
no
t
College Science
95
MSP Regional Conference
March 30-31, 2006
Earning a “4” predicts increasing college grade
by 4 points over honors
100
90
wit h controls
raw scores
85
80
5
4
3
2
1
ex
am
AP
.
N
o
no
AP
AP
o
ho
no
rs
,
re
gu
la
r,
n
ta
ke
n
75
no
t
College Science
95
MSP Regional Conference
March 30-31, 2006
Modeling the Impact of AP Courses
100
90
wit h controls
raw scores
85
80
5
4
3
2
1
ex
am
AP
.
N
o
no
AP
AP
o
ho
no
rs
,
re
gu
la
r,
n
ta
ke
n
75
no
t
College Science
95
MSP Regional Conference
March 30-31, 2006
Conclusions
92
Conclusions
•
AP students do earn somewhat higher grades in
college science
– Partial proxy for demographic, general scholastic performance,
math preparation
– and performance in high school science courses that are
prerequisites to AP in most schools
MSP Regional Conference
March 30-31, 2006
Conclusions
•
AP students do earn somewhat higher grades in college
science
– Partial proxy for demographic, general scholastic performance,
math preparation
– and performance in high school science courses that are
prerequisites to AP in most schools
•
Course order is unimportant
MSP Regional Conference
March 30-31, 2006
Conclusions
•
AP students do earn somewhat higher grades in college
science
– Partial proxy for demographic, general scholastic performance,
math preparation
– and performance in high school science courses that are
prerequisites to AP in most schools
•
•
Course order is unimportant, amount is
The best preparation comes from HS courses that:
– Use lots of math
– Concentrate on key concepts, not coverage
– Use labs judiciously to change misconceptions
MSP Regional Conference
March 30-31, 2006
What do we know now?
Misconceptions often unchanged after taking science.
1.
Necessary step in learning
The standards are hard to master.
Teachers are knowledgeable, but does not assure student
learning.
Teachers do not know their students’ misconceptions, but should.
Teacher knowledge builds slowly.
Professional development must be
2.
3.
4.
5.
•
•
6.
targeted to specific standards at grade levels
evaluated with relevant tools.
AP courses help the most if they focus on quantitative science,
conceptual labs, fundamentals.
MSP Regional Conference
March 30-31, 2006
Acknowledgments
•
Co-investigators:
– Robert Tai, University of
Virginia, Matthew Schneps,
•
•
Survey Staff:
Video Staff:
– Yael Bowman, Toby
McElheny, Nancy Finkelstein,
Alexia Prichard, Alex
Griswold
•
Graduate Students:
– Zahra Hazari, John Loehr
MSP Regional Conference
Board of Advisors
–
–
–
–
– Jamie Miller, Nancy Cook
Smith, Cynthia Crockett, Marc
Schwartz (McGill), Annette
Trenga, Bruce Ward
•
Advice
– NSF: Janice Earle, Barry
Sloane, Elizabeth VanderPutten,
Larry Suter
Project Managers:
– Hal Coyle, Michael Filisky
•
•
•
Joel Mintzes, Mary Atwater
Brian Alters, Lillian McDermott
Eric Mazur, James Wandersee
Dudley Herschbach
Financial Support
– NSF
– Annenberg/CPB
•
– DoEd
– NIH
Center for Astrophysics
– Irwin Shapiro, Judith Peritz.
March 30-31, 2006
Any opinions, findings and conclusions or
recommendations expressed in this material are
those of the authors and do not necessarily reflect
the views of the
National Science Foundation,
National Institutes of Health,
U.S. Department of Education
98
Harvard-Smithsonian
Center for Astrophysics
Science Education Department
60 Garden Street, MS-71
Cambridge, MA 02138
Phone: 617-496-7598
Fax:
617-496-5405
Email: [email protected]
MSP Regional Conference
March 30-31, 2006
Leo Tolstoy
"I know that most men, including those at ease
with problems of the greatest complexity, can
seldom accept even the simplest and most
obvious truth if it be such as would oblige them
to admit the falsity of conclusions which they
have delighted in explaining to colleagues, which
they have proudly taught to others, and which
they have woven, thread by thread, into the fabric
of their lives."
MSP Regional Conference
March 30-31, 2006