Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Preparing for Success in College Science: The Dance of Mathematics, Misconceptions, Teacher Knowledge, and the Advanced Placement Program Philip M. Sadler, Director Science Education Department Harvard-Smithsonian Center for Astrophysics, Cambridge, MA 1 Why someone from the Harvard-Smithsonian Center for Astrophysics? 2 Harvard-Smithsonian Center for Astrophysics • • Largest astronomical research institution in the world A partnership between: QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. Harvard’s Department of Astronomy Harvard College Observatory Smithsonian Astrophysical Observatory • • • More than 250 scientists in a staff of over 800 Telescopes on earth and in space Search for earth-like planets MSP Regional Conference QuickTime TIFF (Uncompressed are needed to see QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. March 30-31, 2006 Why listen? • • • Education – MIT B.S. in Physics – Harvard Ed.M., Ed.D.’92 Teaching – middle school science and math – Harvard University • Astronomy • Ed Research • Ed Methods • 200 teachers Developer of – Starlab Planetarium – Project STAR – MicroObservatory • Publications – 5 texts – 38 papers and book chapters – 4 award-winning videos • Editorial Board • • MSP Regional Conference – 2 ed journals Sponsored research – $56m, $5m/yr – 40 staff Honors – – – – ASP Brennan Prize Project ASTRO Education Award 3 AIP Computers in Physics 1999 JRST Award March 30-31, 2006 How do you rigorously measure the conceptual understanding of teachers and students in science? 5 How do you rigorously measure the conceptual understanding of teachers and students in science? 6 Psychological Foundations “The unlearning of preconceptions might very well prove to be the most determinative single factor in the acquisition and retention of subject-matter knowledge.” David Ausubel 1978 MSP Regional Conference March 30-31, 2006 Psychological Foundations “The unlearning of preconceptions might very well prove to be the most determinative single factor in the acquisition and retention of subject-matter knowledge.” David Ausubel 1978 MSP Regional Conference March 30-31, 2006 Clinical Interviews Minds of Our Own consists of 3-one hour programs broadcast on PBS in 1997-98. It explores the ideas of students as they come to understand scientific concepts On-on-one with students A Private Universe documents students’ ideas through their own drawings and explanations www.learner.org MSP Regional Conference March 30-31, 2006 Students and teachers have preconceptions • • • • • Exist prior to instruction At odds with accepted scientific thought, “misconceptions” Commonly held, not idiosyncratic embedded in larger knowledge structures, not just an “error” resistant to change MSP Regional Conference March 30-31, 2006 MOSART – Misconception Oriented Standardsbased Assessment Resource for Teachers Our criteria for conceptual understanding – Students and teachers must: – Prefer accepted scientific explanations over widely-held misconceptions – Apply their knowledge to make accurate predictions MSP Regional Conference March 30-31, 2006 Our criteria for conceptual understanding Students and teachers must: – Prefer accepted scientific explanations over widelyheld misconceptions – Apply their knowledge to make accurate predictions For assessments to do this, test items must: – – – – Include the scientifically correct answer Include the most popular misconceptions Be easy to score and use Value predictive over “why” questions MSP Regional Conference March 30-31, 2006 5-8 Physical Science: Motions and Forces • The motion of an object can be described by its position, direction of motion, and speed. That motion can be measured and represented on a graph. MSP Regional Conference March 30-31, 2006 The Problem 108. Kevin starts walking from a store a certain distance from his home. Which sentence is a correct description of Kevin’s motion as shown on the graph? MSP Regional Conference March 30-31, 2006 The Correct Answer 108. Kevin starts walking from a store a certain distance from his home. Which sentence is a correct description of Kevin’s motion as shown on the graph? b. He walks toward home, stops for a while, then walks away from home. MSP Regional Conference March 30-31, 2006 The Fraction Who Choose Correctly 108. Kevin starts walking from a store a certain distance from his home. Which sentence is a correct description of Kevin’s motion as shown on the graph? b. He walks toward home, stops for a while, then walks away from home. 30% MSP Regional Conference March 30-31, 2006 Other answers that students give? 108. Kevin starts walking from a store a certain distance from his home. Which sentence is a correct description of Kevin’s motion as shown on the graph? a. He walks toward home down a hill, then walks along a level path, then walks up a hill. b. He walks toward home, stops for a while, then walks away from home. 30% c. He walks away from home, stops for a while, then walks toward home. d. He walks toward home down a hill, stops for a while, then walks up a hill. e. He walks down a hill and gets trapped in a valley. MSP Regional Conference March 30-31, 2006 Which answers do your students give? 108. Kevin starts walking from a store a certain distance from his home. Which sentence is a correct description of Kevin’s motion as shown on the graph? MSP Regional Conference a. He walks toward home down a hill, then walks along a level path, then walks up a hill. 28% b. He walks toward home, stops for a while, then walks away from home. 30% c. He walks away from home, stops for a while, then walks toward home. 18% d. He walks toward home down a hill, stops for a while, then walks up a hill. 20% e. He walks down a hill and gets trapped in a valley. 5% March 30-31, 2006 Research Questions • To what degree have students who completed science courses mastered the NRC standards? – At grade level – At prior grade levels • Are there patterns of strength and weakness? MSP Regional Conference March 30-31, 2006 Research Questions • To what degree have students who completed science courses mastered the NRC standards? – At grade level – At prior grade levels • • • • Are there patterns of strength and weakness? Have primary, middle, and high school science teachers mastered the standards that they teach? How well can teachers predict the knowledge state of their students (including misconceptions)? What is the impact of professional development activities on teacher content knowledge? MSP Regional Conference March 30-31, 2006 Mining the Research Literature 76. An electric cord runs from a wall outlet along the floor to a lamp. The lamp’s light is on. You carefully stack books, one at a time, on top of each other on the wire until you have 100 pounds of books. Assuming the wire does not break, what do you think would happen to the brightness of the light? a) b) c) d) e) The brightness of the light would decrease gradually as more books were added to the stack. The light would dim all at once at some point, then remain dim. The light would go out as soon as the first book was placed on the wire. The light would flicker or briefly dim as each book was added, then return to normal. The light would not change in brightness. 14% MSP Regional Conference March 30-31, 2006 Mining the Research Literature 76. An electric cord runs from a wall outlet along the floor to a lamp. The lamp’s light is on. You carefully stack books, one at a time, on top of each other on the wire until you have 100 pounds of books. Assuming the wire does not break, what do you think would happen to the brightness of the light? a) b) c) d) e) The brightness of the light would decrease gradually as more books were added to the stack. The light would dim all at once at some point, then remain dim. The light would go out as soon as the first book was placed on the wire. The light would flicker or briefly dim as each book was added, then return to normal. The light would not change in brightness. 14% MSP Regional Conference March 30-31, 2006 Student Preference 76. An electric cord runs from a wall outlet along the floor to a lamp. The lamp’s light is on. You carefully stack books, one at a time, on top of each other on the wire until you have 100 pounds of books. Assuming the wire does not break, what do you think would happen to the brightness of the light? a) b) c) d) e) The brightness of the light would decrease gradually as more books were added to the stack. 39% The light would dim all at once at some point, then remain dim. 10% The light would go out as soon as the first book was placed on the wire. 6% The light would flicker or briefly dim as each book was added, then return to normal. 11% The light would not change in brightness. 14% MSP Regional Conference March 30-31, 2006 Test Item Development Breakdown of the NRCs What concepts are the standards really asking kids to know? What are the relevant misconceptions reported in the literature? Item Construction Items (M/C for ease) that represents the standard and captures kids’ knowledge based on research protocols. Validation Are the questions accurate in terms of the science? Readable? Pilot Testing (N=100/item) selection of core items that represent the most variance Large scale sample (Physical Science Example, N=1000/item) Item characteristics for 100-200 items/domain Characterization of domains: 7,000+ students, 50+ teachers Finalization of Instruments Made available for evaluation of programs like yours MSP Regional Conference March 30-31, 2006 National Data: MSP Regional Conference March 30-31, 2006 Pr op er IB t Ch ie s of em ica Ma IC tte lR Ch r e em ac tio ic al IIA ns El G e m ra en ph IIB ts i n 1s g IIC tL M ot U a io w ni n di o fM m III en A sio otio Pr n na op lf er or t ie ce s s of En er III III B gy C H Be ea tf ha III lo D vi w or El ec of tri Li ca gh III E lC t En ir c er ui ts gy III Tr F an So sf er la rE ne rg y IA Grade 7/8 Physical Science Students After Taking a Year of Physical Science 1.00 0.90 0.80 0.70 0.60 0.50 0.40 0.30 0.20 0.10 grade 7-8 physical science students n=6503 0.00 MSP Regional Conference March 30-31, 2006 Adding HS Chemistry and Physics Students 1.00 0.90 grade 10-11 chemistry students n=209 0.80 0.70 0.60 0.50 0.40 0.30 grade 11-12 physics students n=178 0.20 0.10 IA Pr op er IB t Ch ie s of em ica Ma IC tte lR Ch r e em ac tio ic al IIA ns El G e m ra en ph IIB ts i n 1s g IIC tL M ot U a io w ni n di o fM m III en A sio otio Pr n na op lf er or t ie ce s s of En er III III B gy C H Be ea tf ha III lo D vi w or El ec of tri Li ca gh III E lC t En ir c er ui ts gy III Tr F an So sf er la rE ne rg y 0.00 MSP Regional Conference grade 7-8 physical science students n=6503 March 30-31, 2006 Adding HS Science Teachers 1.00 HS Chemistry and PhysicsTeachers n=16 0.90 0.80 0.70 0.60 0.50 grade 10-11 chemistry students n=209 0.40 0.30 0.20 0.10 grade 11-12 physics students n=178 grade 7-8 physical science students n=6503 MSP Regional Conference March 30-31, 2006 IA Pr op er IB t Ch ie s of em ica Ma IC tte lR Ch r e em ac tio ic al IIA ns El G e m ra en ph IIB ts i n 1s g IIC tL M ot U a io w ni n di o fM m III en A sio otio Pr n na op lf er or t ie ce s s of En er III III B gy C H Be ea tf ha III lo D vi w or El ec of tri Li ca gh III E lC t En ir c er ui ts gy III Tr F an So sf er la rE ne rg y 0.00 Adding MS Physical Science Teachers HS Chemistry and PhysicsTeachers n=16 1.00 0.90 0.80 0.70 grade 10-11 chemistry students n=209 0.60 0.50 0.40 grade 11-12 physics students n=178 0.30 0.20 0.10 IA Pr op er IB t Ch ie s of em ica Ma IC tte lR Ch r e em ac tio ic al IIA ns El G e m ra en ph IIB ts i n 1s g IIC tL M ot U a io w ni n di o fM m III en A sio otio Pr n na op lf er or t ie ce s s of En er III III B gy C H Be ea tf ha III lo D vi w or El ec of tri Li ca gh III E lC t En ir c er ui ts gy III Tr F an So sf er la rE ne rg y 0.00 MSP Regional Conference grade 7-8 physical science students n=6503 MS Physical ScienceTeachers n=35 March 30-31, 2006 MSP Regional Conference II I F H C ns f y er ne rg rE t ts ig h ui irc Tr a So la gy al w gy En er s ea tf lo of ce fo r io ro fL ric er ec t En El ha v II I B er t ie s Be II I E II I D II I C op n ot io n ot io M en ts of M g on al aw en si tL ph in ns at te r tio ea c of M El em lR es al ic ica rti G ra 1s di m Pr Un i II I A II C m he m II A C Ch e op e Pr II B IC IB IA Teacher Content and Predictive Knowledge 1.00 0.90 0.80 0.70 grade 7-8 physical science students n=6503 0.60 0.50 MS Physical ScienceTeach ers n=35 0.40 0.30 0.20 0.10 MS Predictions 0.00 March 30-31, 2006 Patterns in Classroom Data MSP Regional Conference March 30-31, 2006 Comparison of Item Formats with misconception without misconception 76. Electric Cord 68. Refraction of laser 61. Weight Loss 21. Mass of Baking Soda + Vinegar 13. Candle wax 0.00 0.10 0.20 0.30 0.40 MSP Regional Conference 0.50 0.60 0.70 0.80 0.90 1.00 March 30-31, 2006 Teacher Content and Predictive Knowledge Across 38 Classrooms Student Performance 60% 50% Low Content Knowledge High Content Knowledge Over-Prediction 40% 30% 20% Accurate Prediction 10% 0% Content Knowledge Predictive Knowledge MSP Regional Conference March 30-31, 2006 Teacher Content Knowledge and Teacher Prediction Accuracy across 38 Classrooms Student Performance 60% 50% 40% Low Content Knowledge High Content Knowledge 30% 20% 10% 0% OverPrediction Accurate Prediction MSP Regional Conference March 30-31, 2006 Patterns For each standard at each level – – – – Students have not achieved mastery Teachers generally overestimate student knowledge. Teachers know far more than their students Teacher knowledge is a not a guarantee of student knowledge – Subject do much better on items if misconceptions are not a choice • Teachers’ knowledge of student ideas is associated with higher performance than content knowledge MSP Regional Conference March 30-31, 2006 Patterns in Professional Development Data MSP Regional Conference March 30-31, 2006 Which factors predict teacher content knowledge of the curriculum concepts? 1. 2. 3. 4. 5. 6. 7. 8. Grade level Gender Years Teaching Years Teaching science subject Certification in the science subject Degrees (BS, BA, MS, PhD) Grad Courses taken in domain Professional development in science teaching/content MSP Regional Conference March 30-31, 2006 Predicting Teacher Mastery linear models with significant factors Model B 36% of variance Source Const Grade band Gender Years Teaching subject Certification Yrs Teaching subject * Cert Error df 1 2 1 1 1 1 73 ∑sq 57.56 0.14 0.06 0.08 0.03 0.06 0.61 Total 79 0.95 MSP Regional Conference F-ratio Prob 6836.20 ≤ 0.0001 8.70 0.0004 8.29 0.0052 10.58 0.0017 4.49 0.0374 7.90 0.0063 March 30-31, 2006 Which factors predict teacher content knowledge of the curriculum concepts? 1. 2. 3. 4. 5. 6. 7. 8. Grade level Gender Years Teaching Years Teaching science subject Certification in the science subject Degrees (BS, BA, MS, PhD) Grad Courses taken in domain Professional development in science teaching/content MSP Regional Conference March 30-31, 2006 Interaction of Years Teaching Subject and Certification 100% Certified Not Certified 95% 90% 85% 80% 75% 0 1 2 3 4 5 6 7 8 # of Years Teaching Earth and Space Science MSP Regional Conference March 30-31, 2006 2-Week Astronomy Institute • • • • • • MSP Regional Conference Basics To boost astronomy background General astronomy test Speakers Activities Observing March 30-31, 2006 2-Week Astronomy Institute Astronomy Form 611 Pre- • 1.00 .90 • .80 Posttest .70 • .60 .50 .40 • .30 .20 • .10 .00 .00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0 0 • Basics To boost astronomy background General astronomy test Speakers Activities Observing Pretest total MSP Regional Conference March 30-31, 2006 2-Week Astronomy Institute Astronomy Form 611 Pre- • 1.00 .90 .80 • Posttest .70 .60 .50 • .40 .30 .20 Moderate initial knowledge Gains at all levels of teacher knowledge Few teachers with no or negative growth .10 .00 .00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0 0 Pretest total MSP Regional Conference March 30-31, 2006 1-Week Astronomy Institute • • • • • • MSP Regional Conference Instrumentation Earth-Sun connection only Only relevant items Speakers Activities Observing March 30-31, 2006 1-Week Astronomy Institute Astronomy Form 611 Pre- • 1.00 .90 .80 Posttest .70 • .60 .50 .40 • Learn to use professional instrumentation Disciplinary domain focus Speakers .30 .20 .10 .00 .00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0 0 Pretest total MSP Regional Conference March 30-31, 2006 1-Week Astronomy Institute Astronomy Form 611 Pre- • 1.00 .90 • .80 Posttest .70 .60 .50 .40 • .30 .20 .10 High initial knowledge No gains at highest level of teacher knowledge Many teachers with no or negative growth .00 .00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0 0 Pretest total MSP Regional Conference March 30-31, 2006 Comparison of 2 MSP Institutes Astronomy Form 611 Pre- 1.00 1.00 .90 .90 .80 .80 .70 .70 Posttest Posttest Astronomy Form 611 Pre- .60 .50 .60 .50 .40 .40 .30 .30 .20 .20 .10 .10 .00 .00 .00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0 0 .00 .10 .20 .30 .40 .50 .60 .70 .80 .90 1.0 0 Pretest total Pretest total MSP Regional Conference March 30-31, 2006 Patterns • • • Some teacher content weakness at all grade levels: weakest at MS levels Content knowledge grows very slowly for the noncertified teacher Professional development can make a difference in teacher content knowledge – Length of program – Focus on content knowledge at grade level vs. “science apprenticeships” – Must evaluate the fulfillment of goals • Content knowledge at higher levels does not translate to knowledge at lower levels MSP Regional Conference March 30-31, 2006 Seeking Research Partners • Professional Development – Increase in teacher content knowledge – Increase in teacher pedagogical content knowledge – Customized assessment instruments • Linking to Student Pre-Post Assessment – Curricular and Pedagogical Innovation – Impact of professional development • Teacher Subject Matter Knowledge • Accuracy of Teacher Prediction – Breakout session tomorrow MSP Regional Conference March 30-31, 2006 Overview of Research $3M, 4-year IERI study to investigate the kinds of high school courses that best prepare college students for: – introductory courses in biology, chemistry, or physics MSP Regional Conference March 30-31, 2006 Overview of Research $3M, 4-year IERI study to investigate the kinds of high school courses that best prepare college students for: – introductory courses in biology, chemistry, or physics Drawing hypotheses from – research literature – high school teachers – Professors MSP Regional Conference March 30-31, 2006 Overview of Research $3M, 4-year IERI study to investigate the kinds of high school courses that best prepare college students for: – introductory courses in biology, chemistry, or physics Drawing hypotheses from – research literature – high school teachers – Professors 67 items survey, sample of – 18,000 college students – 1st and 2nd semester – 63 randomly-chosen colleges MSP Regional Conference March 30-31, 2006 FICSS Study Goals Identify the HS pedagogy and curriculum that prepare students for college science: 1. • • • From HS science teachers From college professors From educational researchers MSP Regional Conference March 30-31, 2006 FICSS Study Goals Identify the HS pedagogy and curriculum that prepare students for college science: 1. • • • From HS science teachers From college professors From educational researchers Collect evidence that supports or refutes these beliefs concerning: 2. • • • Physics First Block Scheduling Advanced Placement MSP Regional Conference •Labs and Demos •Mathematics •Project Work March 30-31, 2006 Comparison of Teacher and Professor Views of Factors Predicting Success in College Science <10% Fraction of Teachers >100 <10% Student-designed labs Multi-sensory approach 10-30% Use tech software Classroom discussions >30% Multiple exposures Fraction of Professors 10-30% Independent projects Improve reading skills Humanize science Mimic scientists Teach a specific topic (cell/mol-bio, genetics) Teacher enthusiasm Science is cool Estimation without a calculator Improve writing skills Parental encouragement MSP Regional Conference >30% Develop thinking skills Avoid memorization Conceptual learning Improve study skills Real life contexts Cooperative learning Math in context Multiple problems and contexts Alternative representations Math proficiency March 30-31, 2006 Comparison of Teacher and Professor Views of Factors Predicting Success in College Science <10% Fraction of Teachers >100 <10% Student-designed labs Multi-sensory approach 10-30% Use tech software Classroom discussions >30% Multiple exposures Fraction of Professors 10-30% Independent projects Improve reading skills Humanize science Mimic scientists Teach a specific topic (cell/mol-bio, genetics) Teacher enthusiasm Science is cool Estimation without a calculator Improve writing skills Parental encouragement MSP Regional Conference >30% Develop thinking skills Avoid memorization Conceptual learning Improve study skills Real life contexts Cooperative learning Math in context Multiple problems and contexts Alternative representations Math proficiency March 30-31, 2006 Comparison of Teacher and Professor Views of Factors Predicting Success in College Science <10% Fraction of Teachers >100 <10% Student-designed labs Multi-sensory approach 10-30% Use tech software Classroom discussions >30% Multiple exposures Fraction of Professors 10-30% Independent projects Improve reading skills Humanize science Mimic scientists Teach a specific topic (cell/mol-bio, genetics) Teacher enthusiasm Science is cool Estimation without a calculator Improve writing skills Parental encouragement MSP Regional Conference >30% Develop thinking skills Avoid memorization Conceptual learning Improve study skills Real life contexts Cooperative learning Math in context Multiple problems and contexts Alternative representations Math proficiency March 30-31, 2006 Comparison of Teacher and Professor Views of Factors Predicting Success in College Science <10% Fraction of Teachers >100 <10% Student-designed labs Multi-sensory approach 10-30% Use tech software Classroom discussions >30% Multiple exposures Fraction of Professors 10-30% Independent projects Improve reading skills Humanize science Mimic scientists Teach a specific topic (cell/mol-bio, genetics) Teacher enthusiasm Science is cool Estimation without a calculator Improve writing skills Parental encouragement MSP Regional Conference >30% Develop thinking skills Avoid memorization Conceptual learning Improve study skills Real life contexts Cooperative learning Math in context Multiple problems and contexts Alternative representations Math proficiency March 30-31, 2006 Predictor Categories • Background – Parents Ed – SES – Type of physics – School – Class attributes – Course choice • Grades • SAT’s MSP Regional Conference March 30-31, 2006 Predictor Categories • Background – Parents Ed – SES – Type of physics – School – Class attributes – Course choice • Grades • SAT’s • Pedagogy – Instructional approach – Demos – Labs – Autonomy – Technology – Homework/text – Teacher – Tests/assignments – Discipline MSP Regional Conference March 30-31, 2006 Predictor Categories • Background – Parents Ed – SES – Type of physics – School – Class attributes – Course choice • Grades • SAT’s • Pedagogy – Instructional approach – Demos – Labs – Autonomy – Technology – Homework/text – Teacher – Tests/assignments – Discipline MSP Regional Conference • Content – – – – – – – – – – Facts Concepts Skills Mechanics Electricity Stoichiometry Periodic table Genetics Evolution Dissection March 30-31, 2006 Views on Factors Block Scheduling QuickTime™ and a DV/DVCPRO - NTSC decompressor are needed to see this picture. Graphing by Hand QuickTime™ and a DV/DVCPRO - NTSC decompressor are needed to see this picture. MSP Regional Conference Teacher Quality QuickTime™ and a DV/DVCPRO - NTSC decompressor are needed to see this picture. Mathematics QuickTime™ and a DV/DVCPRO - NTSC decompressor are needed to see this picture. March 30-31, 2006 …our most recent findings. 63 Predictor College or University B SE B b Included (Constant) Demographic and General Educational Background Highest Parental Educational Level College Enrollment Status Graduate Student Non-degree Student Race/Ethnicity Native American Asian Black Latino Multi-racial Not Reported a SAT Verbal b SAT Ma thematics Last HS Mathematics Grade Calculus (Non Ğ Advanced Placement) Advanced Placement Calculus AB 40.87 *** 2.64 0.69 ** .23 0.07 8.26 4.77 *** ** 2.33 1.70 0.08 0.06 -0.01 -0.03 0.01 -0.07 -0.03 0.02 0.06 0.17 0.20 0.06 0.14 -1.57 -0.98 0.48 -3.04 -1.52 2.65 0.01 0.02 2.79 1.58 3.24 * *** *** * *** 2.56 0.86 1.18 1.09 1.37 2.44 0.003 0.003 0.35 0.70 0.61 Advanced Placement Calculus BC 5.18 *** 0.97 0.13 Last HS Science Grade Advanced Placement Chemistry Science as a Means to a Better Career 1.72 2.15 1.48 *** *** ** 0.37 0.63 0.51 0.12 0.08 0.07 No Encouragement t o take Science 0.99 * 0.49 0.05 MSP Regional Conference ** March 30-31, 2006 Predictor College or University B SE B b Included (Constant) Demographic and General Educational Background Highest Parental Educational Level College Enrollment Status Graduate Student Non-degree Student Race/Ethnicity Native American Asian Black Latino Multi-racial Not Reported a SAT Verbal b SAT Ma thematics Last HS Mathematics Grade Calculus (Non Ğ Advanced Placement) Advanced Placement Calculus AB 40.87 *** 2.64 0.69 ** .23 0.07 8.26 4.77 *** ** 2.33 1.70 0.08 0.06 -0.01 -0.03 0.01 -0.07 -0.03 0.02 0.06 0.17 0.20 0.06 0.14 -1.57 -0.98 0.48 -3.04 -1.52 2.65 0.01 0.02 2.79 1.58 3.24 * *** *** * *** 2.56 0.86 1.18 1.09 1.37 2.44 0.003 0.003 0.35 0.70 0.61 Advanced Placement Calculus BC 5.18 *** 0.97 0.13 Last HS Science Grade Advanced Placement Chemistry Science as a Means to a Better Career 1.72 2.15 1.48 *** *** ** 0.37 0.63 0.51 0.12 0.08 0.07 No Encouragement t o take Science 0.99 * 0.49 0.05 MSP Regional Conference ** March 30-31, 2006 Predictor College or University B SE B b Included (Constant) Demographic and General Educational Background Highest Parental Educational Level College Enrollment Status Graduate Student Non-degree Student Race/Ethnicity Native American Asian Black Latino Multi-racial Not Reported a SAT Verbal b SAT Ma thematics Last HS Mathematics Grade Calculus (Non Ğ Advanced Placement) Advanced Placement Calculus AB 40.87 *** 2.64 0.69 ** .23 0.07 8.26 4.77 *** ** 2.33 1.70 0.08 0.06 -0.01 -0.03 0.01 -0.07 -0.03 0.02 0.06 0.17 0.20 0.06 0.14 -1.57 -0.98 0.48 -3.04 -1.52 2.65 0.01 0.02 2.79 1.58 3.24 * *** *** * *** 2.56 0.86 1.18 1.09 1.37 2.44 0.003 0.003 0.35 0.70 0.61 Advanced Placement Calculus BC 5.18 *** 0.97 0.13 Last HS Science Grade Advanced Placement Chemistry Science as a Means to a Better Career 1.72 2.15 1.48 *** *** ** 0.37 0.63 0.51 0.12 0.08 0.07 No Encouragement t o take Science 0.99 * 0.49 0.05 MSP Regional Conference ** March 30-31, 2006 Mathematics Preparation QuickTime™ and a Sorenson Video 3 decompressor are needed to see this picture. MSP Regional Conference March 30-31, 2006 Student Comments: math • • • • • • Fewer topics, more in-depth. Make honors physics calculus based. I was in honors physics in HS and it was hardly math-based at all, much less calculus-based. The high school course I took gave me a good conceptual basis, but the mathematics was not stressed as much as in college. More focus on the mathematical side of physics My high school teacher taught us step by step methods to obtaining the answers mathematically, this was very beneficial when doing word problems in college. High school students should be learning to think about physical situations mathematically, and gaining familiarity with the kinds of problems we would do in college. More of the mathematical transformations needed to properly do physics at the college level is required. MSP Regional Conference March 30-31, 2006 High School Science Laboratory Experiences Some examples of predictors used in this analysis: Full Understanding (=5) versus Memorization (=1) Labs Frequently Addressed Student’s Beliefs Labs for Improving Conceptual Understanding Time Discussing Labs Analyzing Pictures or Illustrations Draw/Interpret Graphs by Hand Student-Designed Projects Read & Discuss Labs a Day Before Labs Frequently Built Upon Previous Experience Understanding of Lab Procedure Freedom in Designing & Conducting Labs Use of Computer Simulations 30 Variables were studied in this analysis MSP Regional Conference March 30-31, 2006 What Appears to: Help: • Often Analyzed Pictures or Illustrations • Often Draw/Interpret Graphs by Hand • Quantitative problems • Labs Addressed Student’s Beliefs • More Freedom in Designing & Conducting Labs (high math achievers) • Testing for facts • Mastery of select foundational concepts • Physics – – – – • more mechanics more history of physics less relativity More prediction, less demo discussion Chemistry – – More stoichiometry Less nuclear chemistry MSP Regional Conference March 30-31, 2006 What Appears to: Help: • Often Analyzed Pictures or Illustrations • Often Draw/Interpret Graphs by Hand • Quantitative problems • Labs Addressed Student’s Beliefs • More Freedom in Designing & Conducting Labs (high math achievers) • Testing for facts • Mastery of select foundational concepts • Physics – – – – • more mechanics more history of physics less relativity More prediction, less demo discussion Chemistry – – More stoichiometry Less nuclear chemistry Hinder: • • • • • • • • • MSP Regional Conference Read & Discuss Labs a Day Before Greater Understanding of Lab Procedure Student-Designed Projects More Freedom in Designing & Conducting Labs (low math achievers) Coverage of entire domain Standardized exam prep Testing on labs Using class time to teach facts and vocabulary Reading the textbook March 30-31, 2006 Lab Experience QuickTime™ and a Video decompressor are needed to see this picture. MSP Regional Conference March 30-31, 2006 Student Comments: labs • • • • In a basic high school physics course I would advise a lot of hands on activities and labs to help students understand the basic concepts of kinematics which tend to hinder a lot of students at the college level. I would change the labs. Although the labs completed were excellent in high school, detailed lab reports were not required and did not prepare me for college physics lab reports. Also, less physics labs in high school would be better and more focus on the math and free body diagram aspect of physics. I would suggest that the labs should be more challenging and less emphasis should be placed on memorization and more emphasis on comprehension. MSP Regional Conference March 30-31, 2006 Physics First • Leon Lederman’s Project ARISE • A physics-chemistry-biology sequence leads the student from the simple to the complex, an approach which is in harmony with current understanding of how the brain learns. • Understanding modern biology, for example the function of DNA, requires a background in chemistry, physics, and mathematics. • Moreover, chemistry is based upon the charge structure of atoms and the forces between these charges, concepts learned in physics. MSP Regional Conference March 30-31, 2006 Testing Physics First Hypotheses 1. 2. 3. 4. Taking HS physics will have a positive impact on chemistry performance Taking HS chemistry will have a positive effect on college biology Students who take HS physics before HS chemistry (2%) will perform better in college chemistry (4%) Students who now take HS chemistry before HS biology (6%) will perform better in college biology MSP Regional Conference March 30-31, 2006 College Performance in Biology, Chemistry and Physics Based on HS Coursework College Grade 90 High School Biology High School Chemistry High School Physics 85 College Biology College Chemistry 80 College Physics 75 none Reg AP Reg none Reg AP Reg none Reg AP Reg only only & AP only only & AP only only & AP MSP Regional Conference March 30-31, 2006 College Performance in Biology based on high school coursework College Grade 90 High School Biology High School Chemistry High School Physics 85 College Biology 80 75 none Reg AP Reg none Reg AP Reg none Reg AP Reg only only & AP only only & AP only only & AP MSP Regional Conference March 30-31, 2006 College Performance in Biology and Chemistry Based on Amount of HS Coursework College Grade 90 High School Biology High School Chemistry High School Physics 85 College Biology 80 College Chemistry 75 none Reg AP Reg none Reg AP Reg none Reg AP Reg only only & AP only only & AP only only & AP MSP Regional Conference March 30-31, 2006 The Advanced Placement Program • AP began as a way for “exceptional students” at elite private schools: – – – To take rigorous courses in HS No planned impact on college admissions (1952) No planned impact on GPA MSP Regional Conference March 30-31, 2006 The Advanced Placement Program • • AP began as a way for “exceptional students” at elite private schools: – To take rigorous courses in HS – No planned impact on college admissions (1952) – No planned impact on GPA Expanded to >2.1M exams/yr in 35 subjects MSP Regional Conference March 30-31, 2006 The Advanced Placement Program AP began as a way for “exceptional students” at elite private schools: • – – – To take rigorous courses in HS No planned impact on college admissions (1952) No planned impact on GPA Expanded to >2.1M exams/yr in 35 subjects Benefits to the student (other than learning): – – – – – Higher HS Grade Point Average Taking college courses in High School High probability of getting into college and financial aid Higher college grades if repeated College credit (advanced standing), cost savings • • MSP Regional Conference March 30-31, 2006 What the public hears “It is better to take a tougher course and get a low grade than to take an easy course and get a high grade.” Clifford Adelman, Senior Research Analyst, U.S. Dept. of Ed. MSP Regional Conference March 30-31, 2006 Our Research Questions For students taking introductory college biology, chemistry, and physics: • What grades do students earn based on high school AP performance? • What is the predicted advantage taking AP when controlling for student background, preparation, and SES? MSP Regional Conference March 30-31, 2006 College Performance in Introductory Science Courses 100 90 wit h controls raw scores 85 80 5 4 3 2 1 ex am AP . N o no AP AP o ho no rs , re gu la r, n ta ke n 75 no t College Science 95 MSP Regional Conference March 30-31, 2006 Covariates with AP Score: The need for regression models to model the unique contribution of AP courses. 90 80 Highest Science Course Type Honors or Regular Course Grade SAT Mat h Score MSP Regional Conference Highest Math Course Level Highest Math CourseGrade AP 5 AP 4 AP 3 AP 2 AP 1 A B C D Calculus BC Calculus AB Calculus precalculus ŠAlgebra II 750 650 550 450 350 250 A B C D AP honors regular 70 not taken College Science 100 AP Exam Score March 30-31, 2006 Modeling the Impact of AP Courses After controlling for covariates 100 90 wit h controls raw scores 85 80 5 4 3 2 1 ex am AP . N o no AP AP o ho no rs , re gu la r, n ta ke n 75 no t College Science 95 MSP Regional Conference March 30-31, 2006 Modeling the Impact of AP Courses After controlling for covariates 100 90 wit h controls raw scores 85 80 5 4 3 2 1 ex am AP . N o no AP AP o ho no rs , re gu la r, n ta ke n 75 no t College Science 95 MSP Regional Conference March 30-31, 2006 Modeling the Impact of AP Courses Students who do not take the exam perform at the same level as those earning a 3 100 90 wit h controls raw scores 85 80 5 4 3 2 1 ex am AP . N o no AP AP o ho no rs , re gu la r, n ta ke n 75 no t College Science 95 MSP Regional Conference March 30-31, 2006 Earning a “5” predicts increasing college grade by 5 points over honors 100 90 wit h controls raw scores 85 80 5 4 3 2 1 ex am AP . N o no AP AP o ho no rs , re gu la r, n ta ke n 75 no t College Science 95 MSP Regional Conference March 30-31, 2006 Earning a “4” predicts increasing college grade by 4 points over honors 100 90 wit h controls raw scores 85 80 5 4 3 2 1 ex am AP . N o no AP AP o ho no rs , re gu la r, n ta ke n 75 no t College Science 95 MSP Regional Conference March 30-31, 2006 Modeling the Impact of AP Courses 100 90 wit h controls raw scores 85 80 5 4 3 2 1 ex am AP . N o no AP AP o ho no rs , re gu la r, n ta ke n 75 no t College Science 95 MSP Regional Conference March 30-31, 2006 Conclusions 92 Conclusions • AP students do earn somewhat higher grades in college science – Partial proxy for demographic, general scholastic performance, math preparation – and performance in high school science courses that are prerequisites to AP in most schools MSP Regional Conference March 30-31, 2006 Conclusions • AP students do earn somewhat higher grades in college science – Partial proxy for demographic, general scholastic performance, math preparation – and performance in high school science courses that are prerequisites to AP in most schools • Course order is unimportant MSP Regional Conference March 30-31, 2006 Conclusions • AP students do earn somewhat higher grades in college science – Partial proxy for demographic, general scholastic performance, math preparation – and performance in high school science courses that are prerequisites to AP in most schools • • Course order is unimportant, amount is The best preparation comes from HS courses that: – Use lots of math – Concentrate on key concepts, not coverage – Use labs judiciously to change misconceptions MSP Regional Conference March 30-31, 2006 What do we know now? Misconceptions often unchanged after taking science. 1. Necessary step in learning The standards are hard to master. Teachers are knowledgeable, but does not assure student learning. Teachers do not know their students’ misconceptions, but should. Teacher knowledge builds slowly. Professional development must be 2. 3. 4. 5. • • 6. targeted to specific standards at grade levels evaluated with relevant tools. AP courses help the most if they focus on quantitative science, conceptual labs, fundamentals. MSP Regional Conference March 30-31, 2006 Acknowledgments • Co-investigators: – Robert Tai, University of Virginia, Matthew Schneps, • • Survey Staff: Video Staff: – Yael Bowman, Toby McElheny, Nancy Finkelstein, Alexia Prichard, Alex Griswold • Graduate Students: – Zahra Hazari, John Loehr MSP Regional Conference Board of Advisors – – – – – Jamie Miller, Nancy Cook Smith, Cynthia Crockett, Marc Schwartz (McGill), Annette Trenga, Bruce Ward • Advice – NSF: Janice Earle, Barry Sloane, Elizabeth VanderPutten, Larry Suter Project Managers: – Hal Coyle, Michael Filisky • • • Joel Mintzes, Mary Atwater Brian Alters, Lillian McDermott Eric Mazur, James Wandersee Dudley Herschbach Financial Support – NSF – Annenberg/CPB • – DoEd – NIH Center for Astrophysics – Irwin Shapiro, Judith Peritz. March 30-31, 2006 Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation, National Institutes of Health, U.S. Department of Education 98 Harvard-Smithsonian Center for Astrophysics Science Education Department 60 Garden Street, MS-71 Cambridge, MA 02138 Phone: 617-496-7598 Fax: 617-496-5405 Email: [email protected] MSP Regional Conference March 30-31, 2006 Leo Tolstoy "I know that most men, including those at ease with problems of the greatest complexity, can seldom accept even the simplest and most obvious truth if it be such as would oblige them to admit the falsity of conclusions which they have delighted in explaining to colleagues, which they have proudly taught to others, and which they have woven, thread by thread, into the fabric of their lives." MSP Regional Conference March 30-31, 2006