Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Educational psychology wikipedia , lookup
Classroom management wikipedia , lookup
Inquiry-based learning wikipedia , lookup
Learning theory (education) wikipedia , lookup
Learning disability wikipedia , lookup
Response to intervention wikipedia , lookup
Learning through play wikipedia , lookup
Project-based learning wikipedia , lookup
Cooperative learning wikipedia , lookup
Constructivist teaching methods wikipedia , lookup
Improving your classroom assessments Dr. Patty LeBlanc Southeastern University Agenda Introductions Objectives of the Seminar Purpose of Educational Assessments Educator and Educational Standards Bloom Test Blueprints Item Writing Commandments Item Critique Activity Item Analysis Example from grad students Authentic Assessment Rubrics Summary Action Plan Questions? Objective of the Seminar To help teachers design valid assessments of instruction that inform instruction What is the purpose of testing? Participant Ideas To encourage students and improve teaching To assess the extent to which students have met the intended learning outcomes, and to look for and evaluate unintended outcomes To recognize achievements and diagnose learning difficulties so that students learn to build on their strengths and overcome or cope with their weaknesses To refine instruction and learning experiences in order to improve both individual and class learning To help students set meaningful and realistic learning goals and assume responsibility for their own learning. To communicate meaningful information to students, parents, and school authorities about student learning To give realistic and helpful feedback about achievement, capabilities, behavior, attitudes, and dispositions To put teachers, students, and parents/guardians in touch with one another about progress over time To provide guidance for educational and vocational choices To report learning achievement to school authorities such as school boards and governments Florida Educator Accomplished Practices in Assessment The effective educator consistently: Analyzes and applies data from multiple assessments and measures to diagnose students’ learning needs, informs instruction based on those needs, and drives the learning process; Designs and aligns formative and summative assessments that match learning objectives and lead to mastery; Uses a variety of assessment tools to monitor student progress, achievement and learning gains; Modifies assessments and testing conditions to accommodate learning styles and varying levels of knowledge; Shares the importance and outcomes of student assessment data with the student and the student’s parent/caregiver(s); and Applies technology to organize and integrate assessment information. Educational Standards Florida Sunshine State Standards Common Core Standards Other “Backward” Design The standards dictate the lesson objectives… The objectives dictate the instruction The assessment must align with the objectives, standards, and instruction in order to be valid. Blessed AssurAnce… A fantastic way to ensure that your test items align with your objectives: A Test Blueprint! Content Validity Think matrix of instruction and assessment: Objective 1 2 Item 1 X Item 2 Item 3 Item 4 X Item 5 X X 3 4 X X Bloom’s Taxonomy of the Cognitive Domain Another blueprint… Item # Standard/Objective Knowledge/Comp Above Knowledge/Comp Comment Sung to the tune of the Farmer in the Dell The teacher follows the standards… The teacher creates the objectives… The teacher writes the test… The teacher teaches the lesson… The students learn a lot… The parents beam with pride… Comunicación y Gerencia The Commandments of Test ItemWriting Writing Item Commandments By W.James Popham Click to add Text General Item Writing Items Thou shalt not provide opaque directions to students regarding how to respond to your assessment instruments. Thou shalt not employ ambiguous or wordy statements in your assessment items. The term hypothesis, as used in research, as defined as: a. A conception or proposition formed by speculation or deduction or by abstraction and generalization from facts, explaining or relating an observed set of facts, given probability by experimental evidence or by factual or conceptual analysis but not conclusively established or accepted. b. A statement of an order or relation of phenomena that so far as is known is invariable under the given conditions, formulated on the basis of conclusive evidence or tests and universally accepted, that has been tested and proven to conform to facts. *c. A proposition tentatively assumed in order to draw out its logical or empirical consequences and so test its accord with facts that are known or may be determined, of such a nature as to be either proved or disproved by comparison with observed or know facts. Better… The term hypothesis, as used in research, is defined as: a. An assertion explaining an observed set of facts that has not been conclusively established. b. A universally accepted assertion explaining an observed set of facts. *c. A tentative assertion that is either proved or disproved by comparison with an observed set of facts. Thou shalt not Provide unintentional clues as to the correct response. What meter is used to measure resistance? a. Voltmeter b. Ohmmeter c. Ammeter d. Oscilloscope Thou shalt not employ complex or give-away syntax in your assessment items. In item response theory, the one-parameter model assumes that each item ……. a. discriminates equally well. b. students perform equally well. c. students score the same across items. d. guessing affects all items the same. Thou shalt not use vocabulary that is more advanced than required. As the level of fertility approaches its nadir, what is the most likely ramification for the citizenry of a developing nation? a decrease in the labor force participation rate of women a dispersing effect on population concentration a downward trend in the youth dependency ratio a broader base in the population pyramid an increased infant mortality rate Thou shalt Use plausible distractors Which of the following artists is known for painting the ceiling of the Sistine Chapel? a. Warhol b. Flintstone *c. Michelangelo d. Santa Claus Thou Shalt Avoid using "always" and "never" in the stem or the distractors. Thou shalt Rarely us negative statements… Which of the following is NOT a symptom of osteoporosis? decreased bone density frequent bone fractures raised body temperature lower back pain Binary Choice Items [True/False] One of several types of Selected Response Items Thou shalt not Phrase items so that a superficial analysis by the student suggests a wrong answer. Which of the following artists is known for painting the ceiling of the Sistine Chapel? a. Warhol b. Flintstone *c. Michelangelo d. Santa Claus Thou shalt NEVER use double negatives. True or False: There is no advantage to not using specific determiners in true-false items. Thou shalt Have an approximately equal number of items representing the two categories being tested. Keep item length similar for both categories being tested. Thou Shalt Stick with absolute truths or falsehoods. True or False: Nearsightedness is hereditary in origin. Thou Shalt Not Use extreme qualifiers. All sessions of Congress are called by the President. (F) The Supreme Court frequently rules on the constitutionality of law. (T) An objective test is generally easier to score than an essay test. (T) Multiple Choice Items Thou shalt Write the stem so that it consists of a self-contained question or problem. California: a. Contains the tallest mountain in the United States b. Has an eagle on its state flag. c. Is the second largest state in terms of area. *d. Was the location of the Gold Rush of 1849. Better… What is the main reason so many people moved to California in 1849? a. California land was fertile, plentiful, and inexpensive. *b. Gold was discovered in central California c. The east was preparing for a civil war. d. They wanted to establish religious settlements. Thou shalt Include only one concept in each statement and only one correct answer. The United States should adopt a foreign policy based on: a. A strong army and control of the North American continent. b. Achieving the best interest of all nations. c. Isolation from international affairs. *d. Naval supremacy and undisputed control of the world’s sea lanes. Thou shalt NEVER use ‘all of the above’ alternatives, only occasionally use ‘none of the above’ alternatives [to increase item difficulty]. Alternative Weakness “All of the above” Distractor can be eliminated by noting that one of the other alternatives is incorrect “None of the above” Answer measures the ability to recognize incorrect answers rather than correct answers [remember educational psychology 101?]. Distractor does not appear plausible to some students. Well-designed MC items CAN measure HOTS A common goal of the Salt March in India, the Boxer Rebellion in China, and the Zulu resistance in southern Africa was to: A. overthrow totalitarian leaders B. force upper classes to carry out land reform programs C. remove foreign powers D. establish Communist parties to lead the countries Numbers of Items is Important! Number of 4-Alternative MC Items 2 5 10 15 20 25 Chance of Scoring 70% or Higher by Blind Guessing • • • • • • 1 out of 16 1 out of 64 1 out of 285 1 out of 8,670 1 out of 33,885 1 out of 942,651 Pop Quiz Can you identify the problem with this item? Due to budget cutbacks, the university library now subscribes to fewer than _?_ periodicals. 25,000 20,000 15,000 10,000 Matching Items Most effective when you need to measure the learner’s ability to identify the relationship or association between similar items. They work best when the course content has many parallel concepts, for example: Terms and Definitions Objects or Pictures and Labels Symbols and Proper Names Causes and Effects Scenarios and Responses Principles and Scenarios to which they apply Thou shalt Use homogeneous lists. Hitler Mulroney Tank 1939 A. Year in which WWII began B. A Canadian Prime Minister C. A German dictator during WWII D. An armored vehicle used originally to break the trench war stalement in WWI Thou shalt Randomly assign correct answers to alternative positions. Logical order of responses Example Numerical a. 1939 b. 1940 c. 1941 d. 1942 Alphabetical a. Changing a from .01 to .05. b. Decreasing the degrees of freedom. c. Increasing the spread of the exam scores. d. Reducing the size of the treatment effect. Sequential a. Heating ice from -100°C to 0°C. b. Melting ice at 0°C. c. Heating water from 0°C to 100°C. d. Evaporating water at 100°C. e. Heating steam from 100°C to 200°C. Thou shalt not let the length of alternatives supply unintended clues. Which of the following is the best indication of high morale in a supervisor’s unit? a. The employees are rarely required to work overtime. *b. The employees are willing to give first priority to attaining group objectives, subordinating any personal desires they may have. c. The supervisor enjoys staying late to plan the next day. d. The unit gives expensive birthday presents to each other. Thou shalt Use relatively brief lists, placing the shorter words or phrases at the right. Thou shalt Employ more responses than premises. Thou Shalt Provide good directions! Directions: Match the following. ____1. City dwellers A. Wild animals ____2. Hunter-gatherers B. Farm ____3. Pastoral nomads C. Apartments D. Graze animals Better… Directions: On the line to the left of each definition, write the letter of the term in the right hand column that is defined. Use each term only once. There are more responses than items. Each correct response is worth 2 points. _1. Live in areas of high population density. A. Pastoral nomads __2. Move from place to place in search of wild animals. __3. Move from one place to another with grazing animals. __4. Till land for cash crops. B. Ranchers C. Hunter-gatherers D. City dwellers E. Farmers Thou shalt Describe the basis for matching and the number of times responses may be used in the instructions, as well as the point values for each item. Place all premises and responses for an item on a single page. Tip: Be sure to decide ahead of time whether you will give partial credit. Matching HOTS item Directions: Number (1-8) the following events in the history of ancient Egypt in the order in which they occurred, using 1 for the earliest event. _____Egypt divided; ruled by Libyan kings, Nubian pharaohs, Assyrians, and Persians _____Seizure of power by Hyksos kings _____Upper and Lower Egypt are united by Menes _____Alexander the Great conquers Egypt _____Reunification of Egypt under pharaoh Mentuhotop II _____Rise of feudal lords leads to anarchy _____Thutmose III expands empire to the Euphrates _____Many kings with short reigns; social and political chaos Short Answer/Essay Items Aka Constructed Response Items Short Answer/Fill in the Blanks Thou Shalt Ask a direct question. Undesirable: Alloys are ordinarily produced by… Desirable: How are alloys ordinarily produced? Thou Shalt Put a blank at the end of a sentence rather than the beginning. Undesirable: ____________ is the measure of central tendency that is most affected by extremely high or low scores. Desirable: The measure of central tendency that is most affected by extremely high or low scores is the ____________. Thou Shalt Omit only significant words. Undesirable: Every atom has a central ____________ called a nucleus. Desirable: Every atom has a central core called a(n) ____________. Thou Shalt Not Omit so many words that the meaning in unclear. Undesirable: The ____________ were to Egypt as the ____________ were to Persia and as ____________ were to the early tribes of Israel. Desirable: The Pharaohs were to Egypt as the ____________ were to Persia and as ____________ were to the early tribes of Israel. Thou Shalt Not Use essay items to measure trivial ideas Thou Shalt Give very clear directions. Know the criteria for grading and use a rubric. Make expectations clear—have a good idea of what the teacher expects in a good response Not provide options… Indicate point values and suggested time limit per item Take the test yourself and see how you do… Ask a colleague to take the test and critique it. Consider using several short essays vs 1 long one. Analyze student responses to improve the item in future. How do I decide??? Another Pop Quiz Objective or Essay? Rubrics for Essay Items How can a classroom teacher improve his/her tests? Before the test: During the test: Table of specifications/matrix Pilot testing, especially essay items and rubrics Peer review Observation Timing Ask the students to rate the test/items After the test: Item Analysis Go over the test with students Item and Test Revision Re-examine instruction and re-teach if necessary That’s a lot of trouble! Yes, but it is the right thing to do. It’s intellectually honest. It makes grades easier to defend. Test Improvement Project By Christina Salas and Amanda Williams Authentic Assessment Aka performance based assessment Great example: Edutopia Authentic Assessment vs. Traditional Assessment Traditional --------------------------------------------- Authentic Selecting a Response ------------------------------------ Performing a Task Contrived --------------------------------------------------------------- Real-life Recall/Recognition ------------------------------- Construction/Application Teacher-structured ------------------------------------- Student-structured Indirect Evidence -------------------------------------------- Direct Evidence How to Measure Authentic Assessments Rubrics Cooperatively Panel of Experts Steps in Rubric Development 1. 2. 3. 4. 5. Determine the learning outcomes. Determine the criteria Determine measurable criteria [4,3,2,1] Develop a matrix Evaluate overall point value relative to the task. Grading cooperative work… Participation in Class Discussions Sit down with a seating chart and mark contributions to the discussion with a rating system For example, 5 being outstanding and 1 being poor, 0 being absent or silent Grading cooperative work Group grade Recognize that there will always be loafers, workers, and compulsive worker bees See article on grading cooperative projects Assessment should… Encourage and improve student learning Be a blessing for students that embraces, grace, justice, and encouragement Be an integral part of learning, closely related to overall learning goals Foster excellence in all a students’ gifts on the basis of the student’s potential as a bearer of God’s image Emphasize assessment FOR learning as well as assessment OF learning Develop plans that enable teachers and students to interact frequently about students’ progress as well as challenges, with opportunities to refine or revise Use a variety of meaningful tasks, products, and performances, with descriptive feedback to students Use accurate descriptors for all grades based on the achievement of learning outcomes Communicate meaningful and clear information to students, parents, and authorities. Other suggestions? Questions? Resources Van Brummelen, H. 2009. Walking with God in the Classroom, 3rd edition. Purposeful Design Publicatons. Brookhart, S. 2008. Feedback that Fits. Educational Leadership 65, no.4:54-59. Stiggins, R. 1997. Student-centered classroom assessment. 2nd edition. Merrill. More Resources How to prepare effective essay questions http://www.uwgb.edu/oira/teachLearn/bet terTests/betteressays.pdf Steven J. Burton, et al, How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty, (http://testing.byu.edu/info/handbooks/be tteritems.pdf). Still More Resources A Short Guide to Writing Effective Test Questions Rubric Maker Quiz Star Thank You!!! To download this presentation: http://leblancspace.wikispaces.com/home [email protected]