Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Attribute hierarchy method wikipedia , lookup
Observational methods in psychology wikipedia , lookup
Music psychology wikipedia , lookup
Psychological evaluation wikipedia , lookup
Educational psychology wikipedia , lookup
Cognitive science wikipedia , lookup
Learning theory (education) wikipedia , lookup
Cognitive development wikipedia , lookup
Intercultural competence wikipedia , lookup
Knowing What Students Know Ganesh Padmanabhan 2/19/2004 Overview Nature of Assessment Need for Reform Advances in Cognition Advances in Measurement Role of Technology Assessment “Assessment is the means used to measure the outcomes of education and achievement of students with regard to important competencies.” Purpose determines priorities. Context of use imposes constraints on design. Purposes of Assessment Achievement vs Aptitude Formative vs Summative Evaluation of Educational Programs More and more high-stakes decisions One type does not fit all Need for better alignment Reasoning From Evidence “Assessment is a tool designed to observe students’ behavior and produce data that can be used to draw reasonable inferences about what the students know.” Also inferences about how, when, and whether they use what they know. Imprecise by-nature Assessment Triangle Cognition “…refers to a theory or set of beliefs about how students represent knowledge and develop competence in a subject domain.” Cognitive Theories Educational Theories Experience of Expert Teachers Need not be exhaustively complex Observation “Every assessment is also based on a set of beliefs about the kinds of tasks or situations that will prompt students to say, do, or create something that demonstrates important knowledge and skills.” Not arbitrary, needs careful design. Interpretation “…expresses how the observations derived from a set of assessment tasks constitute evidence about the knowledge and skills being assessed.” Methods for drawing inferences Classroom – usually qualitative Large-Scale – usually formal, statistical Making Sound Inferences Depends on making explicit connections in assessment triangle between – Cognition and Observation – Cognition and Interpretation – Observation and Interpretation – Interdependent and Iterative Development Advances in the Learning Sciences “How People Learn” by NRC (1999) Systematic investigation dates back to the late 19th Century Recently, very diverse fields converging “Cognitive Revolution” (1960s onward) Four significant perspectives: Differential, Behaviorist, Cognitive, and Situative Differential Perspective “The differential perspective focuses mainly on the nature of individual differences in what people know and in their potential for learning.” Early 1900s Bell Curve Assumptions Focus on aptitude not achievement Behaviorist Perspective Skills are thought to be composed of stimulus-response associations. Considers learning as the process by which one acquires those associations and assembles them into skills. Stimulus, reinforcement, conditioning theories etc. Cognitive Perspective “In cognitive theory, knowing means more than the accumulation of facts and knowledge; it means being able to integrate knowledge, skills, and procedures in ways that are useful for interpreting situations and solving problems.” Aids assessment in figuring how, when, and whether students use what they know. Situative Perspective Sociocultural View Places individual thinking within a practical context Authentic Assessment Needs more here. Interpreting Evidence Psychometrics Probabilistic Approach to Reasoning Cognition Construct Observation Observational Model Interpretation Measurement Model Student Construct Observations Generation of Data ө x Interpretation of Data “…statistical models can be developed to predict the probability that people will behave in certain ways in assessment situations, and that evidence derived from observing these behaviors can be used to draw inferences about students’ knowledge, skills, and strategies…” Classical Test Theory Construct ө Sum of Responses Observations Generalizability Theory Construct ө Observations Sum of Responses Raters Type of Task Item Response Modeling ө Construct Response 1 Response 2 Response 3 Response 4 Response 5 Observations δ1 δ2 δ3 δ4 δ5 Latent Class Models Fig 4-8 Multi-Attribute Models Fig 4-9 Role of Technology Theory-Based Item Generation Concept Organization Complex Problem Solving (e.g. IMMEX) Text Analysis and Scoring Portal computer-aided approach for design of assessment Questions References All quotes and figures were taken from the NRC Report “Knowing What Students Know” by Pellegrino et al. (2001)