Download Knowing What Students Know

Knowing What Students
Know
Ganesh Padmanabhan
2/19/2004
Overview





Nature of Assessment
Need for Reform
Advances in Cognition
Advances in Measurement
Role of Technology
Assessment



“Assessment is the means used to
measure the outcomes of education
and achievement of students with
regard to important competencies.”
Purpose determines priorities.
Context of use imposes constraints on
design.
Purposes of Assessment






Achievement vs Aptitude
Formative vs Summative
Evaluation of Educational Programs
More and more high-stakes decisions
One type does not fit all
Need for better alignment
Reasoning From Evidence



“Assessment is a tool designed to
observe students’ behavior and
produce data that can be used to draw
reasonable inferences about what the
students know.”
Also inferences about how, when, and
whether they use what they know.
Imprecise by-nature
Assessment Triangle
Cognition





“…refers to a theory or set of beliefs
about how students represent
knowledge and develop competence in
a subject domain.”
Cognitive Theories
Educational Theories
Experience of Expert Teachers
Need not be exhaustively complex
Observation


“Every assessment is also based on a
set of beliefs about the kinds of tasks
or situations that will prompt students
to say, do, or create something that
demonstrates important knowledge
and skills.”
Not arbitrary, needs careful design.
Interpretation




“…expresses how the observations
derived from a set of assessment tasks
constitute evidence about the
knowledge and skills being assessed.”
Methods for drawing inferences
Classroom – usually qualitative
Large-Scale – usually formal, statistical
Making Sound Inferences

Depends on making explicit
connections in assessment triangle
between
– Cognition and Observation
– Cognition and Interpretation
– Observation and Interpretation
– Interdependent and Iterative
Development
Advances in the Learning
Sciences





“How People Learn” by NRC (1999)
Systematic investigation dates back to the
late 19th Century
Recently, very diverse fields converging
“Cognitive Revolution” (1960s onward)
Four significant perspectives: Differential,
Behaviorist, Cognitive, and Situative
Differential Perspective




“The differential perspective focuses
mainly on the nature of individual
differences in what people know and
in their potential for learning.”
Early 1900s
Bell Curve Assumptions
Focus on aptitude not achievement
Behaviorist Perspective



Skills are thought to be composed of
stimulus-response associations.
Considers learning as the process by
which one acquires those associations
and assembles them into skills.
Stimulus, reinforcement, conditioning
theories etc.
Cognitive Perspective


“In cognitive theory, knowing means more
than the accumulation of facts and
knowledge; it means being able to integrate
knowledge, skills, and procedures in ways
that are useful for interpreting situations
and solving problems.”
Aids assessment in figuring how, when, and
whether students use what they know.
Situative Perspective




Sociocultural View
Places individual thinking within a
practical context
Authentic Assessment
Needs more here.
Interpreting Evidence





Psychometrics
Probabilistic Approach to Reasoning
Cognition  Construct
Observation  Observational Model
Interpretation  Measurement Model
Student
Construct
Observations
Generation of Data
ө
x
Interpretation of Data
“…statistical models can be developed to predict the
probability that people will behave in certain ways in
assessment situations, and that evidence derived
from observing these behaviors can be used to draw
inferences about students’ knowledge, skills, and
strategies…”
Classical Test Theory
Construct
ө
Sum of Responses
Observations
Generalizability Theory
Construct
ө
Observations
Sum of Responses
Raters
Type of Task
Item Response Modeling
ө
Construct
Response 1
Response 2
Response 3
Response 4
Response 5
Observations
δ1
δ2
δ3
δ4
δ5
Latent Class Models

Fig 4-8
Multi-Attribute Models

Fig 4-9
Role of Technology





Theory-Based Item Generation
Concept Organization
Complex Problem Solving (e.g.
IMMEX)
Text Analysis and Scoring
Portal computer-aided approach for
design of assessment
Questions
References
All quotes and figures were taken from
the NRC Report “Knowing What
Students Know” by Pellegrino et al.
(2001)