Download Multinomial Processing Tree Models

Multinomial Processing Tree
Models
Agenda
• Questions?
• MPT model overview.
– MPT overview
– Parameters and flexibility.
– MPT & Evaluation
•
•
•
•
Batchelder & Riefer, 1980.
Assignment 1 solution(s).
Math for next week.
Assignment 2 problem statement.
Uses of MPT Models
• Data-analysis tool capable of
disentangling and measuring separate
contributions of different cognitive
processes.
– Provides a means for separately
measuring latent processes that are
confounded in observable data.
• Framework for developing and testing
quantitative theories.
Purview of MPTs
• Categorical data: each observation falls
into one and only one of a finite set of
categories.
– Example 1: Correct or incorrect.
– Example 2: Reaction time < 100, 100 ≥ RT
< 200, 200 ≤ RT.
Multinomial Distribution
• Consider a die with sides: 1, 1, 1, 2, 2, 3.
• If we roll the die 10 times, what is the
probability we get five 1’s, three 2’s, and
two 3’s?
10! 1  1  1 
     
5!3!2! 2  3  6 
5
3
2
Multinomial Distribution
• In general,
n!
n1
n2
n3
p1 p2  p3 
n1!n2!n3!
• Inverse goal is to determine the pi’s
given the ni’s.
Statistical vs Explanatory
• Multinomial models (log-linear, logit) are
statistical.
– The parameters are used to explore main
effects and interactions.
• MPT models are explanatory.
– The parameters of the MPT model
represent the underlying psychological
processes.
MPT Models
Process
State 1
Root
Response 1
Process
State 3
Response 2
Process
State 4
Response 1
Process
State 2
MPT Models
• The probability of each
process state change is
represented by a
parameter.
• The parameters range
from 0 to 1.
• The parameters are
independent.
• There are other
restrictions…
MPT Models
• The response
probabilities are
given by
polynomials, e.g.,
P(E1)=c·r.
• The parameter
estimators (and
other important
properties) are easy
to find.
Estimators
• A parameter is a descriptive measure in a model
– E.g., a is a parameter the describes how quickly the line
increases in y=ax+b.
• An estimator is a function on the data that gives a
parameter estimate, usually denoted with a hat, â.
– E.g., given some data, get an estimate of how quickly the
linear trend increases.
• Estimators are usually picked to minimize the
discrepancy between the predictions and the data.
Parameters and Flexibility
• As a rule of thumb, a model should
have fewer parameters than degrees of
freedom in the data.
• This model is saturated.
p1
C1
Cond Data
Model
p2
C2
p3
C3
1
2
3
.2
.4
.9
.2
.4
.9
Parameters and Flexibility
• In general, the more parameters, the
more flexible the model.
y=b
y=ax+b
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
But…
1 parameter
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
2 parameters
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Further…
• It is not always easy to count
parameters.
y  ax  b
y  d  ex  f
g
2
Parameters and Flexibility
Possible
Events
• A more restricted a
model is
– usually simpler.
– usually easier to
interpret.
– usually more
falsifiable.
Observed
Events
Model 1
Model 2

Nested Models
Model 1 : y  ax  b
Model 2 : y  ax
Model 3 : y  b
•Models 2 and 3 are submodels (nested in) Model 1.
•Models 2 are 3 are NOT nested.
•The main benefit of nested models is that it makes it
relatively easy to compare the goodness-of-fit of the
two models.
Parameters and Flexibility
• What is important is the flexibility of the
model relative to
– the data.
– competing models.
Identifiability
• The parameters in a model are
identifiable if there is a unique set of
parameters that give rise to the model
predictions.
vs
• Identifiability is especially desirable if
the parameter values are to be
interpreted.
Batchelder and Modeling
“The assumption is … clearly an
approximation, but one that greatly
simplifies the analysis of the model and
still allows the model to reflect the main
processing stages of the task” (p. 59).
Batchelder and Modeling
“…there is usually a large number of
parameters used to account for a small
number of categories, leaving few, if any,
degrees of freedom for testing the model’s
fit…However, it is the measurement of the
cognitive processes in the form of parameter
estimates, and not the data-fitting capacity,
that characterizes the usefulness of MPT
models” (p. 81-82).
Batchelder and Modeling
“… there will often be psychologically
uniterpretable MPT models that nevertheless
fit a given set of data well. Thus, the process
of developing a valid model requires that one
fit a number of data sets in the same
paradigm and that the resulting parameter
estimates be interpretable in terms of the
underlying processing assumptions” (p. 76).
Batchelder and Modeling
“Each MPT model is at best an
approximation to a complete process
description of categorical data, and the
task of the modeler is to select the most
important processes and capture them
in a valid way” (p. 75).
Batchelder and Modeling
“… the process of developing a valid
model requires that one fit a number of
data sets in the same paradigm and that
the resulting parameter estimates be
interpretable in terms of the underlying
processing assumptions” (p. 76).
Batchelder and Modeling
“…an even more crucial test of a model’s
validity is to show that the model
performs well under basic experimental
manipulations. If the model’s
parameters behave in a psychologically
interpretable fashion, then the model
gains credence as a valid measurement
tool” (p. 82).
Batchelder and Modeling
“…an acceptable MPT model must not
only be able to fit data but its
parameters must be globally
identifiable, must be psychologically
interpretable, and must pass
appropriate validation experiments” (p.
78).
Batchelder and Modeling
“…there are many models for categorical data
that are not in the MPT class. If one of these
models accounts for data in a a particular
paradigm, then, technically one can infer that
the MPT class is falsified in that paradigm. Of
course, it may be possible to design an MPT
model that closely mimics or approximates
the successful fits of the non-MPT model;
thus, it may be difficult to argue that the MPT
framework is falsifiable in practice” (p. 78).