Download CHIU_-Social_Metacognition

Effects of Social Metacognition on Micro-Creativity:
Statistical Discourse Analyses
of Group Problem Solving
Ming Ming Chiu
State University of New York – Buffalo
[email protected]
I appreciate the research assistance of
Choi Yik Ting and Kuo Sze Wing
Solving problems & Micro-creativity
Under the Universal Texting plan, each text message
costs $.10. Budget Texting costs $.01 per text
message, but charges a monthly fee, $18.
1) How many text messages do you send each month?
2) Which company costs less for you?
3) How many texts should you send for the
Universal plan and the Budget plan
to cost the same?
Solving problems & Micro-creativity
• Difficult problem for students learning algebra
• To solve this problem, novice students create
new ideas and check/justify their utility
(micro-creativity processes).
• More micro-creativity processes
 Solve problem
• What group processes
 micro-creativity processes?
Micro-Creativity Processes
• Creativity processes
– Generate ideas
– Identify/Justify utility
( Sternberg & Lubart, 1999 )
• Big “C” creativity affects society
• Small “c” creativity affects person
( Gruber & Wallace, 1999 )
• Micro-c creativity processes occur at a
moment in time
( Chiu, 2008 )
What Affects Micro-creativity?
• Social Metacognition?
• Face / Rudeness?
Social Metacognition
Metacognition
Monitoring and control of one’s
knowledge and actions
( Flavell, 1971; Hacker, 1998 )
Social Metacognition
Group members’ monitoring and control
of one another’s knowledge and actions
( Chiu, in press)
Most individuals have poor metacognition.
( Hacker & Bol, 2004 )
Social Metacognition
Questions
 indicate knowledge gaps
 Identifies gap in someone’s understanding
 Motivates and points out a way to fill the gap
to create a new idea (+)
 Use old or new info to explain/justify (+)
(Coleman, 1998; Webb, Troper & Fall, 1995; DeLisi & Goldbeck, 1999 )
Disagree
 Identify obstacles
 Overcome via new ideas and/or justifications (+)
(Doise, Mugny & Perret-Clermont, 1975; Piaget, 1985)
Face / Rude
• Disagree Rudely
• Excessive Agreement
• Command !
Face / Rude
Face = Public Self-image
Disagree rudely (attack face)
vs. Disagree politely (save face)
( Brown & Levinson, 1987 )
“Ten times two hundred.”
Disagree Rudely
“No, you’re wrong, it’s one tenth times two hundred.”
 Previous speaker more likely to retaliate
 Emotional argument
 Reduce new ideas & justifications ()
 End cooperation
( Chiu & Khoo, 2003; Gottman & Krokoff, 1989 )
Face / Rude
Disagree politely
“if we want it in dollars,
we can multiply two hundred by one tenth.”
• “if” – Hypothetical distances error away
• No “you” – No direct blame
• “we” – Shared positioning & common cause
 Save previous speaker’s face
 Listen & understand obstacle
 Overcome via new ideas & justifications (+)
( Chiu & Khoo, 2003 )
Face / Rude
Agree too much
Concern for social relationship
 Reluctant to disagree with wrong ideas
 Fewer new ideas & justifications (–)
( Person, Kreuz, Zwaan, & Graesser, 1995; Tann, 1979; Tudge,1989 )
Face / Rude
Command !
 Demand implementation of an old idea
 Suggest that speaker has higher status than audience
 Ruder than question
 Threaten face
 Distract from problem solving
 Fewer new ideas & justifications (–)
(Brown & Levinson, 1987; Chiu,2008 )
Social Metacognition
Ask Questions (+)
Disagree (+)
Micro-creativity
processes
New ideas
Face / Rudeness
Politely Disagree (+)
Rudely Disagree (–)
Excessively Agree (–)
Command (–)
Control variables
Math grade
Peer Friendship
Gender, ethnicity, …
Group mean grade,
Group gender variance …
Justifications
Videotape Group Problem Solving
• 84 9th grade, average ability students in US city
– Work in 21 groups of 4
• Not friends
• Introducing 2 variable algebraic equations
– 1st day of group work
– No group work preparation
– Work on problem for 30 minutes
• Videotape & Transcripts
– Two RAs coded each student turn
– Krippendorf’s 
Content analysis
Jay: A hundred eighty dollars.
Ben: If we multiply by ten cents, don’t we get
a hundred and eighty cents?
• Ben
–
–
–
–
–
Disagrees politely
New information
Correct
Justifies
Question
Multi-dimensional Coding
Evaluation of the previous action
– Agree ( + ), Neutral ( n ), Ignore/New topic ( * ),
Disagree rudely (––), Disagree politely (–)
Knowledge content regarding problem
– New idea, Old idea, Null-content ( {} )
Validity
– Correct (  ), Wrong ( X ), Null-content ( {} )
Justification
– Justify ( J ), No justification ( [] ), Null-content ( {} )
Invitation to participate
– Command ( ! ), Question ( ? ), Statement ( _. )
Invitational Form Decision Tree
Minimize Number of Coding Decisions to  inter-coder reliability
• Minimize Depth of decision tree
• Put highly likely actions at the top
Do any of the clauses proscribe an action?
• Yes, code as command (imperative)
• No, is the subject the addressee?
– No, are any of the clauses in the form of a question?
• No, code as statement (declarative)
• Yes, code as question (interrogative)
– Yes, is the verb a modal?
• No, should the described action have been performed, but not done?
– Yes, code as a command
– No, code as a question
• Yes, Is it a Wh- question (who, what, where, why, when, how)?
– Yes, code as an question
– No, is the action feasible?
• Yes, code as a command
• No, code as an question
Based on Labov (2001), Tsui (1992)
Coded Transcript
ID
Fay
Action
Do ten times eighteen.
Ben
Ten times eighteen is–
+
R

_.
Eva
Twenty-eight.
+
C
X
_.
Jay
Wrong. A hundred eighty dollars.
—
C
X
_.
Ben
If we multiply by ten cents, don’t
we get a hundred and eighty
cents?
Yep.
-
C

Fay
EPA KC Valid? Justify IF
*
C
!

+

Add other variables at each speaker turn:
Student: Gender, ethnicity, mid-year algebra grade, …
Group: Group’s mean mid-year algebra grade, …
J
?
_.
Statistical Discourse Analysis
4 types of Analytical Difficulties
• Time
• Outcomes
• Explanatory variables
• Data set
Statistical Discourse Analysis
Difficulties regarding Time
Strategies
 Time periods differ (T2  T4)  Breakpoint analysis
 Serial correlation (t8 → t9)
Identify Breakpoints
Breakpoints
• Critical events radically change interactions
• Statistically identify breakpoints
– Test possible combinations of breakpoints
– Model with smallest Bayesian Info Criterion (BIC)
 Explain the most variance w/ fewest breakpoints
Breakpoints in 1 group
% Micro-creativity
% New ideas
100%
80%
60%
40%
20%
0%
0
10
20
Time (mins)
30
Statistical Discourse Analysis
Difficulties regarding Time
Strategies
 Time periods differ (T2  T4)  Breakpoint analysis
 Multilevel analysis (MLn, HLM)
 Serial correlation (t8 → t9)
 Test with Q-statistics
 Model with lag outcomes
e.g. Justify (-1)
Statistical Discourse Analysis
Outcome Difficulties
Strategies
 Discrete outcomes (Yes / No)
 Logit / Probit
 Multiple outcomes (Y1, Y2)
New idea & Justify
 Multivariate, multilevel analysis
Statistical Discourse Analysis
Explanatory model Difficulties
 People & Groups differ
 Mediation effects (X→M→Y)
 False positives (+ + 
+ +)
 Effect across turns (X6→Y9)
Effects across several turns
Ben: 10 times 18 is
2 speakers ago = (– 2)
Eva: 28.
1 speaker ago = (– 1)
Jay: Wrong, 180 dollars.
Statistical Discourse Analysis
Explanatory model Difficulties
Strategies
 People & Groups differ
 Multilevel cross-classification
 Mediation effects (X→M→Y)
 Multilevel mediation tests
 False positives (+ + 
+ +)
 2-stage linear step-up procedure
 Effect across turns (X6→Y9)
 Vector Auto-Regression (VAR)
Lag explanatory variables
e.g., Disagree (-1), Girl (-1)
Disagree (-2)
VAR models effects across turns
ID
Fay
Action
Do ten times eighteen.
Justify
0
Disagree
0
Disagree (-1)
-
Ben
Ten times eighteen is–
0
0
0
Eva
Twenty-eight.
0
0
0
Jay
Wrong. A hundred eighty
dollars.
0
1
0
Ben
If we multiply by ten
cents, don’t we get a
hundred and eighty
cents?
1
1
1
Fay
Yep.
0
0
1
Statistical Discourse Analysis
Data Difficulties
Strategies
 Missing data (101?001?10)
 Markov Chain Monte Carlo
multiple imputation
 Robustness
 Separate outcome models
 Use data subsets
 Use unimputed data
Results: Breakpoints
• 2.65 new idea breakpoints per group
3.65 time periods per group (min=1; max =6)
• 2.05 justification breakpoints per group
3.05 time periods per group (min=1; max =6)
• Number of breakpoints did not differ across
groups that solved vs. did not solve the problem
3 Types of Breakpoints
• Creativity process generators
– Sharply increase new ideas or justifications
• Creativity process dampeners
– Sharply decrease new ideas or justifications
• On-task  Off-task transitions
Creativity generator
Ana
How can they be equal?
Bob
I don’t know
Cate
Try another number?
Dan
Which number?
[8 seconds of silence; each student looks at own paper]
Cate
[looks at Ana’s paper] Yours is much closer.
So, try a number close to yours
Dan
[looks at Ana’s paper] Mine’s even closer
Ana
[looks at Dan’s paper] Oh! More messages get us
closer
Creativity dampener
Kay Let’s try a hundred.
Lee Ok. That’s a thousand.
Tom And that’s one, so nineteen.
Kay That’s like over nine hundred away.
Jan Maybe it’s one of those trick questions.
Tom Yeah, like it can’t be done.
Kay So, maybe there’s no answer.
Lee Then, we’re done.
Explanatory model: New Idea & Justify
Previous turn (-1)
Rudely
Disagree (-1)
Current turn
Rudely Disagree
Outcomes
New Idea
Agree
Rudely Disagree
(-1) * Unsolved
Rudely Disagree
(-1) *Wrong (-2)
Command (-1)
Peer Friendship
Politely Disagree
Math grade (-1)
Math grade (-1)
*Unsolved
Justify
Group + Time Period Differences
Unsuccessful groups:
Negative effect of Rudely disagree (-1) on new ideas
Negative effect of Math grade (-1) on justifications
Mathematics grade’s effect on justifications
Differed across both time periods and across groups
-2% to +1% in unsuccessful groups
-1% to +3% in successful groups
Unsupported Hypotheses
Questions were not linked to
New idea or Justifications
Rudely disagreements
were not linked to Justifications
Implications for Teachers & Students
Increase Group Micro-creativity
• Ask questions rather than issue commands
!
• Disagree politely to encourage justifications
• Listen to rude disagreements and use the content to
develop new ideas
Implications for Researchers
• Statistically identify critical moments
(breakpoints) that radically change subsequent
processes
• Effects differ across groups, time periods, turns
– Use statistical model to compute specific effect
• Effects of sequences
– Look beyond the effects of single actions
• New method for statistically modeling
conversations
Further applications…
What major or momentary events affect
people’s behaviors over time during …
– Classroom conversations?
– Online discussions?
– A student’s think-aloud problem solving?
– An infant’s learning of a new word?
– Basketball games?
– Stock market transactions?
– Wars?
Thank you!
ID Action
Ana Do three times four.
Ben Three times four is seven
Eva Three times four is nine.
Jay Three times four is twelve.
ID
Ana
Ben
Eva
Jay
Action
Do three times four.
Three times four is seven
Three times four is nine.
Three times four is twelve.
Previous Valid
Turn # Valid?
Turn
(-1)
–
1

–
2
X
1

3
4
X

2
3
X
X
Respond Valid
Turn # Valid? to turn #? (-1)
–
1

–
2
X
1

3
4
X

1
3

X
Statistical Discourse Analysis
Analytical Difficulty
Strategy
 Differences across topics
 Multilevel analysis
 Time periods differ (T2  T4)
 Breakpoint analysis & Multilevel analysis
 Serial correlation (t8 → t9)
 I2 index of Q-statistics; Model with lag variables
 Parallel talk (→→ )
 Store path: ID prior turn, Vector Auto-Regression
 Discrete outcomes (Yes / No)
 Logit / Probit
 Multiple outcomes (Y1, Y2)
 Multivariate outcome models
 Infrequent outcomes (00010)
 Logit bias estimator
 People & Groups differ  Multilevel analysis
 Mediation effects (X→M→Y)  Multilevel mediation tests
 False positives (+ + 
+ +)
 2-stage linear step-up procedure
 Missing data (101?001?10)
 Markov Chain Monte Carlo multiple imputation
 Robustness
 Separate outcome models;
Data subsets & unimputed data
Knowledge content, Validity, and Justification
Does the speaker express any mathematics or problemrelated information?
• No, code as null content
• Yes, is all the info on the group's log/trace of problem
solving?
– Yes, code as repetition
– No, code as contribution and write specific info in
group's log
– Does this info violate any mathematics or problem
constraints?
• Yes, code as incorrect
• No, code as correct
– Does the speaker justify his or her idea?
• Yes, code as justification
• No, code as no justification
Mathematics
Bayesian Information Criterion
2L
 k ln( n ) 

 

n
 n 
Regression specification
ijk = F(0 + f0jk + g00k+ 00sS00k +00tT00k+ujkUijk
+ vjkV(i-1)jk +vjkV(i-2)jk +vjkV(i-3)jk +vjkV(i-4)jk)