Download this - Athabasca Landing

+
Namsook Jahng
&
Wendy Nielsen
CIDER Session, January 11, 2012
1
+ Overview
 Study

Jahng, N., Nielsen, S. W., & Chan, E. K. H. (2010). Collaborative
learning in an online course: A comparison of communication
patterns in small and whole group activities. Journal of Distance
Education, 24 (2), 39-58.
 Study

I
II
Jahng, N. (2010) Examining collaborative learning in an online
course. (Doctoral dissertation)
2
+
3

Benefits of collaborative learning (Smith, 2008; Wang, 2010;
Walton & Baker, 2009)

Difficulties in experiencing successful collaboration in online
small groups (Ferreira & Santos, 2009; Fisher, 1970; Tubbs,
1995)

The key is instructor’s role

How to monitor and evaluate collaboration to provide timely
support and facilitation (Zhang, Peng, & Hung, 2009)

Importance of teaching presence (Garrison, Anderson, & Archer,
2000).
+
4

Learner Participation


An intrinsic component of learning (Wenger, 1998)
Hrastinski (2008) A Literature Review

Diverse conceptions and approaches on evaluating learner
participation in research studies

Quantitative Measurement

Content Analysis Approach

Qualitative Evaluation
+
5
Assessing Participation
 Quantitative
Analysis Approach (Frequency counts)

e.g., number of access, postings, duration, log-ins, etc.
 Active participation: writing/posting messages, uploading
information,
 Passive participation: reading messages, downloading files

Strength: system-based monitoring, objective
Weakness: incomplete measure; cannot measure quality of
participation

+
6
Assessing Participation

Content Analysis Approach


Qualitative coding

categorize contents of communication into units and count the units
for analysis quantitatively

evaluate the quality of communication units
Strength:


a useful method to assess and diagnose group collaboration
processes (Henri, 1992)
Weakness:

a technique more suited for researchers than for instructors
(Penney & Murphy, 2005)

difficult, frustrating, and time-consuming (Rourke, Anderson
Garrison, & Archer, 2001)
+
7
Assessing Participation
 Qualitative Analysis Approach

Read and analyze all the communications in-depth to
identify/diagnose problems in collaboration processes

Strength:
 Examine complexity of collaboration processes
Weakness:
 Too much time with many groups, many students
 Subjective decision: when and how to intervene problem
groups from trivial conflicts to critical troubles

+
8
1.
Propose quantitative indices for assessing
collaborative learning processes
1.
Demonstrate the evidences and examples how the
indices reveal problems in small groups
1.
Provide instructors with a theoretical model for
assisting collaboration processes
+
9
Key Terms

Ill-structured project-based group activity



Can be designed in a continuum from well-structure to illstructured (Jonassen, 1997)
Less detailed guidelines, constrained rules (Ferreira &
Lacerda Santos , 2009)
More open-ended, learner-centred, complex collaboration
+
10
Key Terms

Collaborative Learning






Differentiate from cooperative learning
More learner centered and less structured (Alavi & Dufner,
2005)
Seek a solution as a group (Stacy, 1999)
Should go beyond a simple “divide-and-conquer” approach
(Graham & Misanchuk, 2004; Stacy, 1999)
a more complex process working together (Ingram & Hathorn,
2004)
More or Less Collaborative Groups



Comparative evaluation among groups
From barely/least collaborative to highly/most collaborative
Identify problems in barely/least collaborative groups
+
11
Key Terms
 Participation
“Online learner participation is a process of
learning by taking part and maintaining
relations with others. It is a complex process
comprising doing, communicating, thinking,
feeling and belonging…” (Hrastinski, 2008, p.
1761)
+
12
Participation Indices
Conceptual Dimensions









Behaviour of Communication
Quantity
Equality
Sharedness
Content of Communication
Cognitive
Social
Managerial/Procedural
+
13
Quantity
1.

How actively a group communicates

Averaged number of words exchanged among members in a group

More collaborative, higher quantity of communication
Group A
Example
Participation
Behaviour
100
300
A1
B1
A2
A3
100
Quantity
Group B
100
100
Group D
100
10
C1
B2
B3
0
0
100
Group C
D1
C2
100
C3
100
100
D2
10
D3
10
10
(least
collaborative)
+
14
Equality
2.

Whether members’ participation is balanced

Individual members’ participation variability (variation coefficient)

More collaborative, less variability in contribution
Group A
Example
Group B
100
300
A1
B1
Group C
Group D
100
10
C1
D1
Participation
Behaviour
A2
100
Equality
A3
100
Perfect (0.0)
B2
B3
0
0
None (3.0)
(least
collaborative)
C2
100
C3
100
Perfect (0.0)
D2
10
D3
10
Perfect (0.0)
+
15
Sharedness
3.

How a group is connected in sharing communication

Shared communication (%) with all the group members (one-to-group
communication)

More collaborative, higher percentage of shared words in a group
Group A
Group B
100
300
A1
B1
Group C
Group D
100
10
C1
D1
Participation
Behaviour
A2
100
Sharedness
A3
100
All (100%)
B2
B3
0
0
All (100%)
C2
100
C3
100
None (0%)
(least
collaborative)
D2
10
D3
10
All (100%)
+
16
Group A: No potential problems indicated; Three indices are satisfied.
Group B: Equality problem (B2, B3 do not participate)
Group C: Sharedness problem (C1 and C3 do not communicate)
Group D: Quantity problem (Very little communication overall)
Group A
Example
Group B
100
300
A1
B1
Group C
Group D
100
10
C1
D1
Participation
Behaviour
A2
100
Potential
Problem
A3
100
None
B2
B3
0
0
Equality
C2
100
C3
100
Sharedness
D2
10
D3
10
Quantity
+ Contents of Communication
Communicatio
n category
Definition
Examples
Cognitive
Statements directly
related to on-task
content
•Sharing knowledge
•Comparing information or facts
•Brainstorming, questioning, refining, elaborating ideas
•Suggested ideas with real life examples
•Evaluating by agreeing or disagreeing
•Integrating and synthesizing conflicting opinions
Social
Statements to build up
friendship and group
membership
•Salutation, greetings, calling names, conventional thanks
•Openness, self-introductions, sharing personal feelings
•Humour, jokes
•Encouragement, compliments
•Off-task information, general information which is not
directly related to the course content
Managerial
Statements to coordinate
the group procedure
and make strategies to
complete the task
•Scheduling
•Dividing jobs
•Arranging meetings
•Clarifying ambiguities
•Assisting members’ technological problems
•Discussing group strategies
* adopted from the works by Garrison et al. (2000), Garrison, Anderson, & Archer
(2001), Henri (1992), Hara, et al. (2000), Newman, Webb and Cochrane (1995)
17
+
18
Theoretical Framework

Constructivism, Situated Cognition

Knowledge is distributed in environments of participation in
which the learner practices the patterns of inquiry and
learning (Dabbagh, 2005)

Learners should actively participate to negotiate meaning,
being collectively engaged in problem solving processes
(Bereiter & Scardamalia, 2003)

Instructor acts as a facilitator, mentor, guide, etc.

Groups develop across the timeline for their projects (Johson
et al., 2002; Gersick, 1988)
+
19
Small Group Collaborative Learning
Model (SGCLM)

Community of Inquiry (CoI) (Garrison et al., 2000)

Input-Process-Output (IPO) model (McGrath, 1964, 1984

Online Interaction Learning Model (Benbunan-Fich, Hiltz, &
Harasim, 2005)
+
Small Group Collaborative Learning Model (SGCLM)
Project-based Small Group Activity
Students’ collaboration process
cognitive
social
managerial
20
+
Small Group Collaborative Learning Model (SGCLM)
Project-based Small Group Activity
Students’ collaboration process
Equality
Sharedness
Students’
cognitive
Students’
social
Students’
managerial
Quantity
21
+
Small Group Collaborative Learning Model (SGCLM)
Project-based Small Group Activity
Teacher’s facilitation process
• direct instruction (cognitive)
• instructional management (managerial)
•Building understanding (social)
Students’ collaboration process
Equality
Sharedness
Students’
cognitive
Students’
social
Students’
managerial
Quantity
During small group
22
+
Small Group Collaborative Learning Model (SGCLM)
Whole Group Activity
Project-based Small Group Activity
Teacher’s facilitation process
• direct instruction (cognitive)
• instructional management (managerial)
•Building understanding (social)
Students’ collaboration process
Equality
Sharedness
Students’
cognitive
Students’
social
Students’
managerial
Quantity
During small group
23
+
Small Group Collaborative Learning Model (SGCLM)
Whole Group Activity
24
Project-based Small Group Activity
Teacher’s facilitation process
• direct instruction (cognitive)
• instructional management (managerial)
•Building understanding (social)
Inputs
• Prior
knowledge
•Group task
(project)
•Instructional
design
•Members’
personality
•Pre-existing
membership
Students’ collaboration process
Equality
Sharedness
Students’
cognitive
Students’
social
Outcomes
•Constructed
knowledge
•Task product
•Satisfaction
•Perceived
learning
•Group marks
•Post-small
group
membership
Students’
managerial
Quantity
Before small group
During small group
After small group
+
25
Research Questions
1.
How do group participation indices (quantity,
equality, sharedness) reveal problems in
cognitive, social, and managerial
communication categories in small groups?
(Study I & II)
1.
Do more/less collaborative groups achieve
higher/lower group marks as assessed by the
three indices? (Study I)
+
26
Research Questions
3.
What are the specific hindering/facilitating
factors for collaboration in individual groups?
(Study II)
3.
How do groups maintain their communication
relationship in whole group setting? (Study II)
+
27
Methods
 Course




context & Data (Study I)
Educational technology course, WebCT Vista course in
2007
5 males 7 females; Four groups consisted of 3 members
Writing two group papers (15%, 35% of the final mark)
Data: 562 small group forum messages
+
28
Methods
 Course




context & Data (Study II)
Educational technology course, WebCT Vista course in
2008
12 males 12 females; Six groups consisted of 3-5
members
Writing a group paper (35% of the final mark)
Data: 732 small group forum messages
+
29
Methods
 Coding


reliability
Study I: Inter-coder agreement 89%
Study II: 87%, Cohen’s Kappa .79; substantial strength of
agreement (k=0.61-0.80) (Landis & Koch, 1977)
 Analytical


approaches
Study I: Quantitative analysis
Study II: Both quantitative and qualitative analyses
+
30
Results (Study I) Group collaboration
assessed by participation indices
Index
Quantity
Equality
Sharedness
Groups
1
2
3
4
1
2
3
4
1
2
3
4
Cognitive
5432
4721
8510
6548
545
518
334
800
38
63
91
91
Social
2731
4233
2187
4485
409
778
278
790
39
56
51
52
Managerial
4545
6150
5072
4298
1149 1456 717 1178
46
56
65
62
• Quantity: total exchanged words,
• Equality: group standard deviation of exchanged words,
• Sharedness: percent of the exchanged words to all other members of total exchanged words
+ Results (Study I) Group collaboration rankings in
communication categories
Participatio
n Index
Quantity
Equality
Sharedness
Communication Group
Category
1
Group
2
Group
3
Group
4
cognitive
3
4
1
2
social
3
2
4
1
managerial
3
1
2
4
cognitive
3
2
1
4
social
2
3
1
4
managerial
2
4
1
3
cognitive
4
3
1
1
social
4
1
2
2
managerial
4
3
1
2
31
+ Results (Study I) Group collaboration rankings and
group marks
Group
1
Group
2
Group
3
Group
4
Quantity
4
3
1
2
Equality
2
4
1
3
Sharedness
4
3
1
2
Group mark
3
3
1
2
Collaboration Measure
Participation
Index
Outcome
32
+
33
Limitations of Study I
 Small
sample: 4 groups
 Outcome

Not necessarily the outcome of genuine collaboration,
could be cooperative work or individual’s work
 Further


measure: group marks
study
Qualitative analysis to examine the quality of collaboration
and to identify hindering/facilitating factors
Use other types of outcomes
+ Results (Study II)
34
Group collaboration rankings
Index
Quantity
Equality
Sharedness
Category
Group A
Group B
Group C
Group D
Group E
Group F
cognitive
6
2
5
1
3
4
managerial
6
2
3
1
3
6
social
6
2
4
1
2
6
cognitive
6
2
1
3
5
4
managerial
6
3
2
1
4
5
social
3
3
1
2
6
5
cognitive
5
4
2
6
3
1
managerial
4
3
2
6
4
1
social
2
4
3
6
5
1
+
35
Results (Study II)
Qualitative evaluation
 Factors
hindered collaboration

Group A (Problem in Quantity, Equality)
 A late start
 A lack of social communication
 Little ice-breaking
 Individualistic approach (split-and-stitch up)

Group F (Problem in Quantity, Equality)
 Indecisiveness
 A lack of leadership
+
36
Results (Study II)
Qualitative evaluation
 Factors
hindered collaboration

Group D (Problem in Sharedness)
 Technology and the Internet problems
 Politeness issues, negative attitudes
 Serious personal conflict between two members

Group E (Problem in Equality)
 Group size
 Time-zone difference
 Preferences to communication tools
+
37
Results (Study II)
Qualitative evaluation
 Factors facilitated collaboration

Group B (no problems indicated)
 A lot of social statement
 Keeping a team spirit
 Making an earlier start

Group C (no problems indicated, a bit low in quantity)
 Keeping connected
 Frequent uses of synchronous tools (MSN, Vista chats)
 Group approaches rather than individualistic approaches
+ Results (Study II) Group membership in WG
25%
Group B
20%
15%
Group F
Group D
Group E
Group A
Group C
10%
5%
0%
Before
During
After
38
+ Results (Study II)
Method
Quantitative
Assessment
Qualitative
Assessment
Membership
development
39
Index/Indi
cators
Group A
Group B
Group C
Group D
Group E
Group F
Quantity
6
2
4
1
3
5
Equality
6
3
1
1
5
4
Sharedness
4
3
2
6
4
1
Hindering
or
Facilitatin
g Factors
Late start,
Little social
communicat
ion,
No-ice
breaking,
cooperative
approach
A lot of
social
communic
ation,
Synchrono
us
communic
ation,
intensive
communic
ation
Technolog
y, internet
problems,
personal
conflict
Overcome
technical
problems
Indecisiv
eness, no
leadershi
p,
decreased
improved
improved
decreased
maintained
decrease
d
Individualist
ic (splitand-stich
up)
Very
collaborati
ve,
successful
group
Efficiently
collaborat
e, good
collaborati
on
Troubled,
angry
throughout the
process
Okay
collaborati
on
Failed
collabora
tion, very
inactive
group
Communic
ation trend
Overall Assessment
+
40
Assessment
results by quantitative
indices (Quantity, Equality,
Sharedness) were consistent with:
 Group
marks
 Group membership development
 Qualitative analysis results
+
41
 How
to use the quantitative indices?
 How
to conduct a content analysis?
 Behaviours
of communication (quantity, equality,
sharedness) indicate collaboration problems
rather than the contents of communication
(cognitive, social, sharedness).
 No need to code for CA
+ Results (Study II) Group collaboration rankings
Index
Quantity
Equality
Sharedness
Category
Group A
Group B
Group C
Group D
Group E
Group F
cognitive
6
2
5
1
3
4
managerial
6
2
3
1
3
6
social
6
2
4
1
2
6
cognitive
6
2
1
3
5
4
managerial
6
3
2
1
4
5
social
3
3
1
2
6
5
cognitive
5
4
2
6
3
1
managerial
4
3
2
6
4
1
social
2
4
3
6
5
1
42
+
Participation rankings without coding
Index
Quantitative
43
Category
Group A
Group B
Group C
Group D
Group E
Group F
Quantity
6
2
4
1
3
5
Equality
6
3
1
1
5
4
Sharedness
4
3
2
6
4
1
Assessment
• Overall amount of communication (number of words) without
coding effectively revealed problems in groups.
• The three indices are independent, revealing different aspects of
group communication structure.
• System-based monitoring can be easily implemented.
+
44
 SGCL
model can be useful for instructors as well
as for researchers.

Monitoring group membership development in WG
setting
 Further
research experimenting with the indices is
suggested.
+
Small Group Collaborative Learning Model (SGCLM)
Whole Group Activity
45
Project-based Small Group Activity
Teacher’s facilitation process
• direct instruction (cognitive)
• instructional management (managerial)
•Building understanding (social)
Inputs
• Prior
knowledge
•Group task
(project)
•Instructional
design
•Members’
personality
•Pre-existing
membership
Students’ collaboration process
Equality
Sharedness
Students’
cognitive
Students’
social
Outcomes
•Constructed
knowledge
•Task product
•Satisfaction
•Perceived
learning
•Group marks
•Post-small
group
membership
Students’
managerial
Quantity
Before small group
During small group
After small group
+
46
Thank you very much!
Namsook Jahng
[email protected]
Wendy Nielsen
[email protected]
+
47
References

Alavi, M., & Dufner, D. (2005). Technology-mediated
collaborative learning: A research prospective. In S. R. Hiltz, & R.
Goldman (Ed.), Learning together online: Research asynchronous
learning networks (pp. 191-213). Mahwah, NJ: Erlbaum.

Benbunan-Fich, R., Hiltz, S. R., & Harasim, L. (2005). The online
interaction learning model: An integrated theoretical framework
for learning networks. In S. R. Hilts, & R. Goldman (Eds.),
Learning together online: Research on asynchronous learning
networks (pp. 19-37). New York: Lawrence Erlbaum Associates.

Bereiter, C., & Scardamalia, M. (2003). Knowledge building
environments: Extending the limits of the possible in education
and knowledge work. In: Di Stefano, A., Rudestam, K.E.,
Silverman, e R. (Eds.), Encyclopedia of Distributed Learning.
Thousand Oaks, CA, Sage Publications, 269–272.
+  Ferreira, D. J., & Lacerda Santos, G. (2009). Scaffolding Online
Discourse in Collaborative Ill-Structured Problem-Solving for
Innovation. Informatics in Education, 8(2), 173-190.

Fisher, B. A. (1970). Decision emergence: phases in group
decision-making. Communication Monographs, 37, 53–66.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical
inquiry in a text-based environment: Computer conferencing
in higher education. The Internet and Higher Education, 2(23), 87-105.

Garrison, D. R., Anderson, T., Archer, W. (2001). Critical
thinking, cognitive presence, and computer conferencing in
distance education. American Journal of Distance Education,
15(1).

Hara, N., Bonk, C. J., & Angeli, C. (2000). Content analyses of
on-line discussion in an applied educational psychology
course. Instructional Science, 28(2), 115-152.
48
+
49

Henri, F. (1992). Computer conferencing and content
analysis. In A. Kaye (Ed.), Collaborative learning through
computer conferencing: The Najaden papers (pp. 117-136).
Berlin: Springer-Verlag.

Johnson, S. D., Suriya, C., Yoon, S. W., Berrett, J. V., & LaFleur, J.
(2002). Team development and group processes of virtual
learning teams. Computers & Education, 39(4), 379-393.

Gersick, C. J. (1988). Team and transition in work teams:
toward a new model of group development. Academy of
Management Journal, 31, 9–41.

Jonassen, D. H. (1997). Instructional design models for wellstructured and ill-structured problem-solving learning
outcomes. Educational Technology: Research and
Development, 45(1), 65-94.
+  McGrath, J. E. (1964). Social psychology: A brief introduction.
New York: Holt, Rinehart & Winston.

McGrath J. E. (1984). Groups: Interaction and Performance.
Englewood Cliffs, NJ: Prentice-Hall.

Newman, D.R., Webb, B., & Cochrane, C. (1995). A content
analysis method to measure critical thinking in face-to-face
and computer supported group learning. Interpersonal
Computing and Technology: An Electronic Journal for the 21st
Century, 3(2), 56-77.

Penny, L. & Murphy, E. (2009). Rubrics for evaluating online
discussions. British Journal of Educational
Technology,40(5),804-820.

Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001).
Methodological issues in the content analysis of computer
conference transcripts. International Journal of Artificial
Intelligence in Education, 12(1), 8-22.
50
+
51

Smith, R. O. (2008). The paradox of trust in online collaborative groups.
Distance Education, 29(3), 325–34.

Tubbs, S. (1995). A systems approach to small group interaction. New
York: McGraw-Hill.

Walton, K.L.W., & Baker, J. C. (2009). Group projects as a method of
promoting student scientific communication and collaboration in a
public health microbiology course. Bioscience 35(2), 16-22.

Wang, Q. (2010). Using online shared workspaces to support group
collaborative learning. Computers & Education, 55(3). 1270-1276.
doi:10.1016/j.compedu.2010.05.023

Wenger, E. (1998) 'Communities of Practice. Learning as a social system',
Systems Thinker, retrieved September 25, 2010 from http://www.co-il.com/coil/knowledge-garden/cop/lss.shtml .

Zhang K, Peng, S. W., & Hung, J. (2009). Online collaborative learning in a
project-based learning environment in Taiwan: a case study on
undergraduate students' perspectives. Educational Media International,
46 (2), 123–135. DOI: 10.1080/09523980902933425